PDA

View Full Version : Solved Need help reducing server load



Simon Lloyd
26-05-11, 08:40
Hi, i'm having trouble with my server load, i have already moved to dedicated to try to cure this but still get high server loads, if i disable vbet the server load drops.

Can someone help?

@Kamil, you already have full access details sent to you in PM if you want to check.

kamilkurczak
26-05-11, 20:57
hello,
here you have a vBET list of hints: http://www.vbenterprisetranslator.com/forum/general-discussions/243-vbet-performance.html

remember - if you have all languages enabled - you have 53 more content (threads, posts etc) on your forum, and of course high-traffic forum.

You can disable some languages and wait when cache for enabled language will be filled, after that - you can enable the next part of translations.

Secondly, In this relase we have a big improve (very big one) - this is memory cache system (4 different options). You can check it in your admincp->vBET Cache. (remember - you must integrate your server to use this kind of cache)

:)

Simon Lloyd
26-05-11, 22:15
I have implemented all that i can of your performance recommendations, i have also had all languages enabled (well most) since buying vbet around 6 months ago, the load is consistently high all the time. As for the Memory Cache(beta) i don't think i have any cache accelerator on my server but evrything else is set as per your instructions.

kamilkurczak
26-05-11, 22:19
from our last relase we worked on this memory cache system - this should be the big one in performance. You need to have your server configured with this cache system (maybe you already have - ask your server support)

Simon Lloyd
26-05-11, 22:31
Hi Kamil, i just checked php info and i do not have it, but i will get them to install xcache or something like this

vBET
26-05-11, 22:59
Hi. Why you have trouble with your server load? I see your forum is translating really fast, so where is the problem?

Please note that translations are made by external services from Google. This means that when translation happens your threads are waiting for response from Google. And that is why your Server Load is bigger, but it doesn't influence the system, because waiting threads take no CPU nor additional memory (just already used). So that is why you will always have higher Server Load with vBET enabled (when disabled no thread is waiting for results from Google) and in same time your forum will still working fast, because waiting threads take no CPU.

So what is really needed to take Server Load lover is to assure that threads will not wait for results from Google, but unfortunately Google allows to cache its results only for 15 days. You can try it yourself - if you disable cache, or just make its time to live smaller then you will se Server Load growing - because less results will be cached and more requests waiting for Google responses. There is no magic - translations comes from somewhere and it takes time to get it from Google.

Using memory cache can help in some way. Threads waiting for Google results will still be waiting (you will not pass it until you have cached all translations, and you will not because it is cleaned according to Google TOS). Still cached results will be generated faster, so requests not waiting for translations will go faster from requests queue.

Here you can find out what exactly server load is: Server load (http://whreviews.com/server-load.htm) and find out that as long as you have free resources Server Load is not an issue.

Knowing what the value of the server load is not very important though. Knowing how to interpret the value is what counts.

remember the definition: the server load represents the number of processes waiting to access the CPU. But not all processes are the same! If the processes are low priority, when a new server request (page request) appears, it can still be handled almost instantly.

Not to mention that the server load is just one factor out of many others (memory usage, CPU usage, size of swap file)

As usual, and as many people directly implicated in the hosting business say, it all comes down to real-life behavior. Are the pages loading fast? Does a process such as searching through a database take a reasonable time? Then you don't really have a problem, whatever the server load is

So, the ultimate test is the way that the server behaves. If the server is fast, a number, even if it's called "server load", really doesn't mean much

So because your server is fast and I can see your forum responses fast, your issue is only virtual - there is no real issue. Server Load is value which gives you no information about real performance, just hints if you know how to interpret it (i.e. if server is running well with server load X, then you can start checking what is going on, if it rapidly grows to 2X or something like that). Just static number gives you nothing, better check your CPU and memory to be sure that this value is OK for your server and if it is OK then no matter what Server Load value is.

Still if you want some additional hints:
- make bigger cache TTL if you changed it to lower (default is maximum allowed by Google TOS).
- if you have no HDD space issues turn off cron task for cleaning guest cache - it do not have to be cleaned at all, because it refreshes results as needed, still removing lot of old files can take really long time for php.
- use memory cache
- at the end (but I really see no sense of it, because you do not have performance issue) start disabling languages which gives you less traffic

By the way – what is your Server Load, average CPU and memory usage, and what hardware do you have (how many CPUs, memory)? Does swap file is used?

Simon Lloyd
27-05-11, 08:34
Following are the details as per your questions
1) Server Load
*********************
08:22:53 up 44 days, 13:31, 1 user, load average: 4.07, 5.09, 5.26
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root pts/2 datacenter1.supp 08:17 0.00s 1.53s 0.00s w
****************
2) CPU and memory usage
********************
total used free shared buffers cached
Mem: 4040 3616 424 0 256 2809
-/+ buffers/cache: 550 3490
Swap: 8001 24 7976
*********************************
3) how many CPUs
3

EDIT: just took a snapshot here http://www.thecodecage.com/forumz/server.png my server is like this all the time, this snapshot is AFTER enabling memcahce.

Simon Lloyd
27-05-11, 17:19
As added information here's the Top Processes log:
xxxx 93.0% /usr/bin/php /home/xxxx/public_html/forumz/vbenterprisetranslator_seo.php
xxxx 83.0% /usr/bin/php /home/xxxx/public_html/forumz/vbenterprisetranslator_seo.php
xxxx 75.0% /usr/bin/php /home/xxxx/public_html/forumz/vbenterprisetranslator_seo.php
mysql 6.8 %/usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/xxxx.thecodecage.com.pid --skip-external-locking
mysql 6.7% /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/xxxx.thecodecage.com.pid --skip-external-locking
mysql 6.6% /usr/sbin/mysqld --basedir/ --datadir/var/lib/mysql --usermysql --pid-file/var/lib/mysql/xxxx.thecodecage.com.pid --skip-external-locking
root 6.0% /bin/sh /usr/local/bin/rkhunter -c --cronjob

vBET
29-05-11, 20:52
Following are the details as per your questions
1) Server Load
*********************
08:22:53 up 44 days, 13:31, 1 user, load average: 4.07, 5.09, 5.26
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root pts/2 datacenter1.supp 08:17 0.00s 1.53s 0.00s w
****************
2) CPU and memory usage
********************
total used free shared buffers cached
Mem: 4040 3616 424 0 256 2809
-/+ buffers/cache: 550 3490
Swap: 8001 24 7976
*********************************
3) how many CPUs
3

EDIT: just took a snapshot here http://www.thecodecage.com/forumz/server.png my server is like this all the time, this snapshot is AFTER enabling memcahce.

I'm not sure how to interpret your measurements. As I see your CPU is used in around 185%... So or there is something hot right in measurement or it applies to 1 CPU - if you have 3 then you can use 300%. If this is correct way then it means that you have still lot of free CPU resources. If not please correct me and tell how to interpret the results.

I'm also not sure how to interpret your server load data:

load average: 4.07, 5.09, 5.26
Are those 3 different in time measurements, or each one is for separate CPU? What is the power of your CPU?

PS. Cache needs time to fill in.

Simon Lloyd
29-05-11, 21:54
Michal, again thanks for the reply the load average you see (and in the snapshot) changes every time you refresh, it is as you point out a sybmolic number, when the load averages shown there are around 6 or 7 then the site is slow but most of the time unresponsive which in turn gives an error.

Here's the cpu data you asked for
Processor #1
Vendor
GenuineIntel
Name
Intel(R) Xeon(R) CPU X3430 @ 2.40GHz
Speed
2394.030 MHz
Cache
8192 KB

When i check in WHM i have 1 quad core cpu, whereas the server people tell me i have 3 cpu's???

I do not know or pretend to know how the server load average is worked out (lately, over the past 2 days it has been around:
Load Averages: 4.49 4.04 3.95

Forum is of average speed and it does not seem to be causing an issue right now, although traffic is around:
271 (3 members and 120 guests and 148 Spiders)
Where the top figure is normally around 500 - 600, but then it is weekend so traffic is lower.

vBET
02-06-11, 23:23
I asked about CPU speed, because as we already told Server Load is just virtual value, so for fast CPU same server load will mean something else than for slow CPU (fast one will do his jobs fast, so even if there are additional tasks waiting in queue those will be handled fast).
About your number of CPUs - at this moment I'm lost just like you. It is your server and I will not help you to determine do you have 1 or 3 CPUs. I can tell you that for servers it is better to have bigger amount of slower CPUs than smaller amount of faster. 2+2=5 it means that 2 processors with power 2 will do better job than 1 processor with power 4, because server have lot of small tasks, and having 1 processor you cannot do it parallel way.
Server Load will also have different meaning according to number of processors. If you really have 3 fast processors then server load 6 (total not per CPU) is OK even without additional checking. If you have 1 processor then you should additionally check real response time for our sewer. As you wrote you already did it and it is OK.

For time you see your server load is bigger and page slower (cached one - please remember that for not cached the real response time is not up to your server at all, but up to Google response time), please check is it because bigger traffic, or maybe it is because of some background tasks (like vBulletin cron jobs, or even your own operation system - like automatic update tasks or something like that).

According to your last response - if there is no real issue - do you need any future assistance in this topic right now?

Simon Lloyd
04-06-11, 08:46
Michal, thanks for the detailed response, it is 4 cpu's, my only concern is the amount of % of cpu that vBET consumes, as i type this there are3 processes shown in TOP for vBET at 55%, 52% and 48% cpu usage and these processes are all for the file /vbenterprisetranslator_seo.php, if there is nothing more you can suggest then i thank you for your comments and will have to live with it as i definately cannot afford another server move or upgrade as i went from having vps to a VM in cloud hosting to dedicated in order to keep vBET running, the next jump to dual quad core (8 cpu's) is out of my price range.

vBET
04-06-11, 21:03
Please note that what ou see it is not vBT CPU consumption but your whole forum consumption. vbenterprisetranslator_seo.php does nothing - just sets some variables and made internal redirect to really requested file - it is just front controller. All requests goes to vbenterprisetranslator_seo.php - you have set it in your .htaccess file.

So this is not vBET consumption - this is your whole forum consumption. For normal pages vBET does nothing - only adds flags. For translated pages whole translation process happens so CPU consumption will be always bigger than for normal page, because it happens after normal page is generated. So first normal generation happens and then result is translated - so there is no chance to have it withour additional cost. Or... We have solution for this ant it is named Guest Cache - for guests whole pages are cached and no translation happens as long as cache didn't expired. So if you are already using Guest Cache then vBET have no more solutions to make translations using less resources. You can only disable some translations - you will have then less traffic to translated pages so less resource consumption for translations.

We made lot of effort for profiling vBET, changing algorithms, adding more performance improvements. And we never treat this issue as closed. Still at this moment even with several layers of cache, we are working with application which have lot of changes and different content for same URL depending of user usergroups, or even for each user (if some plugin adds such things) and this requires retranslate for each logged user request. For logger user we can only cache sentence translations, but not whole pages like for users. Please note that vBET is adding more and more performance improvements – still you will never have magical translations with no resource consumption. Additional functionality always means additional resource consumption.

We checked your forum response time and it is really good. So I wouldn't think about switching to other server at this moment. Just as you wrote - you still have half of resources free. You pay for this resources so there is nothing wrong to use it. And you are still in safe zone - so even when your traffic increases your forum will response in appropriate time. For CPU first light is 70% of average usage (not just in some moment - average). This is first hint to worry about resources. If you ignore this then 90% of average CPU consumption means read light, alarm, and immediate needs to upgrade - otherwise even little more traffic can make drastic performance degradation. For memory it is different and depends on OS settings for SWAP.

And as PS - please consider solutions like VPS - where you can add resources very easy without any reinstalling :)

If you already fallow all performance hints then only thing it left is to disable some languages and made updates of vBET as those come.

AfrikaansAlbanianArabicBelarusianBulgarianCatalanChineseCroatianCzechDanishDutchEnglishEstonianFilipinoFinnishFrenchGalicianGermanGreekHaitian CreoleHebrewHindiHungarianIcelandicIndonesianIrishItalianJapaneseKoreanLatvianLithuanianMacedonianMalayMalteseNorwegianPersianPolishPortugueseRomanianRussianSerbianSlovakSlovenianSpanishSwahiliSwedishTaiwaneseThaiTurkishUkrainianVietnameseWelshYiddish
Translations made by vBET 4.10.1