Since I started writing this I got some information about current problems with MySQL. Rumour has it that MySQL is working on this. http://tweakers.net/ has been doing a lot of testing of running MySQL and PostgreSQL (with different hardware). Below are links to some articles they have written about this. Take a look at the graphs there. It looks to me that there indeed is some problems running MySQL with many concurrent users on some hardware (if I do not misunderstand something there):
Some small steps forward 2007-07-10
It seems to be a bit work to get this started. I have taken some small steps forward:
- I have contacted some web hosts to see if they want to take part in the survey.
- It is a bit cumbersome to produce the graphs. I got a tip that JGraph for PHP might be good. Unfortunately it seems hard to install on Windows. I am currently trying to setup Gruff for Ruby instead. Any tips?
- I took a look at the forums at WebHosting TALK. They seem to be a good place to discuss about web host performance. Any other tips?
- So far it looks to me that perhaps Hosting Reviews at Hostjury.com is a valuable resource.
After looking around a bit more I am even more convinced that a measure of CPU and SQL related performance is necessary to find a good shared web host. Currently disk and bandwidth are cheap, but CPU is not that cheap. So very many web hosts offer you a lot of this and many databases — but you can not use much of that because the fine line says there are CPU restrictions. It is even said that some hosts just lock the customers accounts when they use “too much CPU”.
The Idea 2007-07-05
When you are looking for cheap, good web hosting (for example for blogging) it is quite difficult to find information about CPU or database related performance. Nowadays database and scripting performance are probably what you are looking for. Maybe there are some good comparisons out there, but I have not found them.
So I thought about an easy way to start something. A standard application that is available everywhere is WordPress — so why not try to use that. It will not be a precise measuring of course, but that is not needed (and anyway impossible in our case). The main thing is that it can be done by nearly anyone on any cheap host.
Of course WordPress blogs differ very much, but the admin pages are similar everywhere. And at the bottom of each admin page there is the time from the start of the script to the end of the script. Indeed very good for our purpose!
I did some testing on my current web host. I wrote a short script on my pc that periodically brings up some WordPress admin pages in the web browser and logs the results to a simple text file. Then I took that into OpenOffice.org Calc and made a simple graph.
You can see the result in the pictures below. The results in the picture are actual results from a shared web host I use.
However what I want you to notice is what I have collected here. The figures are explained to the right in the picture and I guess you can understand there meaning. In this particular picture it is the times I mentioned above from the WordPress admin pages. They are collected over several days. The first picture gives an overview over the time it took to create that page on the server:
Note that the scale is logaritmic. That is necessary to show all the values in one chart. The fastest creation time is 0.1 second and the longest is 300 seconds — 3000 times longer. Roughly the results can be divided into three equal sized group:
- Less than one seconds. Clearly acceptable.
- Between 1 and 10 seconcs. Acceptable if it does not take that long every time.
- Over 10 seconds. Unacceptable (except if it occurs just occasionally of course).
The next chart shows the max time divided into the hours of the day (collected over several days):
This graph is a little bit unfair perhaps, so it must be viewed together with the first graph, which says how serious the situation is. This chart merely shows at what time of the day it is worst.
The last chart shows more normal execution time at different times of the day. To get some reasonable values I had to choose 25% lowest percentile values:
This chart together with the previous one shows that the meltdown where the time to create pages are extremely high seems to suggest that in this case the load on the servers influence the meltdown.
Now I need your opinion!
Now what do you think of this idea? Would it be useful to collect this for different hosts? How would it be done? Should the web host collect them or should customers on the web host collect them? Or both?
