running databases inside Vmware
This is just a warning note for people running databases inside Vmware.
The problem:
At my employers company we develop some business critical application for one of our customers. This customer decided to employ Vmware ESX host as a container for a testing farm for this application. One of servers in this farm is a PostgreSQL machine.
The Very Bad Effect that we noticed while debugging some nasty db-related problem is:
Inside a guest OS, time flows differently.
That is, one second can last 1/2 second or 3/4 second or 2 seconds - depending on the disk I/O. The higher disk I/O rates, the slower time flows.
Somehow, this has fatal impact on performance of PostgreSQL. Algorithms which rely on precise timing simply stop to work. This could be a matter of interesting analysis which I don't have time for right now.
I am not a Vmware expert, and probably this is just a matter of bad configuration.
Probably it appears only when you run many single-core guests inside one multi-core host, but anyway - be warned.
