I use a VPS hosted by SliceHost as my personal server. SliceHost uses Xen to host multiple instances of Linux on a single machine. The performance of this setup has been very good.
I have been running:

Apache 2.2 with PHP
MySQL 5
Postfix Mail Server
Courier IMAP Server
ssh for remote access of course

I recently started playing with a site [...]