Today I found Loop11.com (through a job advert in the Australian Web Standards Group mailing), an interesting site allowing for user testing (for $350) of your site for up to 1000 users.
The application puts a JavaScript layer on top of your website and requires you to define tasks for your users to complete (without documentation, of course). You then invite users (you have to invite them yourself) to come and test your app.
I've tried the demo (takes about 3 minutes for 3 tasks) and I must say I like the idea (although I would love to see some results, which is apparently not possible through this part of the demo, but you can setup a trial account for a short user test). The only missing part would probably be to gather users for you (I don't know... invite people to test in exchange of one dollar per test or something like that - I'm sure some unemployed people would spend days there.