Disclaimer: this article does not pretend to be a complete picture of using swap on UNIX.
Most of the time, recent GNU/Linux distributions insist on creating a swap area.
Swap is generally used to temporarily store memory used by running(/sleeping) applications, but might as well be used to store a copy of RAM to disk for hibernation. As disks are still many times slower than RAM, it always comes with huge performances impact.
Historically, RAM was so expensive that UNIX users bought only a fraction of actually used memory, and relied on swap for 2/3 of it (number still found in many advices today, while it is certainly outdated).
Today, you can buy quite a huge amount of RAM (even ECC) for almost nothing, it is not uncommon to be able to buy more than 4GB for less than 100€. So today, you buy RAM based on the total memory used by all the applications that might ever run at the same time. Of course, you probably don’t need then to allocate three times that amount to swap, especially when you know applications won’t use it.
The main danger of allocating much swap is that some application goes mad, eats up all memory, then swap and starts slowing down the whole machine (because it brings I/O in the dance), then (and only then) eventually get killed by the Linux OOM Killer, which is rather basic (just kills the running application taking currently the more memory, so might even miss the real guilty). Without swap (or reasonable amount), in contrast, the kill would happen far earlier, and not generate incredible I/O usage levels.
Of course, I/O impact is even worst in virtualized environments.