Besides EMusic, iTunes, and Amazon mp3, what else is out there? There's quite a few services, mostly filling smaller niches in the huge and growing online music market. I don't use any of them very frequently, but here's what I know:
Amie Street. The idea here is that the market determines the price. The more people download the track, the higher the price climbs. You get a deal if you get in on an artist/album/track early. Next thing you know, they'll be setting up a derivatives market so you can bet on how soon the hipsters will mob any particular indie band. If you can do it with mortgages, why not music too?
CD Baby is a service that allows independent artists to market their own music. The site now also allows artists to sell albums in mp3 format. So far as I've seen, it's up to the artist to decide to allow the option or not. And you can't download individual tracks.
Bandcamp is a pretty nifty site that allows independent artists to sell (or distribute for free) mp3s of their music--sort of a next-generation CD Baby. It lets the artist set the price, or lets them choose to let the customer pay whatever they want (the "In Rainbows" model). I ran across the site while browsing the web page of the California Honeydrops, a soul group from San Francisco. Of course, Bandcamp is basically a distribution platform, not a music site for itself--so you can't browse for artists. You'd have to find out about a group through some other route, and then perhaps get re-directed to Bandcamp to grab some of their music.
Smithsonian Global Sound is truly for the music geeks out there, people who are interested in historical recordings, field recordings, truly diverse sounds from around the globe. They have a lot of the original Folkways recordings, including stuff that founder Moses Asch recorded back in the 1940s. All sorts of interesting music on here, including some fantastic blues compilations like this one. Not necessarily a good source for swing dance music, but then you do have wider taste than just jazz, don't you?
Lala is another music service that plays with the standard iTunes pricing model. It allows you to preview an entire track once (rather than just a 30 sec. snippet), and then buy the rights to stream the song online for just $0.10. Actually downloading mp3's costs more--about the same as Amazon mp3. Lala also scans your music library and gives you free online access to any licensed tracks that you already own. (Added 9/27/2009--Thanks Thomas for the tip.)
Jazz Online has a bunch of jazz recordings available for streaming or download, plus a basic but serviceable search engine. The site claims that all of the music is public domain, though I haven't investigated it enough to know whether that claim is true. (Added 9/27/2009.)