Lou's Records is a great place to find new music, or old music if that's what you're into. They sell LP's, as well as CD's, in addition to some other music-related merchandise.
They have two separate buildings on the lot, one housing the used music and the other with the brand-new stuff. I love perusing the used records for my collection and often find some good ones in good condition, for decent prices.
I feel less comfortable in the new music building, like I don't fit in somehow. The clerks are mostly dressed in very "individual" clothing, etc. I just don't feel cool enough, you know? But they're fairly helpful and they do have a great selection. The prices are pretty fair.