I stumbled upon Mitsuwa Marketplace while doing some shopping at Comp USA (apparently Japanese grocery stores are as difficult to find as a Comp USA here!). It is the only Japanese grocery store I know of in the city. It is fairly small, but within the shop are several smaller retailers as well -- a Japanese book store, a travel agency (specializing in travel to Japan, of course...with really great deals by the way!), a stationery store complete with the requisite Hello Kitty items, and a Japanese ceramic/pottery shop.
The produce section is small, but ample. If you need Japanese eggplants or cucumbers, you'll find them here. You can find all of your pressed/ shaped fish ball type things in the frozen food section -- good for whipping up your own bowl of ramen. There are some premade goodies as well, sushi, sashimi, precooked mushrooms and tofu. The dry good aisle has all the basics of a well-stocked Asian pantry. The snack aisle naturally is my favorite, where I can always find the very delectable Men's Pocky.
In case you get hungry while shopping, there is a small restaurants where you can pick up a pretty decent bowl of noodles or gyoza.