The Belin House Restaurant, tucked in a nook on the grounds of Hagley Museum and Library is a fantastic lunch spot. Although I'm pretty sure you have to purchase a day pass to Hagley to check out the Belin House, it's definitely worth it.
The dining room is rather small, but the service is great; prices are reasonable, and the food is excellent. I love their chicken salad croissant-- it has granny smith apples and walnuts with cubed breast meat. The desserts are out of this world, too-- try their strawberry shortcake if it's in season!
more