President Harry Truman was introduced to Key West when he need to take some time off to destress. He visited and was quoted as saying that he should just move the white house here! The house that he stayed in belonged to the military and it was where the top brass stayed. This house is no longer used for military but instead has turned into the Little White House Museum. And occasionally it is used by political VIPs for receptions and even as a place to stay.
For those that like history, especially political history, you need to visit when you are down in the Keys.