Scholars disagree on the exact provenance of the Captain Leal House, but it is certainly one of the oldest residences in the Cow Hollow neighborhood of Pacific Heights.

The structure was originally the main house for a dairy farm and may have been built as early as 1853. Captain Leale, a ferry boat captain, bought the house in 1883, and either he or the previous owner remodeled the facade with a false front in the popular Italianate style. In the back garden, the Captain built a little study, nautically furnished and fitted to resemble a pilot house.

The building in the background is a residential co-op at the intersection of Jackson and Steiner Streets known in progressive political circles as "Susie's Building." Bill and Hillary have been here. Pelosi and Feinstein and Boxer have been here. President Obama has been here when he was a just a Senator. According to Sam Whiting, writing in the San Francisco Chronicle on 3 June 2007, "...this elegant 1920s tower that stands alone over Alta Plaza Park like a white pencil over green paper. The eraser at the top is a red-tile roof. The point at the bottom is for writing checks to Democratic candidates."