The land on which Rincon Annex is located was previously owned by the Southern Pacific Railroad and was a proposed site for a train station. With the construction of the Bay Bridge, and the increased automobile and public transit commuting on the bridge, the need for a new depot was obviated. The Federal Government obtained the site for its new mail handling facility.

Ground was broken on June 1, 1939. Construction was finished on October 15, 1940, and the facility officially opened on October 26, 1940.

Rincon Annex, designed by Gilbert S. Underwood, is a superb specimen of the Streamline Moderne architectural style, a period piece in mint condition. The only stylistically comparable public building in San Francisco is the Maritime Museum, also built in 1939.

The Rincon Annex Post Office is also National Register Listing 79000537.