Henry Miller lived and wrote in Big Sur from 1945 until 1963. Big Sur and the Oranges of Hieronymus Bosch, published in 1957, contains stories about his life in Big Sur.

Miller is known for his two semi-autobiographical novels, Tropic of Cancer, published in Paris in 1934, and Tropic of Capricorn, published in Paris in 1939. Both books were banned in the United States as pornography.

In 1942, shortly before moving to Big Sur, Miller began writing Sexus, the first novel in The Rosy Crucifixion trilogy which was also banned in the United States. Miller's books were smuggled into the United States and influenced Jack Kerouac and other writers of the Beat Generation.

In 1961, the publication of Tropic of Cancer by Grove Press in the United States led to obscenity trials that tested American pornography laws. The Justice Department ruled that the book was not obscene. In the 1964 case, Jacobellis v. Ohio, the United States Supreme Court ruled that protected speech could not be censored. In his concurring opinion, Justice Potter Stewart coined the enduring phrase, "I know it when I see it."

The Henry Miller Memorial Library is a non-profit organization championing the literary, artistic and cultural contributions of Henry Miller. The library serves the arts, the community and the environment.

The motto of the library is "where nothing happens."