You're bang-on, Alexy. The pier was known as both the Olympic Pier and the Lurline Pier, because it served as the intake for both of those downtown saltwater pools: the Olympic Club and the Lurline Baths.
Here's the entry from Wikipedia: "The Lurline Baths were public salt water baths built in 1894 in San Francisco, California at the corner of Bush and Larkin streets. They closed in 1936. The Lurline Pier, built at Ocean Beach between Anza and Balboa Streets to protect the intake pipe which provided seawater to the Lurline Baths, existed until about 1966."
I might question the date of demolition, though. I remember taking photos at Ocean Beach in 1966-1967 and the pier -- or its remnants -- was still in place.