The House On Telegraph Hill - Yacht Harbor
Victoria and Marc meet at a yacht harbor, the west harbor on Marina Boulevard in the Marina district (map). Below is a map of the harbor marked where the movie scenes took place.
Then ... An establishing opening shot of the yacht harbor is shown below taken from near the St. Francis Yacht Club (location A on the map above). The view looks east across the harbor towards the Russian Hill skyline with Fort Mason on the far left.
... and Now, absent the extra buildings atop Russian Hill you would never guess 60 years have slipped by. (Incidentally, Orson Welles's 1947 movie The Lady From Shanghai featured this same location - see it here).
Then ... Victoria meets Marc outside the harbormaster's office (location B on the map above). There's an old stone lighthouse behind her at the harbor entrance.
... and Now, the office and lighthouse are still there but the harbor entrance has been moved to the east by an extension of the spit on which the lighthouse stands.
Below is a closer look at the lighthouse, built in 1931 but no longer in use.
And here's a recent look at the harbormaster's office.
Then ... They talk outside the harbormaster's office. This scene was filmed in the studio using projected views behind them, a common movie technique known as process photography which gives better control over sound and lighting.
... and Now, the background was indeed filmed outside the harbormaster's office looking southwest across the harbor towards the dome of the Palace Of Fine Arts. The view today looks just the same.
Then ... Victoria reveals to Marc that she isn't Chris's mother after all, that she had assumed Karin Dernakova's identity after she died at the Belsen concentration camp. Marc, unfazed by the revelation, confesses his love for her. This background, supposedly at the same spot, was filmed on the south side of the harbor at location C on the map above - it looks west towards the Golden Gate Bridge.
... and Now, the same view today, with the bridge partially obscured by, no surprise, fog.