None of the talking heads seem to get it with regards to why the Bay Area in general and San Francisco in particular is so expensive. Most focus on one or two reasons which very conveniently dovetail with their pet political beliefs and preconceptions, but almost nobody gets the big picture.
A robust local economy that creates ample amounts of well-paying jobs, ensuring a continual supply of new residents who are able to bid up housing prices should the need arise.
Regulations make it very hard to expand housing supply, but easy to expand the supply of business offices. Commercial areas tend not to have homeowners and long-time tenants who get upset about new neighbors.*
Politics determined largely by narrow self-interest. There’s lots of self-professed liberal and even leftist sentiment in San Francisco, but the politics is often very shallow. If it doesn’t affect them personally, many simply do not care if the policies they advocate hurt others.
Price controls that help shield the narrowly self-interested from the consequences of the hurtful policies they advocate. Both rent control and Proposition 13 play big roles here (funny that those so willing to rake rent control over the coals are silent about Proposition 13).
Regulations elsewhere in the Western USA that make it impossible to build a city like San Francisco, which is pretty much unique in the West in consisting entirely of low- and mid-rise attached housing. If you want a home of your own, but you dislike yard work and don’t like condos, and you don’t want to live on the East Coast, there is basically only one city in the USA that satisfies your requirements.
Worldwide popularity which ensures that the many see San Francisco as a highly desirable place to live.
A mild climate without extremes of either hot or cold. So mild that I personally find it monotonous, but many find it completely to their liking.
Scenic beauty, another geographic attribute that people are willing to pay a premium for.
Just a couple of the above could doom a place to housing affordability problems, yet all of them apply to SF. Worse, some of them are entirely out of the City and County of San Francisco’s ability to fix. (Or are desirable attributes that nobody would want to fix.)
* Much ink is spilled about how rent control supposedly “causes” the problem. Rent control does play an role, but it’s hardly the prime driver of the situation. First, rent control in California by state law exempts new construction, so its supposed baleful effect of discouraging new rental construction simply does not exist. Second, if rent control primarily was the problem, you’d expect a disconnect in prices between rents and ownership costs; SF would be an expensive place to rent in but not all that expensive a spot to buy a home in. The real issue is supply restrictions in the face of robust demand.