If you go to an event (of any type) and see participants - but few spectators, to me it's an indication that it's not an activity that'll be supported by the community. If 99.9% of the community has zero interest, when an owner of a potential site is contacted, it'll be unlikely that they're interested, so there'll be very little chance that they'll support it. Worse, autocross being what it is, the chances of someone going on their head is not zero, so adding the liability only makes it worse. Then there's loud cars causing complains, another thing the property owner has no desire to deal with. Property owners end up with zero reasons to support it, and several to not.
I was fortunate enough to run at Jack Murphy stadium in San Diego back in the 1980s, and it was awesome. It was a huge parking lot - pre-berms, bumpstops, and grass islands, and fast cars could get about 100mph. Then the trolley went in, removing about 30% of the useable area, which reducing interest by the faster car owners. Then there were a few other changes "nibbling away at the cookie", with the available space getting smaller and smaller. Then of course, the entire stadium shut down and is gone. The next closest venue is either Lake Elsinore or AutoClub Speedway, both much further away for me.
The above, and a couple other things reduced my interest in both autocross and trackdays. In the 1980-90s, many people ran older very used sports cars and did much or all of the work themselves. People were very knowledgeable about their cars and between events, it was common to see competitors helping each other out. Things gradually changed, and now that aspect seems to have faded, with people instead seemingly buying something new that's very fast and more reliable, with no need to work on it. Plus, they're more complicated cars now, so owners are less likely to get their hands dirty. That's fine, but has the side effect that the social aspect of the events have faded some, with there no longer being as much reason to interact with others. As others have mentioned, it also seems like the events are going "up market", where it's more common to see trailers, when before, true grassroots cars were driven to events.
Lastly, many here commented about the costs of the activity. Even though I worked and paid my way through college, I was still able to attend autocross events and had a great time. I think when I started, the entry fee was around $20, and maybe there was yearly membership, but I don't remember it being terribly expensive. The last one I went to required joining SCCA and the entry fee was around $45. Given that I wasn't going to run more than a few events, I turned around and went home. I realize everyone's story is different, but that's my take on it.