I agree that in many D&D and co type spaces, the BBEG trope is super common and taken for granted. I also think that it’s good to get away from.
That said, I also agree with @gartsalrisa – I think you’re underselling how common this trope is in media, and to go further, I think it’s important to understand why. Before that, I’ll just say that I use BBEG fairly loosely, I mean it can be someone as conniving and all encompassing as Skeletor, But I think you know the evil grand vizier who is scheming to bring the kingdoms to war etc etc counts, even if they’re not quite as powerful. Just to get on the same page.
I think BBEGs were common before video games. Sauron, for example. And TV shows like Buffy and whatnot. I think they exist because they make it easy to motivate epic fantasy plots. Have you ever tried to run a game of courtly intrigue? It’s hard! Really hard! But knowing there’s this one being sort of playing chess behind the scenes simplifies a lot. Plus, it has an implied structure that is quite straightforward and fits well with traditional expectations of epic fantasy, as well as tradition leveling systems (which again existed before video games). With a BBEG, you get the drama of an unbeatable enemy – who eventually, through leveling and scheming and experience, you bring down. This is super satisfying to groups that want an epic fantasy campaign, because the investment in the foe as such is emergent from the gameplay itself. You have someone to focus your frustrations on, then eventually take down.
Compare that to what someone above said – various factions, then see where people go. This is a good approach to and is probably what I’d choose if I were running a DND type game these days. That said, when power and allegeniances and all that are diffuse, it’s going to take more work to get people to be invested in this side vs that said. It’s easy to want to take down the evil lich king. But it takes more work to get them to identify with faction 1 of 15, with the various shared of grey involved. Not hard or anything, it’s just a different kind of game with much less obvious pacing.
TLDR: BBEGs provide a classic, approachable type of drama that lends itself well to traditional epic fantasy arcs. I agree we can and should get away from that, but by understanding it’s strengths we can make sure we are still creating interesting stories that players will readily relate to!