The rest of your post, yeah, agree to disagree on how I approach the creation of Mario villains, but this statement is a bit bizarre, since the reason most other nonRPG games don't get villains are like, the sports and party titles which when contexualized, aren't story heavy games? And honestly, for Mario Party? Bowser is amazing as a villain there, he's always perfectly suited for the job, because he has entertaining dialogue and has this hamminess and sadism only he could pull off. Only he can be petty to ruin the Mario Partiers vacation because they didn't invite him along.
Perhaps, new RPG villains can shake up storytelling a bit with their different personalities, if they are executed well. Fair enough. I liked the Shroobs and the alien invasion. However, I don't see what new villains would bring to the table in Mario platformers. Those titles have always been more gameplay focused than they are in narrative and I think it's pointless to conceptualize a new villain when first and foremost, they want to make a gameplay fun rather than who is behind the razorthin plot Mario platformers have anyway. From a dev point of view, it's wasted energy that could have gone to more practical purposes such as enemy variety and design, level design, and whatnot, especially since a new villain or a new narrative isn't what people play Mario platformers for.