In my point of view, I really enjoy being a Toad because of their shrimpy nature in spite of all the bigger dangers he has to through. I really like how I can play as a Blue Toad in Super Mario 3D World, or Toadette in NSMBUDX and Super Mario Maker 2. I think his species in general is a very important to Mario and he's been there since Super Mario Bros. Peach is convenient for plot and Bowser reasons, so that really just leaves us with Toad, and to fit the 4 player quota for these games, I think that's why they went with him. Not to mention, he's just an attractive character to use in my pov: small, cute, and brave, which I synergize quite well with. He's also pretty unique, I don't think there's another platformer where I can play as a little mushroom man.
I think him being there is why you can play as a generic Koopa or Shy Guy in Mario spin-offs...some people just like the generic species and love to them represented by a random member of their species. Toad is one of them, though Toad's status as a single character is a bit more defined than the Boo you see playable in Mario Party.