Sega was always gonna be stomped by PS2. Sony were able to make deals with 3rd party left right and centre. Gta, mgs, final fantasy, dmc etc... This is what enabled them to dominate the market as well as catering to an older audience.
Basically, Sony went big on moneyhats and placating to an audience Western Sega had already been appealing to with the Genesis/MegaDrive (although they did build upon that, to be fair).
It wasn't just Nintendo and Sega. Others would have kept trying just like the TurboGrafx, CDi, 3D0, Atari Jaguar, NeoGeo, etc all did and failed. PlayStation was just the first to succeed.
If they hadn't, it would have been someone else. The way Sega was heading, they would have likely failed regardless of PlayStation and Xbox. Their boneheaded decisions began before PlayStation began dominating the industry.
TBF the Neo-Geo AES was not trying to directly compete with Nintendo and Sega, the price point alone gives that away. The Saturn was actually a quite fine system, and in a vacuum with no PS1 and an N64 that was still catridge-based, Saturn would've more or less came out on top that generation, though it'd be a close 2nd for Nintendo globally.
Both N64 and Saturn would've benefitted from no PS1, in somewhat different ways, but even in spite of 32X (which never got going in Japan or even most of Europe, FWIW) Sega were on an upclimb. No PS1 would've meant them supporting MegaDrive/Genesis longer.
1 - The M2 doesn't seem like much of an upgrade over the Sega Saturn, and by that time Sega had been launching way too many consoles and upgrades, and losing way too much money with it. That was probably the motivation behind not going with another console so soon after the Saturn.
2 - There's no reason to believe the MX would perform better than the Dreamcast. Going by number of triangles alone is very reductive even for the late 90s. The PowerVR GPU in the Dreamcast is a tile-based renderer, meaning that by nature it only draws the triangles that are present in the screen. All other GPUs did a lot of overdraw in comparison, and continued to do so for a decade. Not only that, it used
PowerVR Texture Compression for 2bit-per-pixel textures, supported bump mapping and 32bit color rendering, supported translucent polygons, among a bunch of other technologies that made the Dreamcast a lot more advanced than competing GPUs of the time.
3 - Lots of things killed the Dreamcast besides the impending arrival of the PS2: piracy for the console was rampant in a time when everyone could buy a CD-ROM writer and download games from the internet (the console would read CD-ROM games as GD-ROM ones just fine, and at the time there was no forced firmware update to stop it), and Sega of America was also suffering from a massive lawsuit held by 3dfx for supposedly leading them on developing a GPU that Sega ultimately didn't adopt.
Watch Jenovi's Dreamcast documentaries, specifically the 2nd one IIRC which touches on piracy. People today have MASSIVELY overestimated the impact of piracy on hurting Dreamcast. Typical home internet speeds in 2000 were not even T1-level, let alone broadband, so that meant very slow dial-up for games approaching several hundreds of MB (and in some cases, GBs) in size. CDI rips of DC games were always compromised in some way, and circulation of them was very region-based, I'd even say down to specific towns and neighborhoods, so either you were nearby to buy or you didn't get them at all.
The 3DFX situation was actually 3DFX's fault, not Sega's; they leaked details on Dreamcast in court proceedings well before Sega were ready for them to come out, because they wanted to bolster their case. Sega, naturally, retaliated. Tech-wise it was for the better since the Power VR2 was more capable than what 3DFX had going for their prototype, but it was a messy falling out for sure.
Dreamcast could have never beaten the PS2, the PS2 library had everything any anything it had the most amazing weird & wonderful games from japan it was still at the point when playstation consoles had games that were made just because devs wanted to make them so you got to see the oddest and fun games to come out. Owned the DC twice and both times i have regretted it, i still stand that samba de amigo is the only game worth owning one for, compared to the output of the PS2 the DC library is weak.
PS2's library was pretty mediocre until around mid-2001, or over 1 year if going by the Japanese release date. Most people agree Dreamcast had the better library during that period, as PS2's big hitters of 2001 were back-loaded into the latter half of that year.
I don't know what you're going on about with the rest because Dreamcast has some of the weirdest games around; Seaman, SEGAGAGA, Napple Tale (which I'm playing right now and am very charmed with it), Rent-A-Hero, L.O.L (Lack of Love) etc. Lots of weird and charming oddball games there. Of course, not as many as PS2 but that would be obvious considering library size. But trying to boil down its worth to just Samba De Amigo is flat out an idiotic claim on an objective level, and a very odd one on a subjective one.
The problem wasn't the competition against the PS2. I'm sure the Dreamcast would have at least sold as much as the Gamecube if Sega kept selling it during a "normal" console lifecycle (4 to 5 years back in the day). THeir only problem was their finances, they couldn't survive with the failure of the Mega CD, the 32X and the Saturn.
Mega CD wasn't a failure; it did what Sega wanted, they made money off the hardware from Day 1, and it was the best-selling add-on in the industry up until the Wii Fit board and later still the Kinect. The North American library was definitely hampered by too big a focus on mediocre FMV games but the totality of the platform's library is markedly better and has some real gems present.
32X, on the other hand, was a very reactionary product and served no purpose that wasn't better served with expanding on the SVP chip concept they introduced with the MegaDrive/Genesis port of Virtua Racing.
Old Sega fans are more deluded than anyone when it comes to the Dreamcast. It was 150 million vs 10. There is no scenario where Sega wins
Roughly 1/3, or about 50 million, of PS2's sales came after its role as primary commercial platform for Sony (post-2006).
Of course 100 million is still legions more than roughly 10 million, but that's more to the reality of things and even that is pushing it because a better comparison would be PS2 sales during the 2000-early 2001 period since Dreamcast was officially discontinued January 2001. Kind of hard to sell new hardware for a product past its discontinuation period 'ya know.
I agree though that there was no scenario where Sega could outsell the PS2 that generation. That said, some things could've been done differently resulting in them being able to edge out a 5-6 year market presence and getting within Gamecube/Xbox territory of units sold.
Clearly not enough to stop PS2 that gen, but enough to potentially get by.