Virtual reality has mainly three things going against it:
- The cost of entry. You'll have to pay in the regions of a proper console (not necessarily a PS5/Xbox Series X, but still a more versatile console) which comes with its own downsides like mobility but lesser graphics quality or bigger games library with better graphics but tied to a computer. And then you have all these platforms that make it harder for developers to optimise the performance.
- You (often) don't get VR until you've tried it. No trailer can do justice to a VR game, you'll have to experience it - to be IN the game. And let's face it: VR has a more prominent/common honeymoon period than many other things in the gaming world.
- People's laziness. VR comes at your senses and pushes your body in a much more intense way (that's why we love it) than traditional gaming. Many people play games to relax and unwind, not to get an exercise session.
On top of that it's probably both harder and more expensive to develop for VR since it has to render two screens at once and because the camera is not locked (but tied to the player's head) you can't use ordinary performance saving tricks (more of the game world needs to be rendered).
VR is still in an early stage and will always be around in some form or another. When the hardware has matured and you can get more advanced graphics on a mobile VR device then we will see the glory days of virtual reality.