I actually find most of Season 4 onward to be pretty watchable. My wife loves Voyager, and I'm happy to watch when she has it on.
The problem with Voyager (okay, ONE of the problems with Voyager) is that it never had exceptional episodes. Good ones, yes. Even a few very good ones. But no transcendent episodes. The Year of Hell reset button (a variation on "it was all a dream") prevented it from being such. Would've been really cool if they had been able to make good on having those episodes part of a longer arc, with aftereffects that lasted even past then.
Also, the Doctor and Seven were great supporting characters. But because they were so much better than anyone else, they wound up taking over the show. As the show's run went on, they were put in much more of lead roles than they should've been given.
Therefore, instead of incrementally growing as characters over the course of the show, they had to have massive revelations nearly every episode -- only to be reset by the next week.
Their prominence also gave the show a weird paradox. It watered down their impact some when they were onscreen. And yet when they weren't the focus, it was often glaring how much worse the show was (with some exceptions, like "Timeless").