It took me 53 hours to beat the game, and there were several sidequests I didn't have time to finish.
It's true that in single player, you can play as any of the four people in your current party. However, the game defaults to the character in the first position for each battle, and then you have to change it manually in the menus for someone different, so if you plan to play with another character for some length of time, a reordering of the party helps reduce the hassle.
I doubt you could get a group of four to play through any major chunk of the game...only a few of the characters are really fun to control, the rest are heavy into magic, which is boring to use manually. Plus the camera always follows player one, which means the other players will often be fighting offscreen. It's still a cool feature for when friends come over and want to check the game out for a few minutes. I don't recommend buying the game solely on its multiplayer feature.