https://youtu.be/V2Is7A5BNuw Gran Turismo 7's Spec II update included the release of GT Sophy 2.0, the latest iteration of the new artificial intelligence system built for the game by Sony AI. It can now control many different types of cars on a wider range of tracks, and the technology has come
www.gtplanet.net
How Sophy 2.0 is Different, and What Comes Next For Gran Turismo 7’s Super-Human AI
Gran Turismo 7’s Spec II update included the release of GT Sophy 2.0, the latest iteration of the new artificial intelligence system built for the game by Sony AI. It can now control many different types of cars on a wider range of tracks, and the technology has come a long way in since it was first made available to the public back in March of 2023.
What changed, exactly, and how was the team at Sony AI able to pull this off in such a relatively short period of time?
To find out more, we had the chance to speak with Dr. Kaushik Subramanian, a Senior Research Scientist at Sony AI working on the GT Sophy project, and Kazunori Yamauchi, Gran Turismo series creator, during the recent Gran Turismo World Series Finals in Barcelona. Here’s what we learned…
How Sophy 2.0 Is Different
Traditionally, computer-controlled opponents in racing games simply follow a path using a set of rules defined by the game’s programmers. Gran Turismo Sophy is different because it has been “trained” using machine learning. It is literally “taught” to drive similar to a way you would teach a person: you tell it what is “good” (like a fast lap time) and what is “bad” (like crashing) and it just figures out the rest. The details of that process are fascinating and explained in a paper published by Sony AI in the
Nature academic journal, but it did have some limitations in its original form. Sophy could learn how to drive a specific car on a specific track very well, but it didn’t know how to use that knowledge for anything else. “If you take the ‘single car version’ of Sophy out of that one car, it didn’t actually do a good job. When you say to it, ‘I want you to drive 300 cars,’ it’s like, ‘I don’t know how to drive these,'” Subramanian explained.
Training the agent for a specific car and track takes a lot of time, and for a game like Gran Turismo, where players expect to race against large grids of multiple opponents in a wide range of cars, that’s a problem. To solve this, the Sony AI team decided to provide the AI model with more data it could integrate into its learning. “We had to change the way we trained our agent to make sure it can drive all these cars, reliably, because they are all different. They have different power sources, different drivetrain types, and different abilities in some sense,” said Subramanian. “We told it, ‘this is how much the car weighs, this is the power source, this is the drivetrain’, and things like that, to let it know the identity of the car. Then, when you feed all that in, in addition to the changes we made to the neural network architecture, magical things started to happen. Now, Sophy knows something about the car, and the training approach was modified to allow it to learn about those things. Two weeks later, it can drive 200 different cars.”
This new technique allows the team to save a significant amount of time and resources. “If I had to train a single car, that experiment would take about 10 days,” explained Subramanian. “If I want to train for 300 cars, that would take months or even longer, and I have to keep the training time the same. Now, I can train for the same number of cars, or even more cars, within the same amount of time.”
Training Sophy 2.0
When it was first revealed to the public, Sophy was quite intimidating. It was incredibly fast around a track, easily defeating the best Gran Turismo players in the world. For Sophy 2.0, however, the focus was racing closely and fairly with other players.
“We did focus on balancing speed and competitive racing, as people know that it’s not the same as the release that we had in February,” Subramanian noted. “It’s not superhuman, and that was the intention. We wanted to focus more on making Sophy 2.0 accessible to a lot more players than we’ve ever done before. “The way we set up the training is that we essentially spawn different races, and we have to make the races relevant to what Sophy might see when it comes out into the world. There are faster cars that sometimes start at the back, sometimes there are faster cars in the front, and we sort of mix it up like that. “We try to give Sophy an experience of what it will be like when it goes outside into the world, and tell it ‘these are the kinds of races you’re going to see and you have to learn to drive in these conditions.’ We make our best guess, and of course, we don’t get it right all the time. But I think we’ve done pretty well so far.”
Teaching Sophy Right From Wrong
As the training for Sophy gets more complex, it’s not just about fast lap times anymore: clean and courteous racing is extremely important. To help Sophy know what is appropriate behavior on the race track, the Sony AI team is using a combination of both algorithms and human feedback. Subramanian explained how that human feedback loop works: “We have domain experts at Polyphony Digital, and people who know the game so well. We have input from them that tells us, ‘we would love for the car to drive in this way.’ And then we go through an experiment, and we say, ‘we’ve done it, here you go’ and they come back and say, ‘oh, Sophy did this and it’s not acceptable’. “So, then we go back and ask, what caused this? We have a list of components and rewards that tell us which reward most likely resulted in this behavior. They said ‘this was bad’, so we’ll take it out, or ‘this was OK, but maybe modify it’. Then we iteratively fine-tune it. And we get to, finally, where we are now. It’s not perfect: there are things like aggressiveness, and how Sophy behaves in that spectrum. There’s a whole range of behavior and getting that right has been a challenge.”
Slowing Down Sophy
In addition to being a courteous racer, Sophy also needs to feel like you’re racing against a real person and that means making the occasional mistake. It’s a serious area of interest and research for the Sony AI team, which has tried various techniques such as ignoring Sophy’s controller inputs at random intervals and forcing it to recover. “The version that’s in the game right now makes mistakes,” Subramaniam said. “Sometimes, Sophy loses control at the turn, or doesn’t turn at the right time and goes off track. It’s a way to make the agent feel more realistic and like you’re racing another person. “We have spent a lot of time thinking about ways in which it doesn’t have to be a perfect racer. It just has to be the racer that is going to make the race fun and more interesting. There could be many other ways in which we can do that, and that’s something that has been on our minds.”
Gran Turismo Sophy 3.0?
Research and development of Sophy remains in full swing, so I asked Dr. Subramanian what Sony AI was focused on for the next step of its evolution? “It’s been very exciting to see that players are asking for more races with Sophy. They want more tracks, more cars, and more variations in the races,” he replied. “For us, there’s a lot of room for that because there are so many tracks and there are still a few cars Sophy needs to learn how to drive. “Right now, our focus is to have Sophy usable in many more places in the game, so they don’t have to select certain tracks. They can just go in, select any track, and it sort of works out for them.”
Kazunori Yamauchi added, “At
GT7’s launch, [Sophy] could only drive a combination of a certain track and a certain car. We are working on the generalization process of that AI now. In this 2.0 version we released, that generalization process for the cars is pretty much done. It’s able to drive most of the vehicles now, aside from the really difficult-to-drive Vision GT cars. The next step would be to work on the generalization for the tracks.” Although there is clearly more work to do, it is getting easier to imagine a Gran Turismo title where Sophy is fully and transparently integrated into the single-player experience. It’s important to remember that, during its reveal, Yamauchi-san originally described Sophy in three possible “forms”: “As a teacher that will teach driving to players, a student that will learn sportsmanship from players, and as a friend to race with. I wouldn’t rule out the possibility of a B-Spec mode, where the player is the race director and Sophy is the driver.”
With Sophy 2.0 now a permanent fixture of
Gran Turismo 7 less than two years after launch, and with the rate of its development apparently accelerating, it’s feeling more likely that Sophy can live up to the lofty goals and expectations its creators established for it.
Thank you to Dr. Kaushik Subramanian and Sony AI for helping to arrange this interview. Quotations lightly edited for clarity.
