AI driver can beat some of the world’s best players at Gran Turismo

An AI trained to win the PlayStation 4 game Gran Turismo Sport has beaten some of the world’s best players by learning to handle the most difficult parts of racing tracks

By Carissa Wong

An artificial intelligence has beaten four of the world’s best human drivers on three different tracks in the racing video game Gran Turismo Sport, by gaining ground at the most difficult parts of a track.

The AI, named GT Sophy, was able to execute tactical moves such as using an opponent’s slipstream to boost itself forwards and block its opponents from passing.

Peter Wurman at Sony AI in New York and his colleagues trained the system using deep reinforcement learning, a type of machine learning that uses rewards and penalties to teach the AI’s neural network how to win. During training, GT Sophy, which was running on a separate computer, played the game on up to 20 PlayStation 4 consoles simultaneously.

The team gave the AI the ability to accelerate, brake and steer, along with real-time information on the position of the cars in the game, including its own, and a map of the next 6 seconds of the track, which meant sight of a longer distance ahead when the AI travelled faster. The researchers note this information isn’t available to human players, giving the AI a slight advantage, but say that people have other advantages, such as the ability to change gears manually, which the AI didn’t have.

Wurman and his team rewarded GT Sophy for staying on the course and driving faster, and penalised the AI for veering off course or slowing down. Within a couple of days, the AI learned to complete the tracks faster than 95 per cent of human players, compared with rankings from online leader boards.

Over around nine more days of training, GT Sophy shaved tenths of a second off its lap times over a total of 45,000 driving hours to finish faster than any human player on the leader boards.

However, GT Sophy hit a stumbling block. The AI initially avoided overtaking fast opponents in order to minimise the risk of collision and maximise rewards. To overcome this overly safe behaviour, researchers rewarded the AI for passing opponents and penalised it for being overtaken.

By joining forces with a competitive GT player, the team focused on a small set of the most difficult parts of each track, so that GT Sophy could quickly learn to excel in those areas. The researchers then challenged four of the world’s best GT players to compete against four copies of GT Sophy in a team race, so eight cars were on the track in all. The AI won 104 to 52, with points calculated according to racers’ final positions.

“The results suggest that it could be possible for game developers to use deep reinforcement learning to design and test their games, and to produce interesting opponents and teammates for human players,” says Igor Babuschkin at OpenAI in San Francisco.

The findings will have little impact on improving autonomous vehicles, which must sense the environment themselves and navigate more variable conditions.

“While GT Sophy managed to achieve impressive results, it’s very much dependent on fine-tuned parameters and features specific to the game,” says Georgios Yannakakis at the Institute of Digital Games in Malta.

Journal reference: Nature, DOI: 10.1038/s41586-021-04357-7

Join us for a mind-blowing festival of ideas and experiences. New Scientist Live is going hybrid, with a live in-person event in Manchester, UK, that you can also enjoy from the comfort of your own home, from 12 to 14 March 2022. Find out more.