AI Learns To Outsmart Humans In Video Games – And Real Life in 2023

Driving around a French village in Gran Turismo, you might notice a Corvette trying to catch your slipstream.

Skilled players of PlayStation’s realistic racing game prefer to use the draft of an opponent’s racecar to speed up and overtake them.

But this Corvette driver isn’t being guided by a human — it’s GT Sophy, a powerful artificial intelligence agent created by Sony.

Gran Turismo players have been competing against computer-generated racecars since the franchise’s inception in the 1990s; the new AI driver released last week on Gran Turismo 7 is smarter and faster because it was trained using the most advanced AI methods.

“Gran Turismo had a built-in AI from the start, but it has a very narrow performance band and isn’t very good,” said Sony AI’s chief operating officer Michael Spranger. “It’s extremely predictable. It no longer entices you once you’ve reached a certain level.”

But now, he says, “this AI is going to fight back.”

When you visit an artificial intelligence laboratory at a university or a company like Sony, Google, Meta, Microsoft, or ChatGPT-maker OpenAI, it’s not uncommon to see AI agents like Sophy racing cars, slinging angry birds at pigs, fighting epic interstellar battles, or assisting human gamers in creating new Minecraft worlds – all part of the job description for computer systems trying to learn how to get smarter in games.

it’s not uncommon to see AI agents like Sophy racing cars

However, in some cases, they also attempt to learn how to become smarter in the real world. A University of Cambridge researcher who created an AI agent to control Pokémon characters argued in a January paper that it could “inspire all sorts of applications that require team management under conditions of extreme uncertainty, such as managing a team of doctors, robots, or employees in an ever-changing environment, such as a pandemic-stricken region or a war zone.”

While this may sound like a child arguing for three more hours of Pokémon Violet, game research has been used to advance AI research — and train computers to solve complex problems — since the mid-20th century.

Initially, AI was used to test winning strategy games such as checkers and chess. A new field of study now focuses on performing open-ended tasks in complex worlds and interacting with humans rather than just beating them.

“Reality is like a super-complicated game,” said Nicholas Sarantinos, who co-wrote the Pokémon paper and recently turned down a doctoral offer at Oxford University to launch an AI company to assist corporate workplaces in forming more collaborative teams.

Tarantino created an algorithm to analyze a team of six Pokémon in the web-based Pokémon Showdown battle simulator, predicting how they would perform based on all of the possible battle scenarios ahead of them and their comparative strengths and weaknesses.

That real humans behave very differently than fictional video game creatures,

Microsoft, which owns the popular Minecraft game franchise and the Xbox game system, has given AI agents tasks ranging from avoiding lava to chopping trees and building furnaces. Researchers hope that some of their discoveries will eventually play a role in real-world technology, such as how to program a home robot to do certain chores.

While it “goes without saying” that real humans behave very differently than fictional video game creatures, “the core ideas can still be used,” Tarantino says. “If you use psychological tests, you can use this information to determine how well they can collaborate.”

Amy Hoover, an assistant professor of informatics at the New Jersey Institute of Technology who developed algorithms for the digital card game Hearthstone, stated that “there is a reason for studying games,” but it is not always obvious.

“People don’t always get that the point is about the optimization method rather than the game,” she explained.

According to Vanessa Volz, an AI researcher at the Danish startup Modl.ai, which builds AI systems for game development, games also provide a useful testbed for AI, including some real-world robotics or healthcare applications that are safer to try in a virtual world.

“It can get overhyped,” she adds.

“It’s unlikely that there will be one big breakthrough, and everything will be shifted to the real world,” Volz predicted.

Sony launched its own AI research division in 2020

Sony launched its own AI research division in 2020 with entertainment in mind, but it has nonetheless attracted broader academic interest. Its research paper introducing Sophy was featured on the cover of the prestigious science journal Nature last year, with the journal stating that it could have implications for other applications such as drones and self-driving vehicles.

Sophy’s technology is based on an algorithmic method known as reinforcement learning, which trains the system by rewarding it when it gets something right as it runs thousands of virtual races.

“The reward will tell you that you’re making progress. ‘This is good,’ or ‘You’ve gone off the rails. “Well, that’s not good,” I say. Spranger explained.

At tournaments, the world’s best Gran Turismo players continue to finish ahead of Sophy, but average players will find it difficult to beat — and can adjust difficulty settings depending on how much they want to be challenged.

PlayStation players can only race against Sophy on a limited number of circuits until March 31, so it can gather feedback and return to testing. According to Peter Wurman, director of Sony AI America and project lead on GT Sophy, training AI agents on 20 PlayStations takes about two weeks.

“It will take some more breakthroughs and time before we’re ready,” he said.

And getting it onto real-world streets or Formula One track? That could take a long time.

Autonomous vehicle companies use similar machine-learning techniques, but “they don’t hand over complete control of the car the way we can,” Wurman said. “In a simulated world, no one’s life is in danger. You know exactly what you’re going to see in the environment. There are no cars on the road or anything like that.”

SOURCE – (AP)