The computer brain plays poker like the pros

Artificial intelligence is conquering more and more human domains – especially when it comes to games. The AI ​​program developed by American researchers has proven its strength in the game of poker against five other players for the first time. The AI ​​”Pluribus” has outperformed world-class poker players and has won more games than they have. Scientists see this as an important advance in AI research, as their system has tackled two challenges at once: playing against multiple opponents, and the fact that you don’t know your opponent’s poker layout.

Strategy games such as Chess, Go, and the Japanese Shogi Chess Variant are considered to be a particular challenge to the human mind – and a good test model for artificial intelligence. Because in these games you have to memorize and combine complex moves and strategies in order to win over your opponent. Thanks to neural networks and adaptive algorithms, artificial intelligence systems like AlphaGO have beaten even high-level professionals in these strategy games in recent years. The programs tend to get high play in a short amount of time, playing against each other over and over again based on the basic rules of the game. However, chess and Go and Co have one thing in common: in these games, each player sees his opponent’s playing positions – they are games with so-called “perfect information”.

Against five players at once

It is different in card games such as poker: here you either do not know your opponent’s hand at all or only partially. This makes it much harder to develop a strategy – similar to an equation with a few unknowns. In addition, “bluffing” also plays an important role in poker: it is about not revealing whether you have a good hand or a bad hand in order to convince other players to bet more money. For AI, this has long been considered an insurmountable challenge. But in the meantime, artificial intelligence systems have also achieved this feat: a few years ago, the computer won poker for the first time. In 2017, a system called DeepStack appeared in the “Heads-Up No-Limit Texas Hold’em” variant of the game. Each player has two cards that only he or she knows, the rest of the hand is dealt consecutively to open in three consecutive rounds.

The “Pluribus”, an artificial intelligence system developed by Noam Brown and Tuomas Sandholm of Carnegie Mellon University in Pittsburgh, goes one step further. Because this adaptive machine brain not only plays poker with one player like its predecessors, it can also win rounds of poker against five other players. “Until now, AI milestones in strategic thinking have always been limited to competition between the two sides,” says Brown. “Playing a game with six players requires fundamental changes to the way AI develops its game strategy.” Because in two-player games, the so-called Nash equilibrium usually applies, as the researchers explain: Long-term chances of success are best when the player sticks to their strategies and no one fundamentally changes the way the game is played. However, in games with more than one opponent, this no longer applies.

Win against world-class poker players

The Pluribus AI system solves this problem by playing over and over again against its copies, making it get better and better. Based on this experience, Pluribus develops a kind of strategy plan – a predetermined way of playing, by which he starts playing poker with five other players. “While playing, Pluribus then optimizes its strategy by looking for a better real-time solution to the current game situation,” reported Brown and Sandholm. The pluribus can even solve the problem of bluffing: “If a player only increased when he had the best hand possible, his opponents would know that in that case they would always have to fold,” explain the researchers. The AI ​​handles this by considering the likelihood of a bet regardless of your hand in every decision. This allows her to see that even if the bet is bad, it can still be a win-win.

Artificial intelligence has proven how well Pluribus actually plays poker in two tournaments against world-class professional players. “The bot plays against some of the best poker players in the world,” points out Darren Elias, the professional who holds the record for the most titles in the world of poker tours. In the first tournament, the AI ​​played 10,000 rounds against five human players in twelve days. In the second tournament, five copies of Pluribus were played against one professional player. Result: The Pluribus won much more than its competitors and made significantly more money. “His greatest strength is his ability to combine strategies,” says Elias. “It’s the same thing people do.”

According to Brown and Sandholm, Pluribus has reached another milestone in artificial intelligence research. “Being able to beat five other players in such a complex game opens up a whole new world of AI to real problems,” says Brown. However, other AI researchers are looking at it a little more critically: “Such successes are achieved in very specific tasks,” comments Andreas Holzinger, computer scientist at the University of Graz. However, in his opinion, these scenarios cannot be easily transferred to other application areas. Kristian Kersting from TU Darmstadt puts it even more strikingly: a person who can play poker very well is not necessarily a stock market genius. ” Nevertheless, the success of Pluribus can serve as an inspiration to solve many exciting questions.

Source: Noam Brown and Tuomas Sandholm (Carnegie Mellon University, Pittsburgh), Science, doi: 10.1126 / science.aay2400

Leave a Comment