Self-taught computer program finds super poker strategy

A computer program that taught itself to play poker has created nearly the best possible strategy for one version of the game, showing the value of techniques that may prove useful to help decision-making in medicine and other areas.

The program considered 24 trillion simulated poker hands per second for two months, probably playing more poker than all humanity has ever experienced, says Michael Bowling, who led the project.

The resulting strategy still won't win every game because of bad luck in the cards. But over the long run - thousands of games - it won't lose money. "We can go against the best (players) in the world and the humans are going to be the ones that lose money," said Bowling, of the University of Alberta in Edmonton, Canada.

The strategy applies specifically to a game called heads-up limit Texas Hold 'em.

While scientists have created poker-playing programs for years, Bowling's result stands out because it comes so close to "solving" its version of the game, which essentially means creating the optimal strategy.

Poker is hard to solve because it involves imperfect information, where a player doesn't know everything that has happened in the game he is playing - specifically, what cards the opponent has been dealt.

Many real-world challenges like negotiations and auctions also include imperfect information, which is one reason why poker has long been a proving ground for the mathematical approach to decision-making called game theory.

Tuomas Sandholm of Carnegie Mellon University in Pittsburgh, who didn't participate in the new work, called Bowling's results a landmark. He said it's the first time that an imperfect-information game that is competitively played by people has been essentially solved.

Bowling's paper, released Thursday by the journal Science, introduces some techniques that could become useful for applying game theory in real-world situations. Bowling is investigating the possibility of helping doctors determine proper insulin doses for diabetic patients, for example.

Game theory has also been used to schedule security patrols, and it has implications for other areas like developing strategies for cybersecurity, designing drugs and fighting disease pandemics.

But Bowling doubts the poker strategy will let anybody make a fortune on the game itself. The kind of poker it applies to has waned in popularity over the past seven years or so, he said. Even online, the stakes tend to be small and "you'd be winning a few dollars, not raking in millions."

In the two-player game, each contestant creates a poker hand from two cards he is dealt face-down plus five other cards placed on the table face-up. Players place bets before the face-up cards are laid out, and then again as each card is revealed. The size of the wagers is fixed.

Bowling said the computer's strategy is far too complicated for anybody to memorize, with about 1,000 times the amount of information in the English-language Wikipedia. But his university has created a website where people can ask it for advice and even play against it.