IBM Corp researcher Gerald Tesauro has developed a self-teaching programme, TD-gammon, which, with the aid of neural network technology has learned to play backgammon well enough to beat experts – from scratch. The network uses the temporal-difference algorithm created by Richard Sutton of GTE Corp’s GTE Laboratories in Waltham, Massachusetts, which is a kind of delayed reinforcement mechanism. Tesauro’s program runs on an RS/6000 workstation, and sets out the rules and objectives of the game and the layout of the board; the machine is then left to develop its strategy, through trial and error, by playing against itself. It assesses the viability of each potential move against the values stored in the algorithm; these values are adjusted after each game, either upwards when it wins or downwards when it loses – a process that leads to significant improvements in performance after only 100 games. The RS/6000 enables some 300,000 games to be played in a month – vital because of the mere 25,000 synapses contained in the Tesauro program, which is equivalent in complexity to the brain of a slug. The game been declared one of the best by expert backgammon players – the machine recently achieved a 19-19 draw against experts at the World Backgammon Cup tournament. In addition, Tesauro claims it has inspired research into the application of similar techniques to automatic-control problems, such as guiding robot hands and arms, and to prediction problems, such as forecasting trends in the behaviour of financial markets, and has thrown light on human and mechanical learning processes.
