Results of the Computer Go tournament at the 2008 US Go Congress in Portland, OR, USA can temporarily be found at: http://svcs.cs.pdx.edu/cgo2008.
I would like to thank: Hierarchical Systems Research Foundation for providing the bulk of the $1250 prize and travel expense money (the rest was donated anonymously); Bart Massey, Kathi Lee, and everyone at PSU for providing a physical venue and helping with technical setup; Bill Shubert and everyone at KGS for providing a virtual venue; and all of the programmers and operators involved.
The tournament was a double round-robin tournament, so each program played as W and B against each opponent. All games were 19x19, 45 minutes per side sudden death, Chinese rules, 7.5 komi. Here are the results in traditional round-robin format:
1 2 3 4 5 6 7 | Total
-----------------------------------
1. GNU Go XX 11 01 11 11 11 11 | 11
2. Many Faces 00 XX 11 11 11 11 11 | 10
3. Leela 10 00 XX 11 11 11 11 | 9
4. House Bot 00 00 00 XX 11 01 11 | 5
5. First Go 00 00 00 00 XX 11 11 | 4
6. Orego 00 00 00 10 00 XX 11 | 3
7. Butter Bot 00 00 00 00 00 00 XX | 0
The tournament was surprisingly smooth. Every game but one ended in resignation.
SlugGo was not able to attend due to numerous problems encountered while assembling a new hardware cluster. We decided to enter GNU Go (3.7.10, level 12) instead, as GNU Go had only refrained from entering due to the expected presence of SlugGo.
The version of ManyFaces used includes Monte Carlo search. In fact, we believe GNU Go was the only non-MC program in the tournament.
In ManyFaces-Leela (that is, the game with ManyFaces as white and Leela as black), David Fotland discovered that his T61 laptop was unplugged, so ManyFaces was running at half speed to conserve power. The laptop was properly plugged in mid-game and ManyFaces went on to win the game anyway.
In Leela-GNU, KGS reported a win for GNU under Japanese rules, because we had failed to set one of our KgsGtp configuration files to specify Chinese rules. Since both programs believed that they were playing under Chinese rules, we re-scored under Chinese rules and found the game was a win for Leela (as Leela had reported).
In FirstGo-GNU, FirstGo played out a ladder.
In Leela-FirstGo, FirstGo played out a short ladder. It began to run when caught in a second, much longer broken ladder; 13 moves were played before Leela abandoned the chase.
In FirstGo-Orego, Orego resigned after 76 moves – probably a bit premature.
After the event, Fotland had hew new multithreaded version of ManyFaces up and running. It played three games against GNU Go and won all of them.
We definitely want to do this again at next year's Go Congress in Washington, DC. Is there anyone in that area willing to direct the tournament?
Double round-robin ran extremely smoothly. We were able complete our games in less time than in a tournament with fixed rounds, because a new game could start whenever one ended. In fact, we were able to run some programs on multiple machines, thus completing some games in parallel. If there are a lot of entrants, perhaps they could be filtered by a Swiss tournament or the results of previous tournaments before playing round-robin among the strong programs.
The TD's program (Orego) was ineligible to win to avoid any appearance of conflict of interest. In the future, we would do what has been done in the past: appoint a deputy director or committee to make any decisions regarding the TD's program.
If the prize money is very large, it may be important to require that source code be made available for inspection by the TD to avoid any question of plagiarism.
We should do a better job of making rules explicit, e.g., all results stand, even if it is discovered afterward that a program was running with incorrect parameters. (We discovered that FirstGo was running with the incorrect time setting; the result stood.)
It would be good to have some talks by the programmers involved.
The expense of travel is a serious impediment to attendance. Some of our prize money was used to defray such costs. It would be better to clarify the distribution of prize money in advance. Perhaps there could be one pool of prize money for everyone (with more money for programs that place higher, of course), and a second pool for those programmers who travel, with appropriate consideration to prevent a trivial program being entered just to collect travel money while the "programmer" just attends the Congress.
Our intent is to encourage interesting interactions between programmers. There were useful conversations in the room as the tournament ran. It was great when Anders Kierulf dropped in even though his program SmartGo was not entered.
All the above was provided by Peter Drake.
On Thursday 7th August, at the US Go Congress, there was a demonstration game between Myungwan Kim 8 and MoGo. The following report is taken from the AGA Newsletter.
In a historic achievement, the MoGo computer program defeated Myungwan Kim 8P (l) Thursday afternoon by 1.5 points in a 9-stone game billed as "Humanity's Last Stand?". "It played really well," said Kim, who estimated MoGo's current strength at "two or maybe three dan," though he noted that the program – which used 800 processors, at 4.7 Ghz, 15 Teraflops on a borrowed European supercomputer – "made some 5-dan moves," like those in the lower right-hand corner, where Moyogo took advantage of a mistake* by Kim to get an early lead. "I can't tell you how amazing this is," David Doshay – the SlugGo programmer who suggested the match – told the E-Journal after the game. "I'm shocked at the result. I really didn't expect the computer to win in a one-hour game."
Kim easily won two blitz games with 9 stones and 11 stones and minutes and lost one with 12 stones and 15 minutes by 3.5 points. The games were played live at the U.S. Go Congress, with over 500 watching online on KGS. "I think there's no chance on nine stones," Kim told the EJ after the game. "It would even be difficult with eight stones. MoGo played really well; after getting a lead, every time I played aggressively, it just played safely, even when it meant sacrificing some stones. It didn't try to maximize the win and just played the most sure way to win. It's like a machine." The game generated a lot of interest and discussion about the game's tactics and philosophical implications.
"Congratulations on making history today," game organizer Peter Drake told both Kim and Olivier Teytaud, one of MoGo's programmers, who participated in a brief online chat after the game. At a rare loss for words in a brief interview with the EJ after the game, Doshay wondered "How much time do we have left? We've improved nine stones in just a year and I suspect the next nine will fall quickly now."
reported by Chris Garlock
* not a mistake, a deliberate overplay, often necessary in a high-handicap game.
Guillaume Chaslot's page about the game.
The SGF record of the Kim-MoGo game as relayed on KGS.