ICGA/Rybka controversy: An interview with David Levy (1)

ICGA/Rybka controversy: An interview with David Levy (1)

Introduction

In June 2011 it was widely reported in the global media that the International Computer Games Association (ICGA), headed by Dr. David Levy, had found chess programmer Vasik Rajlich in breach of the ICGA’s annual World Computer Chess Championship (WCCC) tournament rule related to program originality. In the ICGA’s accompanying report it was asserted that Rajlich’s chess program Rybka contained “plagiarized” code from Fruit, a program authored by Fabien Letouzey of France.

Rajlich, who cruised to victory in four consecutive WCCC tournaments in 2007, 2008, 2009 and 2010, was retroactively stripped of all titles he had won in competition and was slapped with a lifetime ban. He claimed complete innocence and made it clear that he found the ICGA’s investigatory process and conclusions to be biased and unprofessional, and the charges baseless and unworthy. Rajlich refused to be drawn into a protracted dispute with his accusers or mount a comprehensive defence. This was done by Dr Søren Riis, a computer scientist from London, in a paper we published in four parts. There was vigorous feedback from the chess community, including a rebuttal by the ICGA. Links to the relevant articles are given at the bottom of this page.

In his rebuttal David Levy agreed to do a full one-on-one exclusive interview with ChessBase on the entire ICGA/Rybka scandal. It was conducted by sequential emails, and took a fair amount of time, since David was consulting with experts who live in different time zones around the globe.

Interview with David Levy on the ICGA/Rybka scandal

ChessBase: David, do you believe that the ICGA has conclusively proved that Vasik Rajlich copied actual code (as opposed to ideas and algorithmic techniques) from the Fruit program. Why?

David Levy: At the outset I wish to make it clear that this is not an issue about verbatim copying. We have proven that Rybka is a derivative, but due to different internal board representations the Fruit code required modification when being copied. But a conversion such as mailbox to bitboard still leaves it a derivative, as would changes such as refactoring the code and tuning the numbers.

The forums continually mention code copying instead of Rajlich failing to report a derivative in his ICGA entry submissions. It is important to remember that the offence for which Rajlich was convicted and banned by the ICGA was failing to comply with our Tournament Rule 2. The various reports (here and here) show that Rybka 1.0 beta and Rybka 2.3.2a were clearly derivatives of Fruit 2.1. Postings on the forums and Riis stating that merely retyping or converting a program to another format are clearly wrong. Merely retyping and/or converting code from one format to another is still making a derivative, as too are some changes that affect game play (such as evaluation tuning). So yes, the Rybka entries were derivative of the Fruit code in a manner that broke Rule 2. And Rajilch says that “Rybka is original at the level of source code” meaning “the author either typed his own code or typed the code which generated his own code”, going so far as to say that Rybka derivatives which he had previously censured are “original at the source code level” if they were typed by hand. If this were the case then, if instead of cutting and pasting a programmer just re-types by hand what he has stolen, then according to Rajlich it is original and does not violate ICGA or copyright rules! Really?

In which versions of his program did this occur?

Rybka versions from 1.0 Beta through 2.3.2a were shown to be clear derivatives of Fruit 2.1.

Did any of these versions (for which there is definite proof that they contain derivatives Fruit code) play in an ICCA or ICGA event?

Yes, Rybka 2.3.2a definitely competed in ICGA events. The evaluation of Rybka 2.3.2a clearly was derived from Fruit 2.1. Note that the report states “Rybka 2.3.2 was released Jun 13, 2007, and 2.3.2a about five days later. Rybka competed in and won the ICGA Chess Tournament (Jun 11-18 2007)”. ICGA analysis of several versions of Rybak between 1.0 beta and 2.3.2a show they also contained a derivative of the Fruit 2.1 evaluation function. During this period a version of Rybka 2.0 competed in the 2006 ICGA World Computer Chess Championship tournament (May 2006) under the name “Rajlich”. The closest commercial release was the MP-only Rybka 2.0 Beta 4 with a release date of May 17 (Beta 4).

During the investigation process one member of the ICGA investigation secretariat asked Rajlich “1. Are you able to provide us with the exact .exes that played in all ICGA events particularly those from 2006 and 2007? If not, is it a fair approximation to use Rybka 2.3.2a for the 2007 event?” On the Rybka Forum, Rajlich said “Yes” to the second question, confirming that this version was used in an ICGA event.

Were any sold commercially?

Yes, see above. Rybka 2.0 Beta 4 and Rybka 2.3.2a were sold commercially.

What do the ICGA experts think about the current versions of Rybka (i.e. versions 3 and 4)? Do they also contain code taken from Fruit?

The ICGA has not done extensive analysis of newer versions of Rybka. It was sufficient for the ICGA to determine that Rajlich had broken Tournament Rule 2, which we did. When the ICGA chose to ban Rybka for life due to violating the rules in ICGA events in 2006 and 2007, this required all of Rajlich’s programs be removed, since the newer programs would never have been allowed to compete if the violation of earlier versions was discovered sooner and the lifetime ban issued earlier. So while it is possible that newer versions of Rybka also contained Fruit-derived code, it was not essential to prove this once we had proved the case for the 2006 and 2007 ICGA tournaments.

If Rybka was, as you put it, a derivative of Fruit 2.1, how do you explain that it was massively stronger? Normally by tuning an existing program, even your own, you gain 30 point per year. We know that from all our engine programmers. 150 points is unheard of.

The panel did not do much examination as to what made Rybka stronger, so I would suggest you ask Rajlich directly. But experiments with Strelka, which Vas has said is a clone/derivative of Rybka, show gains due to bitboard speedup, 64 bit code being more efficient than 32 bit code, the addition of a form of futility using verification by a qsearch, and tuning the Fruit evaluation. Harder to test is effect of the the Rybka material imbalance tables, but they could have caused some gain as well.

If you compare the 32 bit version of Rybka 1.0 (Elo 2885) to Fruit 2.1 (Elo 2793), the Elo difference is only 92. The speedup going to 64 bit code (which benefits bitboard programs more than mailbox) is about 34 Elo. In versions of Rybka higher than 1.0, there is speculation that more strength advancement came through additional evaluation tuning, and search changes.

The use of Fruit gave Rajlich a huge head start, as he only had to improve a 2793 Elo program instead of writing one from scratch. To get some idea of what small changes to a strong program can do, consider what an amateur programmer did with Fruit in creating Toga II (an admitted Fruit derivative). He was able to add 135 Elo in strength to the Fruit code base, bringing it to an Elo of 2928. The ratings we quote are from are here.

Full article here.