The success of British intelligence in cracking the Enigma code during the second world war and its crucial role in helping the Allies to win the war has, by now, filtered into the consciousness of today’s popular culture. What is not so well known is that the Enigma machine was a commonly available commercial product, sold for use by business as well as by the military during the interwar decades. The Nazi high command was well aware of Enigma’s potential vulnerability and commissioned its elite cryptographers and engineers to design and implement a far more powerful encryption scheme, known as Tunny. What is amazing is that British intelligence managed to break the Tunny code in addition to the Enigma code, ultimately through the construction of the Colossus, the earliest functioning example of an almost but not quite all-purpose electronic computer (not the same machine as the so-called Bombe designed by Alan Turing to break Enigma, but an even more advanced device). B. Jack Copeland, an historian and director of the Turing archive for the history of computing, along with others tells at last the story of the Colossus in Colossus: The secrets of Bletchley Park’s codebreaking computers, first published in 2006 by the Oxford University Press and based upon information declassified as recently as the first decade of the twenty-first century.
Plan of the work: not a continuous narrative, but collection of some 26 short chapters arranged in six parts, supplemented by twelve technical appendices, covering in Parts 1-2 a history of cryptography since Julius Caesar’s Gallic wars, the establishment of Bletchley Park, a description of the Tunny machine and its evolution from Enigma, a timeline of the building and operation of Colossus and then in Parts 3-6 individual treatments of the sections run by Max Newman, Ralph Tester and Thomas Flowers, who were the principal team leaders at Bletchley Park. A large cast of characters contributed in various roles and recollections from many of them form several of the chapters: for instance, the mathematicians Peter Hilton, Jack Good and Gil Hayward; WRNS [Women’s Royal Naval Service] and ATS [Auxiliary Territorial Service] members Catherine Caughey, Helen Currie, Dorothy Du Boisson and Eleanor Ireland and engineers David Bolam, Harry Fensom and Ken Meyers. The remainder of the chapters are filled out with essays from modern experts in computing and cryptography as well as historians of technology. Their contributions are fairly helpful in giving perspective on the significance of the work on Colossus for post-war developments in the computer industry and such things.
Copeland and his coauthors go into about as much technical depth as a general reader could wish for in describing how the Tunny code functions, illustrated by means of numerous worked examples. They skimp somewhat, nevertheless, on the mathematical theory behind the statistical technique of the actual algorithm employed by Colossus. Here is what this reviewer has extracted from their account, in his terms, not in the language of the authors: roughly speaking, given a large enough sample of text in a natural language, one can estimate the sampling frequency distribution of the letters of the alphabet contained in it. Now the population frequency distribution is characteristic of the language in question; for instance, in English ‘e’ is the most frequently appearing letter, followed by ‘t’ etc. Thus, if one has a guess as to the cipher, one can generate the putative clear text from it and compute its relative frequency distribution. If the guess is wrong, the result should be nonsense and the computed frequency distribution of letters in it should bear no relation to the population frequency distribution in natural language text, but if one’s guess is right, the result will be meaningful text and its sampling distribution should agree with that of the population known in advance. Therefore, if one tries out a large number of guesses for the cipher, the correct one should stand out like a sore thumb upon performing, say, a chi-squared test.
A few complications intervene in the actual scenario the cryptanalysts faced with Tunny: the texts were not written in natural German but in a language laden with military terminology, there were in addition a few special characters and, most important, even with the speed-up afforded by means of electronic computation, it was not realistic to try out every possible setting due to the immense combinatorics involved. Also, errors in transmission due to bad weather could lead to corrupted cipher text, which rendered many intercepts useless. In practice, then, some pre-analysis was required to provide the operators with shrewd priors on what the wheel settings for a given intercept in fact would be. Fortunately, the analysts at Bletchley Park could often take advantage of lapses on the part of the German radio operators. These were labeled as ‘depths’. Sometimes they would forget to dial in the new settings, or commit the serious mistake of transmitting two successive messages based on the same setting and so on. In the latter case, one could take the difference between the two and, with some cleverness and insight based upon knowledge of German vocabulary, work out good guesses for some of the wheel settings. Copeland and his coauthors go into a fair amount of detail on this. The process of decoding intercepts became more challenging as time wore on as the German high command kept raising the bar on its security procedures, such as by publishing a new code on a weekly rather than monthly basis (ultimately, on a daily basis). It is clear that the whole decryption effort would have faltered had it not been for a certain laxness in the early period of deployment of Tunny; if say, the German cryptographers had undertaken steps to identify and avoid such vulnerabilities in the first place – but the German high commanders were guilty of hubris; one is reported to have exclaimed on this very point, ‘But we’re winning the war anyway, aren’t we?’
Copeland does not have very much to say about how knowledge derived from the successful decryptions affected the course of the war – nor could he, given that most of the relevant materials have later been obliterated to protect the secrecy of the operation. Yet, he does describe one case thoroughly: he reproduces the clear text of a message sent on April 25, 1943 outlining the disposition of German forces for the upcoming so-called Zitadelle operation, which was Hitler’s last attempt to gain a breakthrough on the eastern front in southern Russia (rather like the Battle of the Bulge on the western front in the winter of 1944-1945). The failure of this campaign in July 1943, certainly largely to be attributed to the Russians’ possession of superior intelligence, was responsible for turning the tide of the entire war.
This reviewer wishes not to dilate any further on the contents of the present book – in any event, the interested reader can gain a sense of the flavor of the history, in all its aspects (technical, personnel-related, human interest and such), only by going through the work in full. For it is replete with telling incidental observations not just by Copeland himself but also on the part of numerous others, including retrospective accounts by most of the principal actors. Rather, let us suggest a few themes one might be prompted to ponder upon finishing a study of Copeland’s commendable achievement in recreating a large part of the whole story of Bletchley Park. First, one can wonder about what one has learnt about codebreaking in general. The Colossus’ method of analysis is based completely on unigrams; what could one do given knowledge of bigrams etc., or perhaps with modern techniques of natural language processing? The spies at the NSA (which forms the single largest employer of trained mathematicians in the country) must be almost unimaginably far ahead of the rest of us on this score! Second, one could deliberate on the effectiveness of the secrecy protocols enforced by British intelligence at Bletchley Park – what are the pros and cons of operating on a strictly need-to-know basis? It did cause a certain reduplication of efforts and possibly lowered morale, if it had not been for breaches of regulation mentioned by a handful of the coauthors. Feynman reports anecdotally regarding his participation in the Manhattan project that the lower-level scientists and engineers were motivated to work harder and perhaps more savvily once they had been clued in to what the overall project was about and what its technical goals were. Copeland offers nothing about how the decrypted intercepts could be used to gain an advantage in the conduct of the war without giving away to the German commanders the fact that their code had been compromised – in any event, decisions of this nature lay outside the purview of the codebreakers at Bletchley Park. The reader will also want to reflect on the crucial role played by oversights both on the part of German high commanders and on the part of lowly radio operators: though their security procedures were steadily tightened during the course of the war, these very human shortcomings proved ultimately decisive in losing the war for the Nazis. Another interesting point: Copeland declines to speculate on why Churchill ordered the Colossus machines and records of decoded intercepts destroyed after the cessation of hostilities in the European theater. Just a conjecture, but he may have wished to erase from the historical record his role in underwriting the Allies’ inaction with respect to the Holocaust (the Allied leaders felt that their strategic air forces were too tied up by the all-out effort to commit war crimes against German civilians for them to have any spare capacity to dedicate to a fight to stop the Nazis from committing war crimes of their own against Jews).
Most of all, Copeland’s solid history of the Colossus conveys a feeling for what it is like to wage war: one has to exploit literally every opportunity or loophole at one’s disposal in the fight to wrest a marginal advantage over the opponent – this reviewer finds it fascinating how the British cryptanalysts could, for instance, rely on apparently contingent features of the very letter assignments in the binary table used by Tunny to streamline their algorithms just enough to derive a little extra edge. To be canny and wily counts for more than any number of armored divisions!