The American Black Chamber

by Herbert O. Yardley

During the 1920s Herbert O. Yardley was chief of the first peacetime cryptanalytic organization in the United States, the ancestor of today's National Security Agency. Funded by the U.S. Army and the Department of State and working out of New York, his small and highly secret unit succeeded in breaking the diplomatic codes of several nations, including Japan. The decrypts played a critical role in U.S. diplomacy. Despite its extraordinary successes, the Black Chamber, as it came to known, was disbanded in 1929. President Hoover's new Secretary of State Henry L. Stimson refused to continue its funding with the now-famous comment, "Gentlemen do not read other people's mail." In 1931 a disappointed Yardley caused a sensation when he published this book and revealed to the world exactly what his agency had done with the secret and illegal cooperation of nearly the entire American cable industry. These revelations and Yardley's right to publish them set into motion a conflict that continues to this the right to freedom of expression versus national security. In addition to offering an exposé on post-World War I cryptology, the book is filled with exciting stories and personalities.

Genres: History, Nonfiction, Espionage

375 pages, Paperback

First published January 1, 1931

Reviews

[Deana · Rating 4 out of 5]

"The American Black Chamber" is a really interesting book about the organization that preceded today's National Security Agency (NSA) in the United States. The author, Herbert O. Yardley, founded the nation's first cryptologic bureau, when he discovered that enciphered messages could usually be decrypted using logic and mathematics techniques.

The book gives a really interesting history, as well as showing some of the cryptographic messages and simplified explanations of how they can be solved. There are even some messages left as an exercise for the reader to solve.

Ever since I was a small child, I was fascinated by "secret codes". I had memorized the rot-13 alphabet as well as the pigpen cipher and often wrote in my diary using these encryption schemes to try and thwart my brother. I had books and books of how to make various secret codes and tried all of them. I made my own code books, where one English word stood for other words such that a sentence meant something very different from what it said. But in none of these books did I ever learn how to DECRYPT these messages without the key. I did, however, love the "cryptograms" in puzzle books -- but of course those are just simple substitution ciphers. If you were like me in those respects, you will really enjoy this book!

Yardley tells a good story. He knows how to make it interesting. Though I will also say that his attitude is really obnoxious -- it's obvious he thinks he is the best thing since sliced bread and that he looks down on just about everyone else. Other sources say that many of the stories in this book are exaggerated to make a better story, and should not be taken at face value as the absolute truth of what happened, and based on the story telling this doesn't surprise me either. But it was still a fun and very interesting read.

Though I must say, my favorite part is the footnote that says:
"Soviet agents, please note. Yes, I once had copies of these documents, but I don't care to have my throat cut and do not plan to publish them. In fact they have been destroyed. So be reasonable."

SERIOUSLY!?! That just gives you a sense of how amazing he believes himself to be. And it shows a lot in the book. But still entertaining and interesting.

[Jean-Luc · Rating 4 out of 5]

When the United States stumbled into World War 1, the War Department and the State Department were under the impression that their ciphers could not be broken by foreign spies. Herbert Yardley, author and hero of the this secret history, demonstrated to the head of military intelligence that American codes were crap. The chief was so impressed that he quickly placed Yardley in charge the US Army's cryptography department, MI-8 aka the American Black Chamber. But American wartime leaders weren't nearly as interested in creating strong ciphers for American diplomats/spies/generals to use as they were in breaking other country's codes...

Right away, Yardley dives into the nitty gritty of setting up a crack team of cryptanalysts. But reading other nations' telegrams was only part of the story. Plenty of secret messages were hidden in ordinary snail mail using invisible ink, and Yardley's team had to figure a way to detect that ink without alerting the recipients that they were under suspicion. Of course, the Germans invented new methods on the fly: proof that an invisible ink was no longer invisible came when spies were aprehended and executed, so they had plenty of incentive to innovate! Yardley's stories include tracking the notorious German spy Madame Maria de Victorica, infiltrating the Mexican embassy to steal their codebook, forcing the Japanese to accept American-favored limits on the size of their Navy, and more.

After World War I ended, the department was rolled into the State Department as the Cipher Bureau, and they continued their cryptographic operations in New York City. Yardley and his team continued to rack up wins, up until Herbert Hoover was elected president. He appointed Henry Stimson as his Secretary of State, and Stimson's attitude was "Gentlemen do not read other gentlemen's mail." The Cipher Bureau was dismantled and the US military took over responsibility for cryptographic stuff, but wihtout Yardley at the helm, the United States would remain blind to other governments' intentions... Which may explain why America was so blindsided on December 7, 1941.

A few additional points:

* Edgar Allan Poe's The Gold Bug seemed to have influenced quite a few people into becoming amateur cryptographers. I wonder what the modern equivalent is?

*

The Espionage Act had recently passed, making it a crime to publicly oppose the American involvement in World War I.

If it sounds like a complete mocker of the 1st Amendment, that's because it is. That same Espionage Act is still in play and has been used against famous whistleblowers such as Edward Snowden, Chelsea Manning, and Daniel Ellsberg.

*

A few days later I received a most interesting letter from Bacon which reveals another side of the story.

Yardley's willingness to admit he's not the center of the universe lends credibility to the veracity of his text!

*

Let us be fair to the German cryptographers. Perhaps German officials are like our own and do not take the recommendation of cryptographers as seriously as they should.

Nothing ever changes.

*

'Of course I should warn you to be discrete and all that sort of thing, but I don't think it is necessary.' 'It isn't,' she said simply.

After all the ink spilled on the weakness and unreliability of female spies, Miss Abbott seemed singularly designed to change our minds about that.

*

The reader may well appreciate the shock I received as I deciphered a telegram which reported an Entente plot to assassinate President Wilson either by administering a slow poison ... But these are the undeniable facts: President Wilson's first sign of illness occurred while he was in Paris

I'd never before heard anyone claim Woodrow Wilson was assassinated. o_O

*

I had already discussed with General Churchill the advisability of subsidizing the Oriental Language Department of some university

And if you look @ the CIA and the NSA today, you can see how well this plan was followed: not at all. Nothing ever changes!

*

Now the Mexican attempt to mislead the Department of State was, curiously the one that would have no effect, for plain text messages were never intercepted! We had always assumed that plain text messages did not contain information worth considering.

This is of course exactly how the Bataclan attackers in Paris were able to elude the authorities.

*

Sometimes I wondered if the government believe we deciphered telegrams by machinery.

Foreshadowing!

*

If the Department considered the code messages of foreign governments inviolate, then inviolate they must remain. It would be usurpation of power on the part of the War Department if it engaged in activities against the policies of the State Department.

The idea that no one is above the law is considered quaint these days. Damn shame.


The US govt was not pleased, to put it mildly, that Yardley published this history. But that's kind of the best part: as with the CIA's general crankiness over Legacy of Ashes, their anger at revealing these secrets gives this book a huge boost in credibility. And unlike Legacy of Ashes, this book is readable and entertaining: thrilling tales of American derring-do! People who like that Jason Bourne crap probably won't have the patience to appreciate this book, but I strongly recommend it for anyone who enjoys American history or old-school spy thrillers.

[Ensiform · Rating 4 out of 5]

The author headed the titular Black Chamber, a euphemism for MI-8, the branch of US Intelligence that dealt with ciphers, codes and “secret inks” through WWI and beyond, until the branch was summarily closed by a spectacularly naive and short-sighted Secretary of State. Hardley is a fine raconteur, detailing step-by-step the painstaking ways he and his staff decoded, for example, messages that were composed solely of long strings of five-digit numbers (which turned out to be references to a dictionary’s page and line numbers).

The decryption of the Japanese codes, too, considering the lack of available information on the language itself, is incredible. There’s a long stretch of intercepted and translated Japanese telegrams which I suppose Yardley included to make a political point, but which is a bit dreary, when all one wants is more info on how he cracked the codes. Aside from that, it’s fascinating, not only the code breaking but the period detail: how legions of typists and thousands of cards were needed in those days before computers or the level of government spying: letters and cables read as a matter of course, people followed and observed... And here we are whining about the Patriot Act.

[Morgan · Rating 5 out of 5]

Ever wonder what spies did in the 1930's? Now's your chance to find out from an insider. Written in a purposefully glamorized style, this book is what happens when the government abruptly fires the man responsible for a lot of their secret communication. haha.

[Kevin DeLong · Rating 4 out of 5]

I really got into this book. Made codes and ciphers interesting and illustrated their practical use in national affairs. Some parts got complicated or wordy, but I still liked it. I could imagine myself their with the author in history. Sad ending.

[Robert · Rating 4 out of 5]

Herbert Yardley’s “The American Black Chamber” is a probably unique study in the history of the intelligence world as an individual who played a critical role in practically inventing American ciphers and codebreaking over a period of twelve years told virtually all about that work. Starting his career in 1917 as the United States was moving towards war, his career in this field came to an end in 1929 when Herbert Hoover’s Secretary of State, Henry Lewis Stimson, declares that “Gentlemen do not read each other’s mail” – apparently forgetting that his foreign counterparts had not attended Phillips Academy or Yale – and orders the Black Chamber immediately dissolved. Yardley gets some small revenge in his descriptions of a number of State Department officers and senior officials whom he dealt with while heading up the Black Chamber.
At its wartime peak, The Black Chamber encompasses code breakers working in multiple languages and even multiple international stenographic systems that were also used in coded message traffic. It worked on systems for enciphering a good portion of the US government’s message traffic. There were even offices and sections that focused on various forms of invisible ink and methods for both hiding and unveiling messages written with various such inks. The author shares messages intercepted and deciphered with discussions of how such ciphers were attacked by his analysts and eventually deciphered.
When published in 1931, this book also set out one of the significant steppingstones on the path to a Japanese-US conflict in the Pacific. The author discusses at some length how the US read much of the Japanese diplomatic telegraph traffic between Tokyo and the Japanese delegation to the Washington Naval Conference. Knowing the exact limits on naval armaments which Japan would accept even as it pressed at the Conference table for more, allowed the US side to stonewall Japan until it officially accepted the minimal position preferred by the US. Japanese anger over how the US succeeded in reading its instructions to its own negotiating team would feed increasing resentment of the US in Japan.
If you are interested in a backward glimpse of how a significant part of the US intelligence system once worked, you should read this book. If you are interested in recreating this world or in making codes, ciphers, secret messages, invisible inks, etc. a part of your writing whether for fiction or fantasy, you should read this book.

[Nabila Ayu · Rating 4 out of 5]

Herbert Yardley’s The American Black Chamber is a fascinating, at times startlingly candid memoir of the early days of American cryptanalysis. First published in 1931, the book offers a detailed account of Yardley’s tenure as head of the Cipher Bureau—the clandestine codebreaking operation that played a pivotal but largely unheralded role in U.S. intelligence during the interwar period.

Yardley writes with a clarity and directness that makes the technical aspects of cryptography surprisingly accessible, but the true strength of the book lies in its insight into the nature of secrecy, diplomacy, and national security at a moment when America was only beginning to grasp the strategic value of signals intelligence. Through a mix of anecdote, technical discussion, and political commentary, he paints a vivid picture of a quiet war waged with pencil and paper—one that often yielded results as significant as any battlefield maneuver.

What makes the memoir especially poignant is the knowledge of how Yardley’s operation was abruptly shuttered in 1929, not due to failure or irrelevance, but for reasons of political propriety. Secretary of State Henry Stimson’s oft-quoted justification—“Gentlemen do not read each other’s mail”—has come to symbolize a certain naïveté in the face of modern espionage. That Yardley’s office was dismantled at the very moment it was proving its worth makes his account feel, in retrospect, both triumphant and tragic.

Indeed, history would ultimately vindicate Yardley. The outbreak of World War II and the explosive growth of wartime cryptanalysis programs such as Bletchley Park, followed by the eventual establishment of the National Security Agency in 1952, demonstrated beyond doubt the necessity—and inevitability—of organized signals intelligence. Yardley may have been ahead of his time, but the institutions that emerged in his wake owed much to his foundational work.

Though controversial upon publication, and criticized for disclosing sensitive methods and diplomatic secrets, The American Black Chamber remains a seminal work in the history of intelligence. It is a document of immense historical value, and a compelling read—a rare glimpse into the early architecture of a now-vast and permanent surveillance state.

[gargamelscat · Rating 3 out of 5]

Its not well written, but this is an important book, revealing insight into what was then, and for well into the 1970s, the opaque world of cryptology.

There are a number of chapters on invisible inks and mail intercepts during wartime (WW1) leading to the capture of spies which is not so interesting as the discussion of the development of expertise when starting from scratch as code breakers, and the interactions with State and War Office.

But its publication (in the early '30s prompted by the author's need for money I believe) had to have raised ructions at the time with the revelations about decrypting Japanese and other nations diplomatic traffic, from what amounted to an unofficial operation run at arms length from New York on behalf of the State/War Departments.

It is no credit to Henry Stimson, an otherwise towering figure in the establishment, early in his tenure as Secretary of State to have shut down this essential activity on the grounds that "Gentlemen don't read other peoples mail". Naivety or patrician arrogance, might need to read his memoirs to find out.

[John Waldrip · Rating 5 out of 5]

A great read about a man wo was far ahead of his time, and who worked for a nation (the USA) that used him and then discarded him. My conviction since my youth has been that "Government does nothing well." While this is true, there are some functions that only government can do, even when doing such things poorly. This book shows, in the first three decades of the 20th century, a pattern that continues to this day as the USA virtually invites espionage from not only our friends but also our adversaries. This is a history code breaking in the USA, but it speaks to larger issues our nation continues to face ... poorly. I recommend without reservation.

[Don Bennie · Rating 5 out of 5]

Very interesting read though I skimmed the more technical crypto work for the story line and narrative.

Reading the annotated version made this much more interesting as there are fascinating insights and corrections or clarifications made throughout.

[Katherine Schweit · Rating 5 out of 5]

I worked national security matters and counterintelligence cases for the FBI and this is the best book I have ever read to understand a master codebreaker's mind.

[Chris S. · Rating 4 out of 5]

Lots of information on how codes were broken and information was used during World War I and beyond.

[W.P. III · Rating 5 out of 5]

Interesting read, especially how the government works or doesn't.

[Jdblair · Rating 5 out of 5]

This was a fascinating book to me since I spent much of my Air Force career in the intelligence field. When I joined the Air Force and was going through training in 1967/1968, this book was "banned" and was not available. With the appropriate security clearance, I was able to see this book in a secure, controlled-access library. I can certainly see why the book was considered classified in the late 60's.

Herbert O. Yardley was brilliant. However, everyone did not appreciate his work and book. Here is what I found in Wikipedia:

"This book was an embarrassment to the U.S. government and compromised some of the sources Yardley and his associates used. Through this work an estimated 19 nations were alerted that their codes were broken. Much of the post-World War I codebreaking was done by obtaining copies of enciphered telegrams sent over Western Union by foreign diplomats, as was the custom before countries had technology for specialized communications devices. William F. Friedman, considered the father of modern American signals intelligence (SIGINT) gathering, was incensed by the book and the publicity it generated in part because sources and methods were compromised and because Yardley's contribution was overstated."

"While Yardley may have thought that publishing this book would force the government to re-establish a SIGINT program, it had the opposite effect. The U.S. Government considered prosecuting him, but he had not technically violated existing law regarding protection of government records. In 1933, the Espionage Act was amended, PL 37 (USC Title 18, section 952), to prohibit the disclosure of foreign code or anything sent in code.[6] Yardley's second book, Japanese Diplomatic Codes: 1921-1922, was seized by U.S. marshals and never published. The manuscript was declassified in 1979."

This all said, I highly recommend Major Yardley's book to anyone interested in cryptography and cryptology.

[Nick · Rating 4 out of 5]

This embellished non-fiction account of MI-8, the US military intelligence office of codes and cryptography, was a hoot. My favorite chapter dealt with secret inks - spies would wear ties infused with special dyes, which they would use to write messages by dipping in water! Herbert Yardley, who founded and ran MI-8, wrote this in the 1930's after the office was disbanded, and he spends a good deal of time patting himself on the back for all the codes he cracked. There are some interesting expositions on how he solved various ciphers, but I don't think I'm smart enough to follow him. Also cool: MI-8 operated out of a non-descript townhouse in Manhattan, in the east 30s. For some reason, that detail really amuses me. (Reminds me of The Sting, maybe?)

A couple bios came out on Yardley last year, and I'm looking forward to reading at least one of them. I bought this book on ebay for a couple bucks - it's out of print, I think.

[Enso · Rating 3 out of 5]

This book is probably an essential read and written by the man credited with inventing American cryptography when it comes to espionage and sigint. He was blackballed for writing it after the American Black Chamber was disbanded and was never able to work for the American government again (though he worked for others as a consultant). He covers some of the basics of how decipherment occurred as well as how espionage was conducted in his era. Some of his examples go on, at much too great length, with transcripts of decoded telegrams to illustrate his point. I eventually found myself skimming much of the, for example, Japanese telegrams as they were specific to meetings that later became simply historical footnotes. All in all, an interesting book and still readable in our times.

[Tom Mueller]

nf
Interesting 1931 History of Cryptology & the beginnings of MI-8, Military Intelligence, Cryptographic Bureau. Reads partly as an autobiography of Merbert O. Yardley, author and creator of this bureau.

[J. · Rating 5 out of 5]

I loved this book. A friend loaned me a copy from 1931 - he's a collector - and it was just an all around exciting and superb experience - not to mention the thrilling story of American Cryptography. Just an all around cool experience and a very cool book.

[Heather · Rating 3 out of 5]

Interesting because it was once a banned book and is non-fiction. Definitely some gratuitous self-promotion by the author, but worth reading if the topic appeals to you.