“Omega”, a deceptively simple concept, but pregnant with deep implications in many realms of human inquiry: in layman terms, this is a real number expressing the “halting probability” that a randomly constructed program will halt after a finite number of processing steps.
At first glance, this seems just a technical tidbit with limited applicability, and of interest only to computer sciences practitioners. In reality the extraordinary features of such number, in conjunction with the important findings in algorithmic information theory highlighted in this book, deeply influence and raise significant challenges in areas such as:
- the expressive power, informational contents, and necessary limitation of any formal axiomatic system (FAS). As such, these concepts deeply impacts the very foundations of mathematics, and they do so at least to the same extent as Godel's incompleteness theorems
- important mathematical issues such as the computability of Diophantine equations (the famous Hilbert's 10th problem)
- AI and algorithmic information theory (including issues such as algorithmic irreducibility, computability, complexity, and the Turing's halting problem)
- mathematical physics, in relation to issues such as the “fundamental length” (is spacetime continuous or discrete?)
It is important to highlight that we should not adopt an overly-restrictive nor technologically-biased definition of “program” or “computability”. First and foremost, these are fundamental logical/mathematical concepts that play a foundational role in information theory and in all related scientific/mathematical subject matters (with also epistemological implications).
It is quite remarkable how so many of the above issues are interconnected with each other, and to a surprising extent; the “information theory”-driven, multidisciplinary approach followed by Chaitin in his book, brilliantly exposing such deep and surprising inter-connectedness, is very intriguing and refreshing.
This approach, I must disclaim, resonates with my personal outlook at many levels: I believe that reality, at its deepest and most fundamental level, is constituted by nothing but “information” organized in accordance with patterns defined by “logical/mathematical” structures, and evolving through information-processing “computational” processes associated with such underlying structures. The digital ontology of Wheeler's "it from bit" is something I am a quite receptive to; and information theory, within such perspective, naturally plays an important function.
I must dissociate myself, though, from what I think is the overly-optimistic view of Chaitin in pushing for what has been called “digital philosophy”, an approach strongly influenced by Turing's computational models, according to which all information must be digital in nature (must have a digital means of representation/processing). While I have sympathy towards with view, I think that the issue is more complex than presented by Chaitin, and that he neglects to take into account the significant price that has to be paid if we assume that reality is fundamentally discrete (I discuss this point in more detail later).
Chaitin's book does not start immediately with a direct treatment of “omega”, as this concept is first properly positioned within a wider context: the book presents a progression of pre-requisite concepts including computability, Formal Axiomatic Systems (FAS), incompleteness, and fundamentals of algorithmic information theory. After all the above pre-requisite concepts are treated by the author, the proper intellectual framework has been built for a direct treatment of the concept of “omega”, finally explained by the author in all of its glory.
Let me now get into more detail, to provide some flavour of the many fascinating issues treated by Chaitin in this provocative and riveting book:
- The preface contain some discussion about the nature of mathematics and its relationship with the physical world: mathematics is seen as a way of expressing or characterizing structure, and the Universe is seen as built out of such structures. It seems to me that the author adopts a view similar to Shapiro's ontological structural realism (view to which I feel close), but he does not elaborate this position in much further detail. I do agree with his view that there is an aspect of historical contingency in the way mathematical thought developed through the millennia, and that it is very possible that, in a different context, a different “type” of mathematics might have evolved. This does not contradict, per se, a Neo-Platonic view of mathematics; it just implies that human mathematics, as historically developed, only selected a particular subset of existing mathematical structures.
- In the introduction, the important philosophical and mathematical conceptual role played by the notion of computer and computability, which have provoked a paradigm shift and ushered the introduction of digital philosophy, are also briefly treated. Chaitin is clearly an enthusiastic follower of its tenets – to him, “you understand something only if you can program it”
- In chapter 2 we start getting into the heart of the subject: the author uses the example of the distribution of prime numbers to introduce the concept of randomness: the local distribution of primes shows no apparent order, even though we can calculate them one by one. The study of the behaviour of the primes, and number theory in general, are extremely fascinating topics, not just for their implications to the rest of mathematics (just as an example: due to the uniqueness of the prime factorization of integers, prime numbers can be seen as the fundamental building block of the whole integer number system), and for their practical applications but also because, even with apparently simple and innocuous questions, we get right away at the very borders of current human understanding: for example nobody has been able to prove that there are infinitely many “twin” primes (consecutive odd primes separated by just one even number). There are many open questions about prime numbers, which are (somebody stated once) “to mathematicians as the elements are to chemistry, as elementary particles are to physicians, as nucleotides are to geneticists”.
- While discussing the magic of the primes, the author explains and compares three different proofs of the theorem that there are infinitely many primes: the famous Euclid's proof, the more complex but beautiful Euler's proof leading to the Euler's product formula (the latter being a special case of the notorious Riemann's zeta function, whose properties are connected with the statistical distribution of primes), and finally the incredibly simple and concise Chaitin's proof, purely based on basic considerations of complexity.
- By briefly highlighting the unique magic of the primes, the author therefore manages quite brilliantly to introduce to the reader elementary concepts of randomness and complexity. The author does not just uses the properties of the primes as an excuse to introduce such concepts: he also uses a comparison between these three proofs to express his ideas about the nature of mathematical proof: I like Chaitin's idea that, while a false statement has no proof, a true statement will have multiple proofs and can therefore be approached in different ways, some more fruitful and informative than others. “Math is the music of reason, and some proofs sound like jazz, others sound like a fugue”, Chaitin states in a quaintly cute way.
I personally think that the fundamental reason for such multiplicity of proof is that mathematical facts are never isolated, but they are woven into a vast network of multiple interconnections, of which we human have barely some visibility of only an extremely small subset. What comes to my mind is the example of the solution to the (in)famous Fermat's Last Theorem, an amazing achievement reached by studying mathematical objects (such as elliptic curves and modular forms) that seemed completely unrelated to the problem, but that had in reality deep hidden connections with it. Oftentimes, illuminating a different aspect of the problem reveals unexpected connections leading to unexplored and fascinating directions.
- The next sections are dedicated to the subject of “incompleteness”, a concept that has been frequently misunderstood. We are dealing with the famous “Godel's incompleteness theorems”, initially proven by Godel himself and subsequently demonstrated by Turing with an alternative approach based on purely computational considerations. In layman's terms, focusing on the first theorem, Godel's wonderful meta-mathematical result states that any finite, consistent formal axiomatic system that is sufficiently powerful (within which a certain amount of elementary arithmetic operations can be carried out) is necessarily incomplete: there are statements in its language which can neither be proved nor disproved within it.
This proves that mathematics is informationally infinitely rich, and that there is no finite axiomatic system that can exhaust it. The proof of Godel's theorems is notoriously complex and the author expresses his preference towards Turing's approach. Chaitin's own approach and proof to this issue, based on the concept of “omega”, is explained in detail in the last chapters of the book
- The author then explains the concept of FAS (Formal Axiomatic System) which is essentially a set of axioms and rules of inference expressed in a specific formal language, inclusive of a proof-generating/checking algorithm; such algorithm can in principle be run against the axioms of the associated FAS, to automatically generate all the resulting mathematical assertions (theorems, if you like) by using symbolic logic in accordance with the defined rules of inference of the FAS. From this perspective, a specific FAS can be associated with the automated algorithmic generation a “computably enumerable” set of mathematical assertions.
- Next comes a brief discussion on the famous Turing's halting problem. Finally, the important connection between incompleteness and the halting problem is highlighted: Turing demonstrated that the halting problem implies incompleteness, in other words that uncomputability implies incompleteness. This is another proof of the importance of algorithmic information theory to the definition of the foundations of mathematics, and of the deep link between these two subject matters. Chaitin goes one step further, and he demonstrate later in his book that “randomness” implies incompleteness. But regardless of the approach, it all points to the conclusion that Hilbert's goal for one single FAS for all mathematics is impossible – mathematics is infinite.
- In the next section, another surprising link is highlighted by the author: the link between exponential Diophantine equations (algebraic equations in which everything is an integer, and where the only allowed operations are addition, multiplication and exponentiation), the Universal Turing machine (UTM: loosely defined as equivalent to a general-purpose programmable computer), and Turing's halting problem. The surprising connection is that there is one unique, long (20,000 unknowns!) exponential Diophantine equation (aptly called “universal Diophantine equation”) that can perform the same calculations as a UTM!
Coincidence? Very unlikely – we are probably witnessing a deep interconnection between the integer field and the very essence of computation. The existence of such monster able to emulate a UTM, by the way, also means that the unsolvability of Turing's halting problem implies the unsolvability of Hilbert 10th problem (computability of Diophantine equations).
We can clearly see that objects that, at first glance, appear to have no connections whatsoever, demonstrate in fact surprising commonalities, due to their sharing of the same information theoretical structure
- We are now in Chapter 3, mostly dedicated to some of the main concepts of algorithmic information theory. Chaitin's digital-philosophical approach is as evident as ever: “you combine 0's and 1's, and you get everything” - in other words, according to Chaitin, binary information represented by the combinatorial possibilities of 0 and 1 can exhaust the informational contents of the universe. This paradigm is adopted across the board, for example in his utterly fascinating definition of what “understanding” and what “a scientific theory” ultimately mean: the compression of the apparently chaotic, informationally-bloated variety of physical phenomena into a significantly reduced “program”(set of physical laws) that contains in a succinct, concise, informationally efficient but rich manner, all the necessary information to fully account for such multiplicity. The greater the degree of experimental information compression into an efficiently defined law, the better such law, the deeper the understanding of the associated phenomena. Think about the expressive power of the Einstein field equation of general relativity, or the incredible law-generating power of the Euler-Lagrange equation.
“Irreducibility” then means, following this concept, that the underlying data can't be compressed into a physical law, in other words that there is no pattern or structure that allows for such informational compression. The underlying concepts are not dissimilar to the information compression process that is accomplished by standard software that manages digital pictures/videos.
The same applies to any FAS: associated with it there is the collection of all theorems that can be generated by it. In a sense, each FAS already contains, within it, all associated theorems that can be generated from it – from an information management perspective, it is just an extremely efficient way to minimally codify the multitude of associated theorems.
- From the above, the general concepts of “complexity” (the size of the smallest program required to codify the system being analyzed - this is essentially the Kolmogorov's definition), and “irreducibility/randomness” (there is no program smaller than the set of data representing the associated system), emerge quite naturally. Chaitin also takes the opportunity to make a fascinating detour into the informational complexity of biological systems.
- Chapter 4 and 5 get deeper into the main tenets of digital philosophy, and contain a sustained attack against the real number system (and on the continuum). Everybody who is familiar with Chaitin knows that this is one of his personal obsessions, as the very existence of infinite-precision numbers that are not computable and that can't be compressed algorithmically (therefore containing infinite information) presents a real challenge to the very foundations of any digital philosophy.
The vast majority of reals are “non-computable” and “random” (computable numbers can be computed to within any desired precision by a finite, terminating algorithm; random numbers are numbers whose complexity of their first N bits grows with N), as their cardinality of such is the same as the set of all reals! Such reals, actually, are not just un-computable: the large majority of them are also un-nameable (they can't be defined, or referred to, constructively or not, within any FAS nor within any formal language). The probability to pick such quite an unmanageable real is actually 1, as opposed to a probability 0 to pick a computable real, as easily demonstrated by Chaitin using Lebesgue measure theory.
- Chaitin then uses these results to wonder about numbers that can't be constructed/calculated, nor even referred to/named. He also highlights that no finite FAS can ever determine infinite many bits of an uncomputable real, and refers to physical arguments (such as Heisenberg's indetermination) to challenge the physical relevance of the very concept of continuum. This brings Chaitin to a wholesome rejection of the continuum from a mathematical/physical perspective.
While I am sympathetic to such views, I must say however that the ultimate discreteness of spacetime and of its underlying mathematical structure presents issues that he does not address:
1. The continuity of calculus is buried deep in standard mathematical physics. Concepts such as velocity and acceleration as derivatives depend on the concept of continuity.
2. Any geometry that purports to describe physical space and which relies on a line segment of aleph-zero cardinality (same cardinality as the computable numbers), inevitably leads to issues such as the fact that countable additivity ceases to hold at the most fundamental level
3. If we assume that the time evolution of any system is discrete, the applicability of differential equations become an issue; they have to be substituted by difference equations
4. The famous “Weyl's tile argument” highlights that discretization of space, for example, implies a violation of Pythagoras theorem
5. A discrete spacetime model is incompatible with the existence of some continuous physical symmetries (such as rotational, translational and Lorentz symmetry), and it can break the principle of causality. We need also to consider that, according to Noether's theorem, continuous symmetries are associated with conservation of specific quantities (time translation symmetry gives conservation of energy, for example)
6. Many modern physical theories (such as relativity) are inherently “continuous”
So, while I think that the continuum is a nonphysical idealization, and that ultimately spacetime must have some sort of discrete structure, I think that the open issues posed by a discrete structure are significant. Maybe the whole question is ill-posed: simply, when going below a threshold such as the Planck's length, the whole concept of distance and of the discrete/continuous dichotomy, and of spacetime itself, just stop to make sense and have to be replaced by other concepts.
- The final part of the book is dedicated to the “omega” constant, the completely irreducible/ random real number representing the halting probability (i.e. that any randomly chosen “program” will ever stop). Considering all the results in the previous chapter, it is easily seen that any FAS can only determine as many bits of omega as the complexity of the FAS itself. This is an extremely strong incompleteness result, which means that the bits of omega are logically irreducible - they are atomic mathematical facts that can't be derived from simpler axioms. This proves that maths contains irreducible information, of which omega is the prime example.
- The author also explains how omega can be represented by a particular Diophantine equation , and how omega even encodes results of the halting problem. Deep and surprising links, again. I don't have space here to get into more details about this fascinating constant, but it is important to highlight how this constant appears to be at the crossroads of many issues of computational and informational complexity.
- There are actually many other issues addressed in this book (such as Chaitin's brilliant idea of a “quasi-empirical” approach to maths, and his push for an effort to consider the current axiomatic system just a starting point, something that needs to be considered flexibly, and enriched with new axioms wherever advisable), but again I can't get into more detail because of space limitations.
To conclude, this is a fascinating book about many fascinating topics. The only issues are the writing style (occasionally confusing and with not enough detail, and often a bit too self-celebratory), and some significant hand-waving in some areas. However this is a very interesting and thought-provoking book overall, well deserving of a 4 star rating.