Emre Sevinç

“A good program must be written many times. This is true of the programs we show. The first draft may not clearly separate out the concerns, but by making that draft the programmer learns the structure of the problem. We will show two different implementations, which will reveal the evolution of the program as we identify shortcomings in our draft.”
― Chris Hanson, Software Design for Flexibility: How to Avoid Programming Yourself into a Corner

“The moral here is that nature and nurture should not be opposed. Pure learning, in the absence of any innate constraints, simply does not exist. Any learning algorithm contains, in one way or another, a set of assumptions about the domain to be learned. Rather than trying to learn everything from scratch, it is much more effective to rely on prior assumptions that clearly delineate the basic laws of the domain that must be explored, and integrate these laws into the very architecture of the system. The more innate assumptions there are, the faster learning is (provided, of course, that these assumptions are correct!). This is universally true. It would be wrong, for example, to think that the AlphaGo Zero software, which trained itself in Go by playing against itself, started from nothing: its initial representation included, among other things, knowledge of the topography and symmetries of the game, which divided the search space by a factor of eight.

Our brain too is molded with assumptions of all kinds. Shortly, we will see that, at birth, babies' brains are already organized and knowledgeable. They know, implicitly, that the world is made of things that move only when pushed, without ever interpenetrating each other (solid objects)—and also that it contains much stranger entities that speak and move by themselves (people). No need to learn these laws: since they are true everywhere humans live, our genome hardwires them into the brain, thus constraining and speeding up learning. Babies do not have to learn everything about the world: their brains are full of innate constraints, and only the specific parameters that vary unpredictably (such as face shape, eye color, tone of voice, and individual tastes of the people around them) remain to be acquired.”
― Stanislas Dehaene, How We Learn: Why Brains Learn Better Than Any Machine . . . for Now

“So the syntax of the regular-expression language is awful; there are various incompatible forms of the language; and the quotation conventions are baroquen [sic]. While regular expression languages are domain-specific languages, they are bad ones. Part of the value of examining regular expressions is to experience how bad things can be.”
― Chris Hanson, Software Design for Flexibility: How to Avoid Programming Yourself into a Corner

“Our brain is therefore not simply passively subjected to sensory inputs. From the get-go, it already possesses a set of abstract hypotheses, an accumulated wisdom that emerged through the sift of Darwinian evolution and which it now projects onto the outside world. Not all scientists agree with this idea, but I consider it a central point: the naive empiricist philosophy underlying many of today's artificial neural networks is wrong. It is simply not true that we are born with completely disorganized circuits devoid of any knowledge, which later receive the imprint of their environment. Learning, in man and machine, always starts from a set of a priori hypotheses, which are projected onto the incoming data, and from which the system selects those that are best suited to the current environment. As Jean-Pierre Changeux stated in his best-selling book Neuronal Man (1985), “To learn is to eliminate.”
― Stanislas Dehaene, How We Learn: Why Brains Learn Better Than Any Machine . . . for Now