Emre Sevinç

“Note that in the above construction we made a number of choices; here we must beware. Choosing a good categorification – like designing a good algebraic structure such as that of preorders or quantales – is part of the art of mathematics. There is no prescribed way to categorify, and the success of a chosen categorification is rather empirical: its richer structure should allow us more insights into the subject we want to model.”
― Brendan Fong, Seven Sketches in Compositionality: An Invitation to Applied Category Theory

“According to Darwin’s Origin of Species, it is not the most intellectual of the species that survives; it is not the strongest that survives; but the species that survives is the one that is able best to adapt and adjust to the changing environment in which it finds itself. —Leon C. Megginson”
― Jodi Picoult, Wish You Were Here

“Yann LeCun's strategy provides a good example of a much more general notion: the exploitation of innate knowledge. Convolutional neural networks learn better and faster than other types of neural networks because they do not learn everything. They incorporate, in their very architecture, a strong hypothesis: what I learn in one place can be generalized everywhere else.

The main problem with image recognition is invariance: I have to recognize an object, whatever its position and size, even if it moves to the right or left, farther or closer. It is a challenge, but it is also a very strong constraint: I can expect the very same clues to help me recognize a face anywhere in space. By replicating the same algorithm everywhere, convolutional networks effectively exploit this constraint: they integrate it into their very structure. Innately, prior to any learning, the system already “knows” this key property of the visual world. It does not learn invariance, but assumes it a priori and uses it to reduce the learning space-clever indeed!”
― Stanislas Dehaene, How We Learn: Why Brains Learn Better Than Any Machine . . . for Now

“A good program must be written many times. This is true of the programs we show. The first draft may not clearly separate out the concerns, but by making that draft the programmer learns the structure of the problem. We will show two different implementations, which will reveal the evolution of the program as we identify shortcomings in our draft.”
― Chris Hanson, Software Design for Flexibility: How to Avoid Programming Yourself into a Corner