Emre Sevinç

“Yann LeCun's strategy provides a good example of a much more general notion: the exploitation of innate knowledge. Convolutional neural networks learn better and faster than other types of neural networks because they do not learn everything. They incorporate, in their very architecture, a strong hypothesis: what I learn in one place can be generalized everywhere else.

The main problem with image recognition is invariance: I have to recognize an object, whatever its position and size, even if it moves to the right or left, farther or closer. It is a challenge, but it is also a very strong constraint: I can expect the very same clues to help me recognize a face anywhere in space. By replicating the same algorithm everywhere, convolutional networks effectively exploit this constraint: they integrate it into their very structure. Innately, prior to any learning, the system already “knows” this key property of the visual world. It does not learn invariance, but assumes it a priori and uses it to reduce the learning space-clever indeed!”
― Stanislas Dehaene, How We Learn: Why Brains Learn Better Than Any Machine . . . for Now

“Note that in the above construction we made a number of choices; here we must beware. Choosing a good categorification – like designing a good algebraic structure such as that of preorders or quantales – is part of the art of mathematics. There is no prescribed way to categorify, and the success of a chosen categorification is rather empirical: its richer structure should allow us more insights into the subject we want to model.”
― Brendan Fong, Seven Sketches in Compositionality: An Invitation to Applied Category Theory

“Regular expressions are widely used for string matching. Although regular-expression systems are derived from a perfectly good mathematical formalism, the particular choices made by implementers to expand the formalism into useful software systems are often disastrous: the quotation conventions adopted are highly irregular; the egregious misuse of parentheses, both for grouping and for backward reference, is a miracle to behold. In addition, attempts to increase the expressive power and address shortcomings of earlier designs have led to a proliferation of incompatible derivative languages.”
― Chris Hanson, Software Design for Flexibility: How to Avoid Programming Yourself into a Corner

“For advanced analytics, a well-designed data pipeline is a prerequisite, so a large part of your focus should be on automation. This is also the most difficult work. To be successful, you need to stitch everything together.”
― Piethein Strengholt, Data Management at Scale: Best Practices for Enterprise Architecture