Meta Learning: How To Learn Deep Learning And Thrive In The Digital World

by Radek Osmulski

"Meta Learning" documents the key lessons the author — Radek Osmulski — learned on his Deep Learning journey.

He learned to program and do Deep Learning through self-study. His achievements include successfully participating in Kaggle competitions and being gainfully employed in various Deep Learning roles for several years now.

Among other things, from the book, you'll learn:
- how to learn machine learning efficiently
- the proven strategies to improve as a developer
- how to approach the tools you use for work and why it matters
- how to reason about the hardware you use for best results and to make sure you invest your time where it's worthwhile
- what makes sharing your work so powerful
- one way to reason about finding a mentor
- how to keep in touch with the deep learning community and stay up to date with trends in research for the least amount of work
- an effective way to become employable without a formal background (tailored to the digital age we live in now)
- how to build a habit and why the path of little resistance is the way to go
- one way to find the energy to do Deep Learning

All of the above is based on the author's experience. The ideas and strategies that are shared are the ones that worked best for the author, out of many that he tried.

Genres: Artificial Intelligence, Technology, Nonfiction

90 pages, ebook

Published May 23, 2021

Reviews

[Jawakar · Rating 5 out of 5]

A must experience book for enthusiasts who are starting deep learning. This book is kind of his (Radek Osmulski's) biography. He has shared great insights which would lift our progress effectively.

This book starts with his inspiring story then the prerequisites for beginning is cut down to four points. The approach given to the fast.ai course is practically acceptable. Notable tips are given for efficient programming, structuring machine learning projects and winning Kaggle competitions as well.

Points on personnel branding, finding jobs, sharing works, getting a mentor admired me a lot. He doesn't stop here. The fastest way to learn and steps to be followed along with insightful tweets makes this book the coolest one to pick. He has shared his yoga routines and diet plans that would boost up readers mental and physical health.

Pick up everything possible from here and implement to experience the benefits. Happy learning.Radek Osmulski

[Bjoern Rochel]

Sounds like the right approach. Likely need to read it a second time because of all the extra material that’s referenced here

[Min · Rating 4 out of 5]

Protip: Keep a notebook or notion page open by your side so you can write down what actions/materials you're going to look at afterwards because Radek gives out many nuggets of wisdom.

It doesn't fluff about and tells the reader right to their face that they need to do x, y, z to get started in the ML space and no that doesn't mean sitting through hours of pre-calc lectures. This book is straight forward and to the point, it drills down on the fundaments of what you should learn and how you should approach it in a comfortable one sitting read (it's only 90 pages front to back so book out an afternoon and get to it).

[Dorai Thodla]

Book Quotes. I got some good ideas to spread from this book. It is about meta learning and improving your skills.

Reflections:
- Learn in Public, says the book. This does not come naturally. Marking up the book (or copy/pasting fragments) and writing notes is the best way I know how to share. Hence the book notes.
- I wish there was a book like this 20 or even 30 years ago. I stumbled upon many of things mentioned in the book like sharing, branding, building communities, mentoring, learning at my own pace and in some cases (very rare ones unfortunately) practice over theory.

Book Notes - Meta Learning

Quotes

- Your ability as a developer is measured by the utility of the things you can
express in a language. The ability to construct, from crude stone, buildings that withstand the test of time has much more value than the ability to chip stones into exquisite shapes that you are unable to weave together.
Additionally, languages are invented to make programming easier, to serve some purpose. That means if you are at the start of your journey as a programmer, the only way you can really understand a piece of syntax is to observe it as part of a whole—where an expression carries some meaning and integrates in some
fashion into a whole."
- "You don’t sharpen your skills with resources, books, or
articles. You sharpen your skills with practice. If you want
to get better, go do the thing."
— Jason Fried, a tweet
Contrary to popular opinion, there are only two methods that are guaranteed to take you from not knowing much about writing code to being a great developer.
They certainly do not include passively watching YouTube videos, taking MOOC after MOOC, or binging on computer science textbooks. While all of these are good side dishes, they make for a poor main course.
The two time tested practices that work in this regard are reading and writing
code. They lie at the core of what being a developer is all about."
- "This is the reason why many programmers spend a lot of time studying their editors and swear by the ones that don’t require lifting their fingers from the keyboard. When you reach for your mouse, your flow is interrupted. You introduce a delay between your intention and the results appearing on the screen. The chunkiness of operating the mouse comes between you and your
work. Ideally, you want nothing to separate the two."
- "Do genuine work (it compounds)
"Reading a book without taking notes is like discovering a
new territory and forgetting to draw a map."
— Julian Shapiro
Genuine work moves the atoms in the universe."
- "a small set of notes is something you can build on. You can add to them the next day or, based on the experience of writing them down, decide that you do not want to work on this any longer. You are in motion and making progress."
- "At the core of all machine learning lies the ability to generalize to unseen data."
- How to do machine learning efficiently
"A machine learning project is like a flower. It has the potential to spring from a small seed into a magnificent entity, assuming the conditions are right. But it cannot be willed into existence. Fail to supply one of the necessary ingredients and no matter how much effort you put into it, it will not grow. The main condition for a healthy machine learning project is a good train—validation—test split."
- "Machine learning code is notoriously hard to write. It is extremely easy to
introduce subtle bugs. Our code might still run and we might get a decent result, but it won’t be as good as it could otherwise be.[30] Further, it is really hard to determine whether the result we are getting is good if we don’t have anything to compare it against. A baseline is a good way of addressing both of these concerns."
- "Instead of training on the entire dataset, maybe just 1% or 5% of data would be enough. Assuming random sampling of our examples would be appropriate, switching to running
on a subset of our data might require changing just a single line of code."
- "Working on a machine learning problem is a process of learning about the problem domain and data and responding to feedback on what works and what doesn’t. There is a great chance that you will not arrive at a great model from the get-go."
- "Research papers have their younger siblings: the blog posts. Often, they are the superior place to start, though finding good blog posts on more arcane subjects can be tough. Nonetheless, they can still be great learning resources. The quality of blog posts varies greatly, but some are as accurate as the papers, and reading them can be a great way of building intuition on any given subject."
- "it is important to get an idea of what a paper is talking about, what its essence is and whether the described technique has a chance of being useful in the problem you are solving. This is a skill that comes from scanning and reading a lot of papers. There is no magic to it. Some papers will be more approachable than others and that is also okay. If you have a hard time understanding a certain paper, it is best to read related papers first and then take a look at the initial paper only after a while."
- "Now that we have a framework for doing research, we must work on our
solution daily. The idea is to make small tweaks, and the effects will compound
over time. Jumping to the top of the leaderboard might seem impossible at any given moment, but improving our solution just slightly, trying out a new method, is within our reach."
- "The final piece of the puzzle is ensembling. The idea behind it is very straightforward. We want to combine predictions from multiple models in the hopes that their errors are not correlated. If model A makes different errors than model B, some of them will cancel out each other. Using a diverse set of models helps."
- "related technique is training with cross-validation. We train multiple models on our data, withholding different parts of it for validation and combining the results. Both techniques are extremely powerful, and it is hard to imagine a winning Kaggle solution that would not leverage both. Many great resources on both of these techniques are available online"
- "To keep things simpler initially, you might want to start with Google Collab or Kaggle Kernels."
- "When working in Jupyter Notebooks, it is good to get into the habit of using the %%timeit cell magic. This can often lead to interesting insights."
- "Dropping down to using numpy instead of pandas offered a nearly 2000x performance increase!"
- "Timing your code is a very useful technique across the board, but especially when you are constructing datasets and data loaders. The only way to make full use of your GPU is to write efficient code!"

- "We do not learn from experience… we learn from
reflecting on experience." — John Dewey"
- "Cooperative inclinations are one of our strongest instincts. When we share our
work and explain to others how something can be done, we are being helpful
and we give into our true self. This can lead to increased satisfaction with our
lives and a stronger sense of purpose. Now, who wouldn’t want more of that?"
- "What to focus on in sharing your work Speak to your experience. This is one of the most valuable things that we all have to offer. By showing people what you did and how, as well as how you felt along the way and what results you achieved, you draw a map that others can follow and demonstrate what is possible. This is immensely valuable."
- "Peace of mind is the most important prerequisite for creative work." — Richard Feynman"
- "A good mentor is someone very good at something you care about. They also
need to be willing to explain to you, in detail, how to do said thing, using a
language that will be easy for you to follow."
- "if you do interact with your mentor, it might be good to do so via forums or some other public medium. AMAs work well for this purpose. This way, others will also have a chance to benefit from the answers you receive."

[TK · Rating 3 out of 5]

It looks like a book of tweets, which is nice because it focuses on what's important, it's a fast read, and strips out the fluff. On the other hand, it lacks more detailed explanations for each chapter.

It focuses more on productivity and learning than on the machine learning world per se, which was not my expectation. The productivity stuff, like time management, forming habits, and sharing work, is all cliches from the self-help (even though they are important), but I was expecting the book to have more details and be more aligned with the chapters that talk about "How to win at Kaggle", "Time yourself", and "Theory vs Practice (in ML)". They are super valuable, specifically in the ML world.

For anyone reading this book:
- If you're an avid reader of self-help books, you can skip many chapters, as they talk about exactly what's common in these types of books
- If you're an experienced engineer, you can skip the "programming" chapters (e.g. "programming is about what you have to say", "The secret of good developers", and "The best way to improve as a developer").

There's a valuable resource he plugs into the book that talks about how not to fail ML courses (MOOCs), that's basically not to fall into the courses trap, going from one course to the other, without putting in the work, coding, and solving problems. We think we need to keep "learning" everything from the curriculum before doing any project, but doing the project is actually one of the most important parts of "learning".

[Asiful Nobel · Rating 3 out of 5]

This book reads like a self-help guide for aspiring ML engineers. Its strengths lie in the author's relatable journey and the valuable advice it offers.

However, the book's weaknesses include the fact that it could have been condensed into a series of blog posts with a smaller font size, and that some of the advice may be too basic for experienced software engineers.

[Aihua · Rating 5 out of 5]

“A blacksmith is a professional who mixes metals in various proportions, creating alloys and using them to create useful objects. Blacksmiths use at least 20 types of hammers. On top of that, they are encouraged to create their own, special-purpose hammers.”

[Ashwitha Shetty · Rating 5 out of 5]

I needed a simple way to learn how to apply deep learning to real-world problems, but online resources were overwhelming. This was exactly what I needed to start with. Thank you, Radek, for sharing your learnings

[Madhav Goyal · Rating 5 out of 5]

It really helped me with somethings I was struggling with.

[Artem Ryblov · Rating 5 out of 5]

I've read this book and took a lot of notes. Now I will go through them and apply them!

[Ping Cao · Rating 4 out of 5]

Good advice.

[Daniel Noworyta · Rating 4 out of 5]

Few nice tips how to grow as a developer.