What do you think?
Rate this book
Deep Learning at Scale: At the Intersection of Hardware, Software, and Data
Bringing a deep-learning project into production at scale is quite challenging. To successfully scale your project, a foundational understanding of full stack deep learning, including the knowledge that lies at the intersection of hardware, software, data, and algorithms, is required.
This book illustrates complex concepts of full stack deep learning and reinforces them through hands-on exercises to arm you with tools and techniques to scale your project. A scaling effort is only beneficial when it's effective and efficient. To that end, this guide explains the intricate concepts and techniques that will help you scale effectively and efficiently.
You'll gain a thorough understanding
How data flows through the deep-learning network and the role the computation graphs play in building your modelHow accelerated computing speeds up your training and how best you can utilize the resources at your disposalHow to train your model using distributed training paradigms, i.e., data, model, and pipeline parallelismHow to leverage PyTorch ecosystems in conjunction with NVIDIA libraries and Triton to scale your model trainingDebugging, monitoring, and investigating the undesirable bottlenecks that slow down your model trainingHow to expedite the training lifecycle and streamline your feedback loop to iterate model developmentA set of data tricks and techniques and how to apply them to scale your training modelHow to select the right tools and techniques for your deep-learning projectOptions for managing the compute infrastructure when running at scale
This book illustrates complex concepts of full stack deep learning and reinforces them through hands-on exercises to arm you with tools and techniques to scale your project. A scaling effort is only beneficial when it's effective and efficient. To that end, this guide explains the intricate concepts and techniques that will help you scale effectively and efficiently.
You'll gain a thorough understanding
How data flows through the deep-learning network and the role the computation graphs play in building your modelHow accelerated computing speeds up your training and how best you can utilize the resources at your disposalHow to train your model using distributed training paradigms, i.e., data, model, and pipeline parallelismHow to leverage PyTorch ecosystems in conjunction with NVIDIA libraries and Triton to scale your model trainingDebugging, monitoring, and investigating the undesirable bottlenecks that slow down your model trainingHow to expedite the training lifecycle and streamline your feedback loop to iterate model developmentA set of data tricks and techniques and how to apply them to scale your training modelHow to select the right tools and techniques for your deep-learning projectOptions for managing the compute infrastructure when running at scale
661 pages, Kindle Edition
Published June 18, 2024
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 1 of 1 review
July 26, 2024
This is a challenging review to write, as there are quite some mixed feelings on this one.
This book is awesome because it covers a lot of the good stuff, all the buzzwords are in there and it comes so close to the bleeding edge you should keep some bandages and some eosin in the vicinity while reading it, and this applies to all topics it touches from describing the hardware architecture present in Ampere and Hopper to modern ML architectures.
But it isn't a book in the true sense of the book, if you progress beyond the 1st part there are some 'exercises' (yeah go ahead, try them on _your_ GPU cluster). Which completely clashes with the format of the thing being 'a book'. The exercises are mentioned, briefly touched, but not dug into. As for me, I was reading a book, at that moment I didn't want to bother with cloning some random github repo, and run some bogus commands (or go into a fully shift-enter-spree) to run it on my collection of A100's I happen to have lying around, oh wait ... . Okay I understand, these are quite some complex topics to cover and you can't cover them in a five line snippet either.
Hence, extremely up-to-date, great content but struggling with and identity crisis.
This book is awesome because it covers a lot of the good stuff, all the buzzwords are in there and it comes so close to the bleeding edge you should keep some bandages and some eosin in the vicinity while reading it, and this applies to all topics it touches from describing the hardware architecture present in Ampere and Hopper to modern ML architectures.
But it isn't a book in the true sense of the book, if you progress beyond the 1st part there are some 'exercises' (yeah go ahead, try them on _your_ GPU cluster). Which completely clashes with the format of the thing being 'a book'. The exercises are mentioned, briefly touched, but not dug into. As for me, I was reading a book, at that moment I didn't want to bother with cloning some random github repo, and run some bogus commands (or go into a fully shift-enter-spree) to run it on my collection of A100's I happen to have lying around, oh wait ... . Okay I understand, these are quite some complex topics to cover and you can't cover them in a five line snippet either.
Hence, extremely up-to-date, great content but struggling with and identity crisis.
Displaying 1 - 1 of 1 review