What do you think?
Rate this book
Getting Started with Google BERT: Build and train state-of-the-art natural language processing models using BERT
Kickstart your NLP journey by exploring BERT and its variants such as ALBERT, RoBERTa, DistilBERT, VideoBERT, and more with Hugging Face's transformers library
Key FeaturesExplore the encoder and decoder of the transformer modelBecome well-versed with BERT along with ALBERT, RoBERTa, and DistilBERTDiscover how to pre-train and fine-tune BERT models for several NLP tasksBook DescriptionBERT (bidirectional encoder representations from transformer) has revolutionized the world of natural language processing (NLP) with promising results. This book is an introductory guide that will help you get to grips with Google's BERT architecture. With a detailed explanation of the transformer architecture, this book will help you understand how the transformer’s encoder and decoder work.
You’ll explore the BERT architecture by learning how the BERT model is pre-trained and how to use pre-trained BERT for downstream tasks by fine-tuning it for NLP tasks such as sentiment analysis and text summarization with the Hugging Face transformers library. As you advance, you’ll learn about different variants of BERT such as ALBERT, RoBERTa, and ELECTRA, and look at SpanBERT, which is used for NLP tasks like question answering. You'll also cover simpler and faster BERT variants based on knowledge distillation such as DistilBERT and TinyBERT. The book takes you through MBERT, XLM, and XLM-R in detail and then introduces you to sentence-BERT, which is used for obtaining sentence representation. Finally, you'll discover domain-specific BERT models such as BioBERT and ClinicalBERT, and discover an interesting variant called VideoBERT.
By the end of this BERT book, you’ll be well-versed with using BERT and its variants for performing practical NLP tasks.
What you will learnUnderstand the transformer model from the ground upFind out how BERT works and pre-train it using masked language model (MLM) and next sentence prediction (NSP) tasksGet hands-on with BERT by learning to generate contextual word and sentence embeddingsFine-tune BERT for downstream tasksGet to grips with ALBERT, RoBERTa, ELECTRA, and SpanBERT modelsGet the hang of the BERT models based on knowledge distillationUnderstand cross-lingual models such as XLM and XLM-RExplore Sentence-BERT, VideoBERT, and BARTWho this book is forThis book is for NLP professionals and data scientists looking to simplify NLP tasks to enable efficient language understanding using BERT. A basic understanding of NLP concepts and deep learning is required to get the best out of this book.
Table of ContentsA Primer on Transformer ModelUnderstanding the BERT ModelGetting Hands-On with BERTBERT variants I - ALBERT, RoBERTa, ELECTRA, and SpanBERTBERT variants II - Based on knowledge distillationExploring BERTSUM for Text SummarizationApplying BERT for Other LanguagesExploring Sentence and Domain Specific BERTWorking with VideoBERT, BART, and more
Key FeaturesExplore the encoder and decoder of the transformer modelBecome well-versed with BERT along with ALBERT, RoBERTa, and DistilBERTDiscover how to pre-train and fine-tune BERT models for several NLP tasksBook DescriptionBERT (bidirectional encoder representations from transformer) has revolutionized the world of natural language processing (NLP) with promising results. This book is an introductory guide that will help you get to grips with Google's BERT architecture. With a detailed explanation of the transformer architecture, this book will help you understand how the transformer’s encoder and decoder work.
You’ll explore the BERT architecture by learning how the BERT model is pre-trained and how to use pre-trained BERT for downstream tasks by fine-tuning it for NLP tasks such as sentiment analysis and text summarization with the Hugging Face transformers library. As you advance, you’ll learn about different variants of BERT such as ALBERT, RoBERTa, and ELECTRA, and look at SpanBERT, which is used for NLP tasks like question answering. You'll also cover simpler and faster BERT variants based on knowledge distillation such as DistilBERT and TinyBERT. The book takes you through MBERT, XLM, and XLM-R in detail and then introduces you to sentence-BERT, which is used for obtaining sentence representation. Finally, you'll discover domain-specific BERT models such as BioBERT and ClinicalBERT, and discover an interesting variant called VideoBERT.
By the end of this BERT book, you’ll be well-versed with using BERT and its variants for performing practical NLP tasks.
What you will learnUnderstand the transformer model from the ground upFind out how BERT works and pre-train it using masked language model (MLM) and next sentence prediction (NSP) tasksGet hands-on with BERT by learning to generate contextual word and sentence embeddingsFine-tune BERT for downstream tasksGet to grips with ALBERT, RoBERTa, ELECTRA, and SpanBERT modelsGet the hang of the BERT models based on knowledge distillationUnderstand cross-lingual models such as XLM and XLM-RExplore Sentence-BERT, VideoBERT, and BARTWho this book is forThis book is for NLP professionals and data scientists looking to simplify NLP tasks to enable efficient language understanding using BERT. A basic understanding of NLP concepts and deep learning is required to get the best out of this book.
Table of ContentsA Primer on Transformer ModelUnderstanding the BERT ModelGetting Hands-On with BERTBERT variants I - ALBERT, RoBERTa, ELECTRA, and SpanBERTBERT variants II - Based on knowledge distillationExploring BERTSUM for Text SummarizationApplying BERT for Other LanguagesExploring Sentence and Domain Specific BERTWorking with VideoBERT, BART, and more
- GenresTechnology
352 pages, Kindle Edition
Published January 22, 2021
30 people are currently reading
27 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 5 of 5 reviews
April 22, 2022
A good book for beginners.
3 - 4 ⭐
3 - 4 ⭐
February 21, 2021
The preface states the book is for: " [...] for NLP professionals and data scientists looking to simplify NLP tasks to enable efficient language understanding using BERT."
And that, "A basic understanding of NLP concepts and deep learning is required to get the most out of this book."
To that end, if you tick both of the above boxes, then this book is DEFINITELY for you.
Pros:
1) Style
The author writes in a conversational manner which makes it incredibly easy to follow along; it is no easy feat to write about complex subjects in an engaging manner.
2) Examples
The code examples are easy to follow and implement. Additionally, the repo on github provides further explanations.
3) Range
What was most illuminating was the shear number of potential use cases presented (i.e., domain-specific BERT models, BERT in languages other than English, VideoBERT, and BART.
Con:
1) Examples
The only critique I have is the datasets used are toy data sets. For instance, the example for "Fine-tuning BERT for sentiment analysis" in chapter 3 makes use of the IMDB dataset which is loaded through the Hugging Face dataset API. Since this is a toy dataset, it is already in the format BERT prefers. To that end, it would be incredibly useful to provide examples of how to prepare real datasets for analysis using BERT.
In conclusion, if you are looking for a book which provides an overview, potential use cases, and examples of how to implement BERT, then this book is for you.
And that, "A basic understanding of NLP concepts and deep learning is required to get the most out of this book."
To that end, if you tick both of the above boxes, then this book is DEFINITELY for you.
Pros:
1) Style
The author writes in a conversational manner which makes it incredibly easy to follow along; it is no easy feat to write about complex subjects in an engaging manner.
2) Examples
The code examples are easy to follow and implement. Additionally, the repo on github provides further explanations.
3) Range
What was most illuminating was the shear number of potential use cases presented (i.e., domain-specific BERT models, BERT in languages other than English, VideoBERT, and BART.
Con:
1) Examples
The only critique I have is the datasets used are toy data sets. For instance, the example for "Fine-tuning BERT for sentiment analysis" in chapter 3 makes use of the IMDB dataset which is loaded through the Hugging Face dataset API. Since this is a toy dataset, it is already in the format BERT prefers. To that end, it would be incredibly useful to provide examples of how to prepare real datasets for analysis using BERT.
In conclusion, if you are looking for a book which provides an overview, potential use cases, and examples of how to implement BERT, then this book is for you.
April 16, 2021
This book is a good entry point into modern solutions for different NLP tasks. I think this book is great for readers, who are familiar with neural networks in other fields and want to know what is state of the art in NLP now.
The book begins with an exhaustive description of the building blocks of transformers and BERT, then different use cases are discussed.
The book begins with an exhaustive description of the building blocks of transformers and BERT, then different use cases are discussed.
October 23, 2021
I would recommend this book as the first book to start the journey on BERT models for researchers and developers who are already familiar with NLP and deep learning in general but like to learn BERT for problem solving
Currently reading
March 6, 2021Sudarshan brings amazing clarity into the subject - makes a very good reading on the subject.
Displaying 1 - 5 of 5 reviews