The demand for data scientists is well-known, but when it comes time to build solutions based on data, your company also needs data engineers—people with strong data warehousing and programming backgrounds. In fact, whether you’re powering self-driving cars or creating music playlists, this field has emerged as one of the most important in modern business. In this report, Lewis Gavin explores key aspects of data engineering and presents a case study from Spotify that demonstrates the tremendous value of this role.
Data engineers are usually more technical with strong data warehousing and programming backgrounds. Data scientists tend to be more mathematical, but there is a lot of crossover between the roles, notably in programming, as machine learning models usually require writing small applications and heavy data manipulation. It all depends on where your strengths lie and which aspects you enjoy the most.
This book provides a comprehensive understanding of the role of a data engineer, distinguishing it from that of a data scientist and examining the synergy between the two.