What do you think?
Rate this book
Cost-Effective Data Pipelines: Balancing Trade-Offs When Developing Pipelines in the Cloud
The low cost of getting started with cloud services can easily evolve into a significant expense down the road. That's challenging for teams developing data pipelines, particularly when rapid changes in technology and workload require a constant cycle of redesign. How do you deliver scalable, highly available products while keeping costs in check? With this practical guide, author Sev Leonard provides a holistic approach to designing scalable data pipelines in the cloud. Intermediate data engineers, software developers, and architects will learn how to navigate cost/performance trade-offs and how to choose and configure compute and storage. You'll also pick up best practices for code development, testing, and monitoring. By focusing on the entire design process, you'll be able to deliver cost-effective, high-quality products. This book helps
- GenresComputer Science
286 pages, Paperback
Published August 22, 2023
1 person is currently reading
32 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 of 1 review
August 14, 2024
Data Engineering (Domain) | Applied (Usability) | Intermediate (Level)
Summary : Data pipelines are the cornerstone of data engineering, and this book offers numerous practical tips on how to design them effectively. With valuable lessons drawn from hands-on experience, the author emphasizes trade-off analysis, guiding the reader through the complex landscape of scalable data organization, pipeline testing, logging, and monitoring. The book is particularly useful for intermediate-level data engineers, software developers, and architects looking to optimize their data pipelines for cost and performance in the cloud.
Top 5 takeways :
1. Structured Data Environments : Use separate environments for development (small sample data), testing (larger sample data), validation (production-like data), and production (tested on streaming data) to ensure thorough and efficient pipeline development.
2.
3.
4.
5.
Summary : Data pipelines are the cornerstone of data engineering, and this book offers numerous practical tips on how to design them effectively. With valuable lessons drawn from hands-on experience, the author emphasizes trade-off analysis, guiding the reader through the complex landscape of scalable data organization, pipeline testing, logging, and monitoring. The book is particularly useful for intermediate-level data engineers, software developers, and architects looking to optimize their data pipelines for cost and performance in the cloud.
Top 5 takeways :
1. Structured Data Environments : Use separate environments for development (small sample data), testing (larger sample data), validation (production-like data), and production (tested on streaming data) to ensure thorough and efficient pipeline development.
2.
3.
4.
5.
Displaying 1 of 1 review