Organizations collecting and using personal data must now heed a growing body of regulations, and the penalties for noncompliance are stiff. The ubiquity of the cloud and the advent of generative AI have only made it more crucial to govern data appropriately. Thousands of companies have turned to Databricks Unity Catalog to simplify data governance and manage their data and AI assets more effectively. This practical guide helps you do the same.
Databricks data specialists Kiran Sreekumar and Karthik Subbarao dive deep into Unity Catalog and share the best practices that enable data practitioners to build and serve their data and AI assets at scale. Data product owners, data engineers, AI/ML engineers, and data executives will examine various facets of data governance—including data sharing, auditing, access controls, and automation—as they discover how to establish a robust data governance framework that complies with regulations.
Explore data governance fundamentals and understand how they relate to Unity Catalog Utilize Unity Catalog to unify data and AI governance Access data efficiently for analytics Implement different data protection mechanisms Securely share data and AI assets internally and externally with Delta Sharing
I'd easily consider this book's contents as required reading if you're working with Databricks on a daily basis, especially if you need to design architectures where Unity Catalog will be one of the central tenets.
The book not only provides the reader with a very good historical perspective, making the motivation and reasoning behind some current technical features clear, but it also presents how each feature fits together to provide a strong data platform foundation.
I've particularly liked the chapter on how to organize Unity Catalog for various levels of business complexity; how to make optimal use of the three-level namespace of catalogs, schemas, and data assets. On top of that, chapter organizing access control is another winner; it provides many different, concrete scenarios for Role Based Access Control (RBAC) and Fine Grained Access Control (FGAC). Linking these aspects to Workspace organization is another topic tackled in this book, which I'm sure to revisit.
When it comes to Databricks, it's not easy to go wide and deep, because the company constantly adds new features and what is today's limitation can be tomorrow's already-solved problem. Nevertheless, this book does almost a perfect job of organizing Databricks so that it can serve all sorts of team structures in differently sized companies.
If you need a deep dive on data engineering, orchestration, Python, etc. you'll need to find other resources but if you're really into designing and implementing the architecture of a data platform using Databricks, then you can't go wrong with this excellent book.