Building Medallion Architectures: Designing with Delta Lake and Spark

In today's data-driven world, organizations must manage and analyze vast amounts of information to deliver the insights that give them a competitive advantage. Many turn to the medallion architecture because it's a proven and well-known design. Yet implementing a robust data pipeline can be difficult, particularly when it comes to using the medallion architecture's bronze, silver, and gold layers--done wrong, it can hamper your ability to make data-driven decisions. This practical guide helps you build a medallion architecture the right way with Azure Databricks and Microsoft Fabric.

Drawing on hands-on experience from the field, Piethein Strengholt demystifies common assumptions and complex problems you'll face when embarking on a new data architecture. Architects and engineers of all stripes will find answers to the most typical questions along with insights from real organizations about what's worked, what hasn't, and why.

You'll learn:

• Lakehouse and medallion architecture fundamentals and key concepts
• Design considerations for Azure Databricks and Microsoft Fabric
• Scaling considerations, including governance, security, automation, and more
• How to make informed decisions when designing or implementing new data architectures
• Proven patterns for success that align with broader organizational objectives