Master end-to-end data engineering on Azure Databricks. From data ingestion and Delta Lake to CI/CD and real-time streaming, build secure, scalable, and performant data solutions with Spark, Unity Catalog, and ML tools.
Key Features:
- Build scalable data pipelines using Apache Spark and Delta Lake
- Automate workflows and manage data governance with Unity Catalog
- Learn real-time processing and structured streaming with practical use cases
- Implement CI/CD, DevOps, and security for production-ready data solutions
- Explore Databricks-native ML, AutoML, and Generative AI integration
Book Description:
Data Engineering with Azure Databricks is your essential guide to building scalable, secure, and high-performing data pipelines using the powerful Databricks platform on Azure. Designed for data engineers, architects, and developers, this book demystifies the complexities of Spark-based workloads, Delta Lake, Unity Catalog, and real-time data processing.
Beginning with the foundational role of Azure Databricks in modern data engineering, you'll explore how to set up robust environments, manage data ingestion with Auto Loader, optimize Spark performance, and orchestrate complex workflows using tools like Azure Data Factory and Airflow.
The book offers deep dives into structured streaming, Delta Live Tables, and Delta Lake's ACID features for data reliability and schema evolution. You'll also learn how to manage security, compliance, and access controls using Unity Catalog, and gain insights into managing CI/CD pipelines with Azure DevOps and Terraform.
With a special focus on machine learning and generative AI, the final chapters guide you in automating model workflows, leveraging MLflow, and fine-tuning large language models on Databricks. Whether you're building a modern data lakehouse or operationalizing analytics at scale, this book provides the tools and insights you need.
What You Will Learn:
- Set up a full-featured Azure Databricks environment
- Implement batch and streaming ingestion using Auto Loader
- Optimize Spark jobs with partitioning and caching
- Build real-time pipelines with structured streaming and DLT
- Manage data governance using Unity Catalog
- Orchestrate production workflows with jobs and ADF
- Apply CI/CD best practices with Azure DevOps and Git
- Secure data with RBAC, encryption, and compliance standards
- Use MLflow and Feature Store for ML pipelines
- Build generative AI applications in Databricks
Who this book is for:
This book is for data engineers, solution architects, cloud professionals, and software engineers seeking to build robust and scalable data pipelines using Azure Databricks. Whether you're migrating legacy systems, implementing a modern lakehouse architecture, or optimizing data workflows for performance, this guide will help you leverage the full power of Databricks on Azure. A basic understanding of Python, Spark, and cloud infrastructure is recommended.
Table of Contents
- The role of Azure Databricks in modern data engineering
- Setting up an end-to-end Azure Databricks environment
- Data ingestion strategies for Azure Databricks
- Deep dive into Apache Spark on Azure Databricks
- Streaming architectures with structured streaming
- Working with Delta Lake: ACID transactions & schema evolution
- Automating data pipelines with Delta Live Tables (DLT)
- Orchestrating data workflows: from notebooks to production
- CI/CD and DevOps for Azure Databricks
- Optimizing query performance and cost management
- Security, compliance, and data governance
- Machine learning, AutoML, and generative AI in Databricks
外文書商品之書封，為出版社提供之樣本。實際出貨商品，以出版社所提供之現有版本為主。部份書籍，因出版社供應狀況特殊，匯率將依實際狀況做調整。
無庫存之商品，在您完成訂單程序之後，將以空運的方式為你下單調貨。為了縮短等待的時間，建議您將外文書與其他商品分開下單，以獲得最快的取貨速度，平均調貨時間為1~2個月。
為了保護您的權益，「三民網路書店」提供會員七日商品鑑賞期(收到商品為起始日)。
若要辦理退貨，請在商品鑑賞期內寄回，且商品必須是全新狀態與完整包裝(商品、附件、發票、隨貨贈品等)否則恕不接受退貨。
