Darum lohnt sich der Kurs
In diesem Training erfahren Sie, wie Sie Apache Spark und Azure Databricks in großen Data-Engineering-Projekten für Streaming, ETL, Delta Live Tables, Monitoring, Performanceoptimierung und automatisierte Workflows einsetzen.Seminarinhalt
Erfahren Sie, wie Sie die Leistungsfähigkeit von Apache Spark und leistungsstarke Cluster auf Basis der Azure Databricks-Plattform nutzen können, um große Data-Engineering-Workloads in der Cloud zu bewältigen.
Programm
Perform incremental processing with spark structured streaming
- Set up real-time data sources for incremental processing
- Optimize Delta Lake for incremental processing in Azure Databricks
- Handle late data and out-of-order events in incremental processing
- Monitoring and performance tuning strategies for incremental processing in Azure Databricks
- Event driven architectures with Delta Live tables
- Ingest data with structured streaming
- Maintain data consistency and reliability with structured streaming
- Scale streaming workloads with Delta Live tables
- Optimize performance with Spark and Delta Live Tables
- Perform cost-based optimization and query tuning
- Use change data capture (CDC)
- Use enhanced autoscaling
- Implement observability and data quality metrics
- Implement version control and Git integration
- Perform unit testing and integration testing
- Manage and configure your environment
- Implement rollback and roll-forward strategies
- Implement job scheduling and automation
- Optimize workflows with parameters
- Handle dependency management
- Implement error handling and retry mechanisms
- Explore best practices and guidelines
- Implement data encryption techniques in Azure Databricks
- Manage access controls in Azure Databricks
- Implement data masking and anonymization in Azure Databricks
- Use compliance frameworks and secure data sharing in Azure Databricks
- Use data lineage and metadata management
- Implement governance automation in Azure Databricks
- Get started with SQL Warehouses
- Create databases and tables
- Create queries and dashboards
- Understand Azure Databricks notebooks and pipelines
- Create a linked service for Azure Databricks
- Use a Notebook activity in a pipeline
- Use parameters in a notebook
Zielgruppen
- Data Engineer
- Developer
- Data Scientist
- Data Analyst
- ELT Developers
Vorkenntnisse
keine
