Azure Data Engineer with Databricks Expertise
Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision! • toronto, Canada
Role Description
Elevate your career as an Azure Data Engineer specializing in Databricks and PySpark. This role emphasizes hands-on experience with cutting-edge technologies like Azure SQL and Data Factory.
You will work directly with Azure services including Databricks, ADLS, and Azure SQL, focusing on ETL/ELT processes and data warehousing solutions. The ideal candidate possesses practical skills in troubleshooting pipeline failures and performance bottlenecks while utilizing CI/CD pipelines through Azure DevOps.
Key Responsibilities:
• Utilize Databricks notebooks for data processing
• Implement ETL/ELT solutions using Azure services
• Troubleshoot pipeline failures and cluster issues
• Monitor performance with Azure Monitor and Databricks logs
• Collaborate with teams using Git and CI/CD pipelines
Requirements:
• Strong expertise in Azure Data Factory and Azure SQL
• Proficient in PySpark, Spark SQL, and Delta Lake
• Experience with data ingestion patterns and ware...
You will work directly with Azure services including Databricks, ADLS, and Azure SQL, focusing on ETL/ELT processes and data warehousing solutions. The ideal candidate possesses practical skills in troubleshooting pipeline failures and performance bottlenecks while utilizing CI/CD pipelines through Azure DevOps.
Key Responsibilities:
• Utilize Databricks notebooks for data processing
• Implement ETL/ELT solutions using Azure services
• Troubleshoot pipeline failures and cluster issues
• Monitor performance with Azure Monitor and Databricks logs
• Collaborate with teams using Git and CI/CD pipelines
Requirements:
• Strong expertise in Azure Data Factory and Azure SQL
• Proficient in PySpark, Spark SQL, and Delta Lake
• Experience with data ingestion patterns and ware...