Lead Data Engineer for Databricks Migration
NLB Services • hamilton, Canada
Role Description
Join a dynamic team focused on migrating from Glue and Redshift to Databricks. We're seeking a Lead Data Engineer with 10+ years of experience to spearhead this initiative.
In this role, you will lead the migration project, leveraging your expertise in Apache Spark, Databricks, and AWS services. Your hands-on experience will be pivotal as you manage key areas of data engineering, ensuring optimal performance and alignment with best practices. Strong communication skills will be essential for managing relationships between stakeholders and engineers throughout the project.
Key Responsibilities:
• Lead the migration from Glue/Redshift to Databricks
• Tune performance with Apache Spark and optimize SQL processes
• Oversee production workloads using Databricks and AWS Glue
• Develop and document migration plans for stakeholders
• Mentor team members in best practices of data engineering
Requirements:
• 10+ years in ...
In this role, you will lead the migration project, leveraging your expertise in Apache Spark, Databricks, and AWS services. Your hands-on experience will be pivotal as you manage key areas of data engineering, ensuring optimal performance and alignment with best practices. Strong communication skills will be essential for managing relationships between stakeholders and engineers throughout the project.
Key Responsibilities:
• Lead the migration from Glue/Redshift to Databricks
• Tune performance with Apache Spark and optimize SQL processes
• Oversee production workloads using Databricks and AWS Glue
• Develop and document migration plans for stakeholders
• Mentor team members in best practices of data engineering
Requirements:
• 10+ years in ...