Experienced Databricks Developer Role
Atlas Corporation • vancouver, Canada
Role Description
Join Seaspan Corporation as an experienced Databricks Developer focused on building scalable data pipelines and leveraging Apache Spark for enterprise analytics. This role emphasizes strong knowledge of data transformation and integration.
The ideal candidate will possess over 5 years in data engineering, working with Databricks, PySpark, and SQL to develop efficient ETL/ELT processes. You will collaborate closely with data teams to create unified datasets that enhance analytics and reporting. A deep understanding of Delta Lake and Azure Data Factory is essential to succeed in this impactful role.
Key Responsibilities:
• Design and maintain scalable data pipelines in Databricks
• Develop ETL/ELT pipelines for varied data types
• Optimize Spark jobs and manage Delta Lake performance
• Implement data ingestion pipelines for RESTful APIs
• Collaborate with Data Scientists and Analysts for quality datasets
Requirements:
The ideal candidate will possess over 5 years in data engineering, working with Databricks, PySpark, and SQL to develop efficient ETL/ELT processes. You will collaborate closely with data teams to create unified datasets that enhance analytics and reporting. A deep understanding of Delta Lake and Azure Data Factory is essential to succeed in this impactful role.
Key Responsibilities:
• Design and maintain scalable data pipelines in Databricks
• Develop ETL/ELT pipelines for varied data types
• Optimize Spark jobs and manage Delta Lake performance
• Implement data ingestion pipelines for RESTful APIs
• Collaborate with Data Scientists and Analysts for quality datasets
Requirements: