Sr. Data Engineer - Databricks

Are you a highly skilled Data Engineer with expertise in Databricks ready to lead a major data platform modernization? Apply Now!

Working with one of our top financial clients, this role calls for a Sr. Data Engineer - Databricksspecializing in Databricks and Data Platform Migration. This position offers a compelling opportunity to work on a large-scale initiative to transition trading and market data from legacy systems into modern, scalable data frameworks, supporting strategic modernization efforts within Capital Markets.

Responsibilities

• Design and develop data ingestion pipelines, both batch and real-time, leveraging Databricks platform.
• Build and optimize Spark/Scala-based data processing workflows for high performance and reliability.
• Implement streaming ingestion solutions using Kafka, APIs, or similar technologies.
• Develop medallion architecture models (bronze, silver, gold layers) consistent with best practic...