Data engineer
NLB Services • ciudad nezahualcóyotl, Mexico
Role Description
Role: Data EngineerLocation: Remote (Mexico)
Type: Full Time
Data Engineer
Engineer will be part of the data store-migration Factory team that will be responsible for performing the end-to-end data store migration from on-prem Data Lake to AWS hosted Lakehouse.
Responsibilities of the Engineer include:
1. Pipeline Migration
a. Logic & Scheduling: Refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment.
Data Transfer: Executing the physical migration of underlying datasets while ensuring data integrity.
Stakeholder Engagement: Acting as a technical liaison to internal clients, facilitating hand-off and sign-off conversations with data owners to ensure migrated assets meet business requirements.
2. Consumption Pattern Migration
Code Conversion: Translating and optimizing legacy SQL and Spark-based consumption patterns (raw and modeled) for compatibility with Snowflake and Iceberg.
Usage analysi...
Type: Full Time
Data Engineer
Engineer will be part of the data store-migration Factory team that will be responsible for performing the end-to-end data store migration from on-prem Data Lake to AWS hosted Lakehouse.
Responsibilities of the Engineer include:
1. Pipeline Migration
a. Logic & Scheduling: Refactoring and migrating extraction logic and job scheduling from legacy frameworks to the new Lakehouse environment.
Data Transfer: Executing the physical migration of underlying datasets while ensuring data integrity.
Stakeholder Engagement: Acting as a technical liaison to internal clients, facilitating hand-off and sign-off conversations with data owners to ensure migrated assets meet business requirements.
2. Consumption Pattern Migration
Code Conversion: Translating and optimizing legacy SQL and Spark-based consumption patterns (raw and modeled) for compatibility with Snowflake and Iceberg.
Usage analysi...