Principal Data Migration Specialist (Databricks)

Role : Databricks Senior Developer

Location : Kolkata

Experience : 8 Plus Years

Role

• Databricks Senior Developer

Required Technical Skill Set

Desired Competencies (Technical / Behavioral Competency)

Must-Have

1. 8+ years with strong Databricks, Spark and Data Migration experience

2. Should have experience of end-to-end migration from AWS Redshift or other RDS to Databricks Lakehouse Platform. Need to have knowledge in Cloud Data Engineering, ETL Modernization, Spark Optimization and SQL-to-PySpark migration.

3. Key Responsibilities :

a) Migration Strategy and Execution : analyze the existing DWH / DB architecture, Design & Implement scalable data pipelines in Databricks, SQL Script conversion, etc.

b) Development & Optimization : Build Delta Lake tables and Implement ACID-complaint data models, Manage metadata, schema evolution and lineage for migrated datasets

Mentor junior members in the team

Exposure to machine learning / AI pipelines, etc.