Senior Data Migration EngineerInnover Digital • Bengaluru, Republic Of India, IN
No longer accepting applications
Senior Data Migration Engineer
Innover Digital • Bengaluru, Republic Of India, IN
24 days ago
Job descriptionHands-on Migration and Development Pipeline Translation : Directly develop and migrate 20+ complex data pipelines from Azure Data Factory (ADF) into optimized Fabric Data Pipelines and Spark Notebooks (PySpark / Scala). Stored Procedure Conversion : Rewrite and optimize 30+ complex stored procedures into robust, scalable transformations using PySpark in Fabric Lakehouses or high-performance T-SQL in Fabric Data Warehouses. Data Modeling : Implement the designed data architecture (Medallion layers : Bronze, Silver, Gold) within OneLake, focusing on Delta Lake best practices and V-Order optimization for efficient querying. Large Data Handling : Execute the migration of 30+ TB of historical data efficiently and accurately, ensuring data integrity and reconciliation between the legacy and Fabric environments. Unit & Integration Testing : Develop and execute comprehensive unit and integration tests to validate the correctness and performance of migrated data pipelines and stored procedures. Performance Tuning : Work closely with the Technical Lead to identify and resolve data bottlenecks and optimize Spark code and Warehouse queries to meet latency requirements. Collaboration and DevOps Support CI / CD Adherence : Operate within the established DevOps pipeline, ensuring all code changes, notebooks, and object definitions are managed under Git version control and deployed via the automated CI / CD framework.
Required Qualifications
- Experience : 5+ years of dedicated experience in ETL / ELT development and data warehousing, with involvement in at least one major cloud data migration.
- Core Skills : Strong, demonstrable experience with the Azure Data Stack (ADF, Synapse) and hands-on development experience in Microsoft Fabric.
- Data Languages : Highly proficient in T-SQL for querying and procedural logic, and mandatory expertise in Python (PySpark) for data transformation.
- Data Modeling : Solid understanding of Kimball-style data warehousing, dimensional modeling, and modern Lakehouse concepts (Delta Lake, Parquet).
- Tools : Experience working with Git for source control and familiarity with the principles of CI / CD deployment.
Create a job alert for this search
Senior Data Engineer • Bengaluru, Republic Of India, IN
Related jobs
Fenx Data Migration Engineer
Luxoft • Bengaluru, Republic Of India, IN
Fenergo is a leading provider of Client Lifecycle Management, AML / KYC Compliance, and Client Data Management solutions for investment, corporate, commercial, and private banks.Luxoft is looking for...Show more
Last updated: 2 days ago • Promoted
Lead Data Migration Engineer
Confidential • Bengaluru / Bangalore
Perform gap analysis on incoming data sets, offering guidance to clients on how to handle gaps and other data related challenges.
Quickly troubleshoot complex technical issues and roll out fixes wit...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer(Big Data)
Andor Tech • Bengaluru, India
Andor Tech is looking for a highly skilled Senior Big Data Engineer with 8–15 years of experience to join our dynamic team.
If you're passionate about big data technologies and thrive in solving com...Show more
Last updated: 23 days ago • Promoted
Data Migration and Integration Engineer
Tata Consultancy Services • Bengaluru, Republic Of India, IN
Required Technical Skill Set : Exp of Moving Informatica Power Centre to Cloud, IICS, Python, API.Location : Kolkata / Delhi / Bangalore / Chennai.
Exp of Moving Informatica Power Centre to Cloud, IICS, P...Show more
Last updated: 1 day ago • Promoted
Data Migration Engineer
Luxoft • Bengaluru, Republic Of India, IN
This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure.
The goal is to enhance data access...Show more
Last updated: 3 days ago • Promoted
Senior Data Migration Engineer
Confidential • Bengaluru / Bangalore
We are seeking a skilled Senior Data Migration Engineer with expertise in AWS, Databricks, Python, PySpark, and SQL to lead and execute complex data migration projects.
The ideal candidate will desi...Show more
Last updated: 28 days ago • Promoted
FenX Data Migration Engineer - Bangalore
Luxoft • Bengaluru, Karnataka, India
Fenergo is a leading provider of Client Lifecycle Management, AML / KYC Compliance, and Client Data Management solutions for investment, corporate, commercial, and private banks.Luxoft is looking for...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer
Publicis Production • Bangalore Urban, Karnataka, India
We are seeking a proactive and self-motivated Senior Data Engineer with a proven track record in building scalable cloud-based data solutions across multiple cloud platforms to support our work in ...Show more
Last updated: 3 days ago • Promoted
Senior Data Engineer
ACL Digital • Bengaluru, India
Tech Lead design, build, and maintain the infrastructure for an organization's data, ensuring it is reliable, accessible, and optimized for use by data scientists and analysts.Key responsibilities ...Show more
Last updated: 22 days ago • Promoted
Fenx Data Migration Engineer - Bangalore
Luxoft • Bengaluru, Republic Of India, IN
Fenergo is a leading provider of Client Lifecycle Management, AML / KYC Compliance, and Client Data Management solutions for investment, corporate, commercial, and private banks.Luxoft is looking for...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer
London Stock Exchange Group • Bangalore, India
Senior Data Engineer- Customer Lifecycle Engineering, CMR ABOUT US : LSEG (London Stock Exchange Group) is more than a diversified global financial markets infrastructure and data business.We are de...Show more
Last updated: 30+ days ago • Promoted
Senior Cloud Data Engineer
UST • Bengaluru, Republic Of India, IN
We are seeking an experienced Cloud Database Engineer to design, engineer, and deliver best-in-class cloud database services.
This role involves building scalable, secure, and resilient database pla...Show more
Last updated: 3 days ago • Promoted
Senior Data Migration Engineer
Luxoft • Bengaluru, Republic Of India, IN
Fenergo is a leading provider of Client Lifecycle Management, AML / KYC Compliance, and Client Data Management solutions for investment, corporate, commercial, and private banks.Luxoft is looking for...Show more
Last updated: 2 days ago • Promoted
Senior Data Engineer
Pretium Enterprise Services, India • Bengaluru, Karnataka, India
Would be a lead responsible for developing data products & solutions for the organization.IT with experience in Data warehouse, ETL, BI projects is required.
Would collaborate within Agile teams to ...Show more
Last updated: 21 days ago • Promoted
SENIOR DATA ENGINER
Tredence Inc. • Bengaluru, Karnataka, India
Primary Roles and Responsibilities : .Developing Modern Data Warehouse solutions using Databricks and Azure Stack.Ability to provide solutions that are forward-thinking in data engineering and analyt...Show more
Last updated: 30+ days ago • Promoted
Senior Data Engineer
TEKsystems Global Services in India • Bengaluru, Karnataka, India
Develop and manage data pipelines using ADF and Azure data bricks.Expert knowledge in data warehousing, data modeling and ETL.
Good to have datastage exposure.Expertise in on-prem to azure cloud mig...Show more
Last updated: 30+ days ago • Promoted
PLM Data Migration Engineer
L&T Technology Services • Bengaluru, Republic Of India, IN
DEXPERIENCE Migration Specialist with EDAT Tool.Job location - Mysore, Chennai, Bangalore, HYD, Mumbai & Pune.Strong knowledge of PLM systems (preferably 3DEXPERIENCE).
Experience with EDAT DS tool ...Show more
Last updated: 19 days ago • Promoted
Data Migration Engineer
Confidential • Bengaluru / Bangalore
Construct and schedule automated processes and create custom processing scripts.Regularly troubleshoot complex client application, integration and environment issues.
Troubleshoot and resolve custom...Show more
Last updated: 30+ days ago • Promoted