Senior Data Migration Engineer
Innover DigitalBengaluru, Republic Of India, IN6 days ago
Job descriptionHands-on Migration and Development Pipeline Translation : Directly develop and migrate 20+ complex data pipelines from Azure Data Factory (ADF) into optimized Fabric Data Pipelines and Spark Notebooks (PySpark / Scala). Stored Procedure Conversion : Rewrite and optimize 30+ complex stored procedures into robust, scalable transformations using PySpark in Fabric Lakehouses or high-performance T-SQL in Fabric Data Warehouses. Data Modeling : Implement the designed data architecture (Medallion layers : Bronze, Silver, Gold) within OneLake, focusing on Delta Lake best practices and V-Order optimization for efficient querying. Large Data Handling : Execute the migration of 30+ TB of historical data efficiently and accurately, ensuring data integrity and reconciliation between the legacy and Fabric environments. Unit & Integration Testing : Develop and execute comprehensive unit and integration tests to validate the correctness and performance of migrated data pipelines and stored procedures. Performance Tuning : Work closely with the Technical Lead to identify and resolve data bottlenecks and optimize Spark code and Warehouse queries to meet latency requirements. Collaboration and DevOps Support CI / CD Adherence : Operate within the established DevOps pipeline, ensuring all code changes, notebooks, and object definitions are managed under Git version control and deployed via the automated CI / CD framework.
Required Qualifications
- Experience : 5+ years of dedicated experience in ETL / ELT development and data warehousing, with involvement in at least one major cloud data migration.
- Core Skills : Strong, demonstrable experience with the Azure Data Stack (ADF, Synapse) and hands-on development experience in Microsoft Fabric.
- Data Languages : Highly proficient in T-SQL for querying and procedural logic, and mandatory expertise in Python (PySpark) for data transformation.
- Data Modeling : Solid understanding of Kimball-style data warehousing, dimensional modeling, and modern Lakehouse concepts (Delta Lake, Parquet).
- Tools : Experience working with Git for source control and familiarity with the principles of CI / CD deployment.
Create a job alert for this search
Senior Data Engineer • Bengaluru, Republic Of India, IN
Related jobs
NASDAQ : CMPR) is the world leader in mass customisation.Our unmatched technology, production, and supply chain operations allow us to offer products that can be personalised by an individual custom...Show moreLast updated: 26 days ago
- Promoted
Data Migration Consultant
Numeric Technologieshosur, tamil nadu, in Title : SAP Data Conversion Senior Software Engineer.Duration : Permanent - Immediate Joiner.As a Sr Software Engineer (SAP) in Data Migration we expect this developer to participate in project data ...Show moreLast updated: 16 days ago
- Promoted
Senior Data Engineer - Databricks
Tredence Inc.bangalore, karnataka, in Tredence is a global data science solutions provider founded in 2013 by Shub Bhowmick, Sumit Mehra, and Shashank Dubey focused on solving the last-mile problem in AI.
Headquartered in San Jose, Cali...Show moreLast updated: 1 day ago
- Promoted
Senior AWS Data Engineer_Exp : 5+ Years
Atyeti Inchosur, tamil nadu, in Required Skills & Qualifications.Bachelor’s or Master’s degree in Computer Science or equivalent experience.Application Developer or in similar software engineering roles.Python; strong SQL and clo...Show moreLast updated: 5 days ago
- Promoted
Senior Data Engineer (AWS / Databricks)
Accoladehosur, tamil nadu, in The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
- Promoted
Migration Engineer(AWS services : EC2, VPC, ELB, S3, IAM, CloudWatch
Tata Consultancy ServicesBengaluru, Karnataka, India Greetings from TATA Consultancy Services.Migration Engineer(AWS services : EC2, VPC, ELB, S3, IAM, CloudWatch).Pls find the Job Description below.
Perform lift-and-shift migration of on-premises serv...Show moreLast updated: 20 days ago
- Promoted
- New!
Senior Test Engineer (Data Migration)
ZakAppsBengaluru, IN Zakapps is a technology company specializing in retail and digital media solutions.They offer services like native mobile app development, e-commerce solutions, and digital distribution systems.The...Show moreLast updated: 4 hours ago
- Promoted
Senior Data Engineer
SGS & Cohosur, tamil nadu, in Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show moreLast updated: 6 days ago
- Promoted
Senior Data Engineer-Remote
Cimpress Indiahosur, tamil nadu, inRemote
- Promoted
Senior Data Engineer
Incept Labshosur, tamil nadu, in Incept Labs is an AI research lab based in San Francisco, California.Our small team of scientists, engineers, and builders is passionate about developing domain-specific, next-generation AI solutio...Show moreLast updated: 30+ days ago
- Promoted
Senior Snowflake Data Engineer
iVoyanthosur, tamil nadu, in One of our clients is looking for an experienced Senior Snowflake Data Engineer to join their team.We are seeking a Senior Data Engineer with 8+ years of experience in end-to-end data engineering a...Show moreLast updated: 6 days ago
- Promoted
Senior Data Engineer
Futran SolutionsBengaluru, IN We are seeking a highly skilled and experienced Senior Data Engineer with a strong background in Microsoft Azure data platforms.
The ideal candidate will have a proven track record of designing, imp...Show moreLast updated: 17 days ago
- Promoted
Senior Data Engineer
MeghGen TechnologiesBengaluru, Karnataka, India MeghGen Technologies is a trusted technology solutions partner specializing in data engineering, cloud-native applications, and cloud migrations.
We design and implement end-to-end cloud solutions t...Show moreLast updated: 5 days ago
- Promoted
Senior Data Engineer
SAIVA AIhosur, tamil nadu, in We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
HealthyMe AIhosur, tamil nadu, in We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Pretium Enterprise Services, IndiaBengaluru, Karnataka, India Would be a lead responsible for developing data products & solutions for the organization.IT with experience in Data warehouse, ETL, BI projects is required.
Would collaborate within Agile teams to ...Show moreLast updated: 3 days ago
- Promoted
SENIOR DATA ENGINER
Tredence Inc.Bengaluru, Karnataka, India Primary Roles and Responsibilities : .Developing Modern Data Warehouse solutions using Databricks and Azure Stack.Ability to provide solutions that are forward-thinking in data engineering and analyt...Show moreLast updated: 15 days ago
- Promoted
Senior Data Engineer
Mastekhosur, tamil nadu, in We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show moreLast updated: 30+ days ago
- Promoted
Senior Azure Data Engineer
Programmers.iohosur, tamil nadu, in Job Title : Senior Azure Data Engineer.Skills Required : Azure Synapse, Azure Data Factory, Azure Databricks, and ADLS Gen2, PySpark, SQL,Microsoft Fabrics.
Experience Range : 5 Years - 12 years.We are...Show moreLast updated: 30+ days ago