Senior Data Migration Engineer
ConfidentialBengaluru / Bangalore22 days ago
Job descriptionWork with Databricks platform for big data processing and analytics. Develop and maintain ETL processes using Databricks notebooks. Implement and optimize data pipelines for data transformation and integration. Utilize AWS services (e.g., S3, Glue, Redshift, Lambda) and Databricks to build and optimize data migration pipelines. Leverage PySpark for large-scale data processing and transformation tasks. Stay updated on the latest industry trends, tools, and technologies related to Python, SQL, and Databricks. Share knowledge with the team and contribute to a culture of continuous improvement. Utilize expertise in SQL to design, optimize, and maintain relational databases. Write complex SQL queries for data retrieval, manipulation, and analysis. Education : Bachelor s degree in Computer Science, Engineering, Data Science, or a related field. Advanced degrees are a plus. 4 to 8 Years of experience in Databricks and big data frameworks Proficient in AWS services and data migration Experience in Unity Catalogue Familiarity with Batch and real time processing Data engineering with strong skills in Python, PySpark, SQL Certifications : AWS Certified Solutions Architect, Databricks Certified Professional, or similar are a plus. Strong problem-solving and analytical skills. Excellent communication and collaboration abilities. Ability to work in a fast-paced, agile environment. Design, develop, test, and deploy high-performance and scalable data solutions using Python, PySpark, SQL Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications. Implement efficient and maintainable code using best practices and coding standards. Work with Databricks platform for big data processing and analytics. Develop and maintain ETL processes using Databricks notebooks. Implement and optimize data pipelines for data transformation and integration. Utilize AWS services (e.g., S3, Glue, Redshift, Lambda) and Databricks to build and optimize data migration pipelines. Leverage PySpark for large-scale data processing and transformation tasks. Stay updated on the latest industry trends, tools, and technologies related to Python, SQL, and Databricks. Share knowledge with the team and contribute to a culture of continuous improvement. Utilize expertise in SQL to design, optimize, and maintain relational databases. Write complex SQL queries for data retrieval, manipulation, and analysis.
Job Summary :
We are seeking a skilled Senior Data Migration Engineer with expertise in AWS, Databricks, Python, PySpark, and SQL to lead and execute complex data migration projects. The ideal candidate will design, develop, and implement data migration solutions to move large volumes of data from legacy systems to modern cloud-based platforms, ensuring data integrity, accuracy, and minimal downtime.
Job Responsibilities
Software Development :
- Design, develop, test, and deploy high-performance and scalable data solutions using Python, PySpark, SQL
- Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications.
- Implement efficient and maintainable code using best practices and coding standards.
AWS & Databricks Implementation :
Continuous Learning :
SQL Database Management :
Qualifications & Skills :
Soft Skills :
Career Level - IC2
Job Responsibilities
Software Development :
AWS & Databricks Implementation :
Continuous Learning :
SQL Database Management :
Skills Required
Aws, Databricks, Python, Pyspark, Sql
Create a job alert for this search
Senior Data Engineer • Bengaluru / Bangalore
Related jobs
![Senior Engineer Data [T500-20882]](https://cdn-logos.talent.com/v1/logo/image?feedcode=jobtraffic-in&company_name=Albertsons%20Companies%20India)
- Promoted
Senior Data Engineer
TEKsystems Global Services in IndiaBengaluru, Karnataka, India Develop and manage data pipelines using ADF and Azure data bricks.Expert knowledge in data warehousing, data modeling and ETL.
Good to have datastage exposure.Expertise in on-prem to azure cloud mig...Show moreLast updated: 30+ days ago
- Promoted
Senior Engineer Data [T500-20882]
Albertsons Companies IndiaBengaluru, Karnataka, India About Albertsons Companies Inc.As a leading food and drug retailer in the United States, Albertsons Companies, Inc.Our well-known banners across the United States, including Albertsons, Safeway, Vo...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Nityo InfotechBengaluru, Karnataka, India We are seeking an experienced Senior Data Engineer with 8-12 years of hands-on experience to join our team on-site.This role requires a self-starter who can architect and implement robust data foun...Show moreLast updated: 26 days ago
- Promoted
Senior Data Engineer
CargillBengaluru, Karnataka, India The Senior Professional, Data Engineering designs, builds, and maintains complex data systems that enable data analysis and reporting.
This role includes a focus on Treasury-specific data solutions,...Show moreLast updated: 9 days ago
- Promoted
Senior Data Engineer
VeraxionBengaluru, Karnataka, India We’re Hiring : Senior Data Engineer (Azure) We are looking for a Senior Data Engineer who can design, build, and scale modern data platforms that power analytics, decision-making, and AI across th...Show moreLast updated: 30+ days ago
- Promoted
- New!
Data engineer - AWS - Inperson drive
Live ConnectionsGreater Bengaluru Area, India Show moreLast updated: 21 hours ago
- Promoted
Senior Data Engineer
Relevance LabBengaluru, Karnataka, India Design, build and maintain enterprise data models and data assets on Microsoft Azure to support analytics, reporting, ML / AI and data products.
Translate business requirements into robust logical and...Show moreLast updated: 4 days ago
- Promoted
Senior Data Engineer
SAIVA AIBengaluru, IN We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Serdes Lead Engineer
Mulya TechnologiesGreater Bengaluru Area, India About Omni Design Technologies.Omni Design Technologies is a leading provider of high-performance, ultra-low power IP cores, from 28nm down through advanced FinFET nodes, which enable differentiate...Show moreLast updated: 21 hours ago
- Promoted
Senior Data Engineer
QuantiphiGreater Bengaluru Area, India Job Title : Senior Data Engineer (Power BI).Location : Bangalore / Mumbai / Hybrid.We are seeking a skilled and proactive.
The ideal candidate will be responsible for designing, developing, and publish...Show moreLast updated: 30+ days ago
- Promoted
Senior AWS Data Engineer
CYAN360Bengaluru, IN Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •.
Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
ACL DigitalBengaluru, Karnataka, India Tech Lead design, build, and maintain the infrastructure for an organization's data, ensuring it is reliable, accessible, and optimized for use by data scientists and analysts.Key responsibilities ...Show moreLast updated: 17 days ago
- Promoted
Senior Azure Data Engineer
Tata Consultancy ServicesGreater Bengaluru Area, India Job Title : Senior Azure Data Engineer.Required Skillset : Azure data factory, data bricks, data lake, automation and performance optimization of ETL.
Good Knowledge of Data Brick lakehouse and Azure ...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Database Reliability Engineer
Tata Consultancy ServicesGreater Bengaluru Area, India Senior Database Reliability Engineer.Snowflake, MongoDB / PostgreSQL, MySQL, Oracle DB.Experience with Terraform, Ansible, Git & Jenkins is must.
Strong understanding of cloud platforms (AWS, Azure)...Show moreLast updated: 21 hours ago
- Promoted
Senior Data Engineer
Pretium Enterprise Services, IndiaBengaluru, Karnataka, India Would be a lead responsible for developing data products & solutions for the organization.IT with experience in Data warehouse, ETL, BI projects is required.
Would collaborate within Agile teams to ...Show moreLast updated: 15 days ago
- Promoted
Senior Data Engineer
SutherlandBangalore Urban, Karnataka, India Location : Bangalore | Hyderabad | Chennai | Mumbai.If interested please reach out to me with your updated resume @Priti.
We’re transforming how we work by harnessing the power of AI, data, and digit...Show moreLast updated: 18 days ago
- Promoted
- New!
Senior Data Engineer
ArcanaGreater Bengaluru Area, India Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Senior Data Engineer.Embark on a journey where your deep-rooted expertise in computer science fundamentals, a...Show moreLast updated: 21 hours ago
- Promoted
Senior Data Engineer(Big Data)
Andor TechBengaluru, Karnataka, India Andor Tech is looking for a highly skilled Senior Big Data Engineer with 8–15 years of experience to join our dynamic team.
If you're passionate about big data technologies and thrive in solving com...Show moreLast updated: 18 days ago