Lead Data Integration Engineer
InfoCeptsPune, Republic Of India, IN1 day ago
Job descriptionIn-depth knowledge of the following AWS services : S3, EC2, EMR, Athena, AWS Glue, Lambda Experience with at least one MPP database : AWS Redshift, Snowflake, SingleStore Proficiency in Big Data technologies : Apache Spark, Databricks Must have strong programming skills in Python Responsible for building data pipelines in AWS And Databricks Experience with Big Data table formats, such as Delta Lake (open source) Must have very strong SQL skills Experience with orchestration tools like Apache Airflow Expertise in developing ETL workflows with complex transformations such as SCD, deduplications, aggregations, etc. Should be a quick and self-learner, ready to adapt to new AWS services or Big Data technologies as required Strong understanding of data warehousing concepts Cloud Databases – Snowflake, AWS Aurora Big Data – Hadoop, Hive Cloud Databases– AWS Aurora Associate Level or Professional Level AWS Certification or Databricks Certification Overall, 7+ years of IT Exp 5+ years of relevant experience in AWS related project Bachelor’s degree in computer science, engineering, or related field (master's degree is a plus) Demonstrated continued learning through one or more technical certifications or related methods. Hold strong technical knowledge and experience Should have the capability to deep dive and research in various technical related fields Self-motivated and focused on delivering outcomes for a fast-growing team and firm Able to communicate persuasively through speaking, writing, and client presentations Able to consult, write, and present persuasively Able to work in a self-organized and cross-functional team Able to iterate based on new information, peer reviews, and feedback Prior experience of working in a large media company would be added advantage
Key Result Areas and Activities :
- Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements.
- Should be able to design complex Big Data use cases using AWS Services and Databricks, under guidance of Architect
- Should be able to assist the architect in reasoning about various architecture choices in terms of cost, performance and durability.
- Should be able to suggest optimizations in existing designs.
- Project documentation adheres to quality guidelines & schedules.
- Works hand in hand with Architect & PM for successful delivery of project and provide estimation, scoping, scheduling assistance.
- Articulate design decisions very clearly with stakeholders.
- Perform Proof-of-Concepts and document all observations before proposing a new solution.
- Conduct design review sessions with other teams and identify scope of improvement.
- Suggests automation to improve existing processes.
Work and Technical Experience :
Must-Have :
Good to have :
Qualifications :
Qualities :
Create a job alert for this search
Data Integration Engineer • Pune, Republic Of India, IN
Related jobs
- Promoted
Data Integration Developer (Level 2)
BluticNagpur, IN Data Integration Developer (Level 2).ETL processes, reports, and database solutions using.This role requires solid technical skills to implement solutions, optimize code, and collaborate effectivel...Show moreLast updated: 1 day ago
- Promoted
AI / ML & Data Engineer
Mindfire SolutionsNagpur, IN We are looking for an experienced AI / ML & Data Engineer to design, develop, and deploy scalable machine learning models and data infrastructure on AWS.
You will work closely with cross-functional te...Show moreLast updated: 14 days ago
- Promoted
Senior Data Engineer
HealthyMe AINagpur, IN We are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data.You will report to the Head of AI and Engineering and be responsible for designi...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Guidanz IncNagpur, IN BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show moreLast updated: 1 day ago
- Promoted
Integration Engineer
BrightEdgeNagpur, IN Integration Engineer – Brightedge.Please note - Candidates with Service / Consulting organization background OR not willing to work in EST / PST timezone OR Focused on Dell Boomi / Mulesoft Integra...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
DigitalzoneNagpur, IN As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.
Develop and maintain scalable ETL / ELT pipelines using Py...Show moreLast updated: 1 day ago
- Promoted
Lead Data Engineer
Cimpressnagpur, maharashtra, in Our Team : Enterprise Business Solutions.Vista’s Enterprise Business Solutions (EBS) domain is working to make our company one of the most data-driven organizations to support Finance, Supply Chain,...Show moreLast updated: 1 day ago
- Promoted
Integration Engineer | 3–6 YoE | Join a Fast-Growing HealthTech Startup in Pune
GeektrustNagpur, IN Are you passionate about building systems that make a.HL7, FHIR, and DICOM standards.Epic, Cerner, NextGen, eClinicalWorks.
Implementing and configuring integration engines like.Collaborating with c...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
Response Informaticsnagpur, maharashtra, in AWS services : Must be proficient in building scalable data pipelines and managing cloud-native ETL workflows.Snowflake : Moderate understanding of Snowflake architecture.
CICD - Terraform or CloudFo...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Masteknagpur, maharashtra, in We are looking for a Senior Data Engineer with good exposure to Cloud Services in AWS(IAM, RDS, Lambda, DMS) along with Snowflake / Postgresql, ETL and Streaming Services like Kafka.Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
RecroNagpur, IN Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
SGS & Conagpur, maharashtra, in Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show moreLast updated: 1 day ago
- Promoted
Data Engineer II
ClearDemandNagpur, IN Building on the foundation of the SDE-I role, the DE- II position takes on a greater level of responsibility and leadership.
You'll play a crucial role in driving the evolution and efficiency of our...Show moreLast updated: 1 day ago
- Promoted
Data Integration & LLM Engineer
Chargebeenagpur, maharashtra, in We are seeking a highly motivated.This role is ideal for engineers who enjoy working at the intersection of.APIs, SaaS connectors, and ETL / ELT pipelines to ensure reliable and scalable data flows.B...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
SAIVA AINagpur, IN We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Data Engineer
ACL DigitalNagpur, IN Tech Lead design, build, and maintain the infrastructure for an organization's data, ensuring it is reliable, accessible, and optimized for use by data scientists and analysts.Key responsibilities ...Show moreLast updated: 9 hours ago
- Promoted
Senior Data Integration Engineer
IndsafriIndia, India Senior Data Integration Engineer.We are seeking a highly experienced and strategic Senior Data Integration Engineer to architect, build, and manage the data pipelines that power our customer intell...Show moreLast updated: 21 days ago
- Promoted
Data Engineer
Insight GlobalNagpur, IN GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts.
Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago