Data Engineering LeadImpetus • Chennai, Republic Of India, IN

Data Engineering Lead

Impetus • Chennai, Republic Of India, IN

17 hours ago

Job description

About Impetus

Impetus Technologies is a digital engineering company focused on delivering expert services and products to help enterprises achieve their transformation goals. We solve the analytics, AI, and cloud puzzle, enabling businesses to drive unmatched innovation and growth.

Founded in 1991, we are cloud and data engineering leaders providing solutions to fortune 100 enterprises, headquartered in Los Gatos, California, with development centers in NOIDA, Indore, Gurugram, Bengaluru, Pune, and Hyderabad with over 3000 global team members. We also have offices in Canada and Australia and collaborate with a number of established companies, including American Express, Bank of America, Capital One, Toyota, United Airlines, and Verizon.

Job Description

Architect and implement enterprise-grade data migration solutions using Java and Python, enabling seamless data transfers from on-premises to GCP (Cloud Storage, BigQuery, Pub / Sub) using Apache Airflow and Google Cloud Composer.
Build secure, scalable, and optimized data architectures leveraging GCP services such as Cloud Storage, Pub / Sub, Dataproc, Dataflow, and BigQuery.
Design and implement automated frameworks for data delivery, monitoring, and troubleshooting.
Develop data observability frameworks to ensure quality, lineage, and reliability across pipelines.
Proactively monitor system performance, identify bottlenecks, and optimize pipelines for efficiency, scalability, and cost.
Troubleshoot and resolve complex technical issues in distributed systems and cloud environments.
Drive best practices in documentation of tools, architecture, processes, and solutions.
Mentor junior engineers, conduct design / code reviews, and influence engineering standards.
Collaborate with cross-functional teams to enable AI / ML and GenAI-driven use cases on LUMI.

Minimum Qualifications :

8+ years of experience in data engineering, software engineering, or platform development.

Strong programming expertise in Java, Python, and Shell scripting.

Advanced knowledge of SQL, data modeling, and performance optimization.

Deep expertise in Google Cloud Platform services : Cloud Storage, BigQuery, Pub / Sub, Dataproc, Dataflow.

Strong background in RDBMS (Oracle, Postgres, MySQL) and exposure to NoSQL DBs (Cassandra, MongoDB, or similar).

Proven track record in CI / CD pipelines, Git workflows, and Agile development.

Demonstrated experience in building and scaling production-grade data pipelines.

Strong problem-solving and troubleshooting skills in distributed and cloud-native systems.

Preferred Qualifications :

Hands-on experience with DevOps best practices, automation, and infrastructure as code.

Exposure to platform engineering (networking, security, IAM, firewalls).

Experience designing and implementing data observability frameworks (monitoring, lineage, anomaly detection).

Hands-on or exposure to GenAI integrations (LLMs, RAG, AI-driven data engineering workflows).

Proven ability to mentor, influence, and lead engineering discussions.

Roles & Responsibilities

Architect and implement enterprise-grade data migration solutions using Java and Python, enabling seamless data transfers from on-premises to GCP (Cloud Storage, BigQuery, Pub / Sub) using Apache Airflow and Google Cloud Composer.

Build secure, scalable, and optimized data architectures leveraging GCP services such as Cloud Storage, Pub / Sub, Dataproc, Dataflow, and BigQuery.

Design and implement automated frameworks for data delivery, monitoring, and troubleshooting.

Develop data observability frameworks to ensure quality, lineage, and reliability across pipelines.

Proactively monitor system performance, identify bottlenecks, and optimize pipelines for efficiency, scalability, and cost.

Troubleshoot and resolve complex technical issues in distributed systems and cloud environments.

Drive best practices in documentation of tools, architecture, processes, and solutions.

Mentor junior engineers, conduct design / code reviews, and influence engineering standards.

Collaborate with cross-functional teams to enable AI / ML and GenAI-driven use cases on LUMI.