Talent.com
Continental - IT Engineer - Data Lakehouse

Continental - IT Engineer - Data Lakehouse

Contitech India Private LimitedBangalore
21 days ago
Job description

Description :

Job Role / Title : IT engineer data lakehouse

Experience Range : 3-6 years

Location : Bangalore

Job Description :

  • Design, develop, and operate scalable and maintainable data pipelines in the Azure Databricks environment
  • Develop all technical artefacts as code, implemented in professional IDEs, with full version control and CI / CD automation
  • Enable data-driven decision-making in Human Resources (HR), Purchasing (PUR) and Finance (FIN) by ensuring high data availability, quality, and reliability
  • Implement data products and analytical assets using software engineering principles in close alignment with business domains and functional IT
  • Apply rigorous software engineering practices such as modular design, test-driven development, and artifact reuse in all implementations
  • Global delivery footprint; cross-functional data engineering support across HR, PUR & FIN domains
  • Collaboration with business stakeholders, functional IT partners, product owners, architects, ML / AI engineers, and Power BI developers
  • Agile, product-team structure embedded in an enterprise-scale Azure environment

Main Tasks :

  • Design scalable batch and streaming pipelines in Azure Databricks using PySpark and / or Scala
  • Implement ingestion from structured and semi-structured sources (e.g., SAP, APIs, flat files)
  • Build bronze / silver / gold data layers following the defined lakehouse layering architecture & governance
  • Implement use-case driven dimensional models (star / snowflake schema) tailored to HR, PUR & FIN needs
  • Ensure compatibility with reporting tools (e.g., Power BI) via curated data marts and semantic models
  • Implement enterprise-level data warehouse models (domain-driven 3NF models) for HR, PUR & FIN data, closely aligned with data engineers for other business domains
  • Develop and apply master data management strategies (e.g., Slowly Changing Dimensions)
  • Develop automated data validation tests using frameworks
  • Monitor pipeline health, identify anomalies, and implement quality thresholds
  • Establish data quality transparency by defining and implementing meaningful data quality rules with source system and business stakeholders and implementing related reports
  • Develop and structure pipelines using modular, reusable code in a professional IDE
  • Apply test-driven development (TDD) principles with automated unit, integration, and validation tests
  • Integrate tests into CI / CD pipelines to enable fail-fast deployment strategies
  • Commit all artifacts to version control with peer review and CI / CD integration
  • Work closely with Product Owners to refine user stories and define acceptance criteria
  • Translate business requirements into data contracts and technical specifications
  • Participate in agile events such as sprint planning, reviews, and retrospectives
  • Document pipeline logic, data contracts, and technical decisions in markdown or auto-generated docs from code
  • Align designs with governance and metadata standards (e.g., Unity Catalog)
  • Track lineage and audit trails through integrated tooling
  • Profile and tune data transformation performance
  • Reduce job execution times and optimize cluster resource usage
  • Refactor legacy pipelines or inefficient transformations to improve scalability

    • Qualifications :

  • Degree in Computer Science, Data Engineering, Information Systems, or related discipline. Certifications in software development and data engineering (e.g., Databricks DE Associate, Azure Data Engineer, or relevant DevOps certifications).
  • 3 to 6 years of hands-on experience in data engineering roles in enterprise environments. Demonstrated experience building production-grade codebases in IDEs, with test coverage and version control.
  • Proven experience in implementing complex data pipelines and contributing to full lifecycle data projects (development to deployment)
  • Experience in at least one business domain : HR, PUR & FIN or a comparable field not required; however, experience mentoring junior developers or leading implementation workstreams is a plus
  • Experience working in international teams across multiple time zones and cultures, preferably with teams in India, Germany, and the Philippines.

    • (ref : hirist.tech)

    Create a job alert for this search

    Data Engineer • Bangalore