Principal Data Engineer

Job Description :

Architect and build the data platform by translating business and analytical requirements into technical designs and data models that align with our long term platform vision. Lead major platform initiatives such as unified master data management, lakehouse style architectures, data mesh patterns or centralized experimentation frameworks. Ensure solutions are secure, compliant with regional regulations and optimized for performance and cost.

Develop data products and pipelines towards building near real time data products and pipelines using best of breed tools. Implement self service paved roads that allow teams to rapidly create data products using approved components. Leverage modern ELT / ETL frameworks, streaming technologies and orchestration engines to deliver reliable and scalable pipelines from transactional databases (e.g., PostgreSQL) to analytics platforms (e.g., Snowflake).

Enable data governance and quality by owning the architecture for the Data Catalog, Data Lineage and Data Quality frameworks. Define data contracts and schemas using registry based systems and integrate observability and alerting to detect anomalies proactively. Collaborate with Platform and DevOps teams to integrate cost and performance regression testing into the CI / CD process.

Ensure operational excellence by clearly defining the disaster recovery strategy and implement business continuity practices with well defined RTO / RPO targets. Create governance and observability frameworks for cost, capacity and performance management, leveraging AI

powered tools. Anticipate scaling challenges and design patterns that enable the platform to grow an order of magnitude without compromising reliability.

Drive cross functional collaboration and communication by partnering with Solutions Architects, Product Analysts, Application Engineering teams and senior stakeholders to align the data platform with product roadmaps and customer needs. Lead cross team forums to review architecture, resolve conflicts, and drive adoption of standards. Publish internal whitepapers, run knowledge sharing sessions and evangelize the benefits of the data platform.

Mentor and develop others by providing technical leadership and coaching to Senior SDEs and other engineers. Shape best practices in data engineering, data modeling, quality assurance and operational excellence. Encourage a culture of learning, adaptability and cross organizational Experience :

1.Deep expertise in data architecture, data modeling and modern data warehouse concepts, including master data management, dimensional modeling and self service data mesh patterns. Experience designing unified data platforms on cloud data warehouses such as Snowflake, Redshift, Azure Synapse or similar.

2.Demonstrated ability to architect and implement data governance frameworks, including data catalogs, data lineage, role based access control and automated PII classification. Experience solving regulatory challenges such as data residency and compliance with privacy laws.

3.Proficiency with ETL / ELT tools (e.g., Informatica, Talend, dbt, Fivetran), streaming and messaging technologies (e.g., Kafka or equivalent), orchestration engines (e.g., Airflow, Prefect) and data quality / observability platforms. Experience building cost governance frameworks and integrating AI powered observability is highly valued.

4. Strong programming and database skills (SQL, Python, Scala or similar) and familiarity with cloud infrastructure (AWS, Azure or GCP) and DevOps practices (CI / CD, infrastructure as code, containerization). Capability to create scalable, reliable and secure data pipelines and services.

5.Experience mentoring : Senior engineers and leading cross team architectural initiatives. Demonstrated ability to influence senior stakeholders through clear communication, whitepapers, presentations and architecture reviews.

(ref : hirist.tech)