Data Engineer - ETL / ELT Pipelines

Job Summary :

The Data Engineer will design, develop, and maintain scalable data pipelines and infrastructure to support data processing, storage, and analytics initiatives. The candidate will work closely with data scientists and business teams to ensure high-quality, reliable data flows, supporting AI and analytics projects by building robust ETL / ELT processes, managing batch and streaming data ingestion, and ensuring data governance and security compliance.

Key Responsibilities :

• Design, build, and maintain efficient, scalable ETL / ELT pipelines for large-scale data ingestion and transformation.
• Manage batch and stream data processing workflows, ensuring high data quality, integrity, and availability.
• Collaborate with data scientists and analysts to understand data requirements and deliver optimized data sets and structures.
• Maintain comprehensive data documentation, lineage, and schemas to facilitate transparency and reproducibility.
• Implement data governance, schema validation, and security measures for sensitive data.
• Automate data operations and monitoring to ensure pipeline reliability and performance.
• Use tools like SQL, NoSQL, Spark, Kafka, Airflow, and cloud platforms (AWS, Azure, GCP) to support data infrastructure.
• Stay updated on emerging data engineering best practices and technologies to optimize data architecture.

Skills & Qualifications :

(ref : hirist.tech)