Data Engineer - Python/ETL

Job Design, build, and optimize ETL/ELT pipelines for batch and streaming data.

- Develop and maintain data warehouses and data lakes (e.g., Snowflake, BigQuery, Redshift).

- Collaborate with data scientists, analysts, and product teams to understand data needs.

- Ensure data quality, reliability, and observability across pipelines.

- Build and manage orchestration workflows (e.g., Airflow, Prefect, Dagster).

- Implement best practices for data modeling, partitioning, and performance tuning.

- Monitor and troubleshoot data infrastructure issues in 5+ years of experience in data engineering or a related role.

- Proficiency in SQL and Python (or Scala/Java).

- Experience with cloud platforms (AWS, GCP, or Azure).

- Hands-on with big data tools: Spark, Kafka, dbt, or similar.

- Familiarity with data warehousing concepts and dimensional modeling.

- Strong understanding of CI/CD and version control (Git).

Nice to Have:

- Experience with real-time/streaming pipelines (Kafka, Flink, Kinesis).

- Exposure to ML pipelines or MLOps tooling.

- Knowledge of data governance and data catalog tools (e.g., Atlan, Alation).

(ref:hirist.tech)