Data Engineer

About the Role

Position : Data Engineer-I

Experience : 2-4 Years

Notice Period : Immediate-60 days

Hiring Locations : Bengaluru, Kolkata, Hyderabad, Chennai, Gurgaon

Design, build, and optimize scalable data pipelines and ETL / ELT workflows using Spark (Scala / Python), SQL, and orchestration tools (e.g., Apache Airflow, Prefect, Luigi).

Implement efficient solutions for high-volume, batch, real-time streaming, and event-driven data processing, leveraging best-in-class patterns and frameworks.

Build and maintain data warehouse and lake house architectures (e.g., Snowflake, Databricks, Delta Lake, BigQuery, Redshift) to support analytics, data science, and BI workloads.

Develop, automate, and monitor Airflow DAGs / jobs on cloud or Kubernetes, following robust deployment and operational practices (CI / CD, containerization, infra-as-code).

Write performant, production-grade SQL for complex data aggregation, transformation, and analytics tasks.

Ensure data quality, consistency, and governance across the stack, implementing processes for validation, cleansing, anomaly detection, and reconciliation.

Collaborate with Data Scientists, Analysts, and DevOps engineers to ingest, structure, and expose structured, semi-structured, and unstructured data for diverse use-cases.

Contribute to data modeling, schema design, data partitioning strategies, and ensure adherence to best practices for performance and cost optimization.

Implement, document, and extend data lineage, cataloging, and observability through tools such as AWS Glue, Azure Purview, Amundsen, or open-source technologies.

Apply and enforce data security, privacy, and compliance requirements (e.g., access control, data masking, retention policies, GDPR / CCPA).

Take ownership of end-to-end data pipeline lifecycle : design, development, code reviews, testing, deployment, operational monitoring, and maintenance / troubleshooting.

Contribute to frameworks, reusable modules, and automation to improve development efficiency and maintainability of the codebase.

Stay abreast of industry trends and emerging technologies, participating in code reviews, technical discussions, and peer mentoring as needed.