Data Engineer

About the Role

We are looking for a Data Engineer with strong experience in Spark (PySpark) , SQL , and data pipeline architecture . You will play a critical role in designing, building, and optimizing data workflows that enable scalable analytics and real-time insights. The ideal candidate is hands-on, detail-oriented, and passionate about crafting reliable data solutions, while collaborating with cross-functional teams.

Responsibilities

Design and architect scalable and efficient data pipelines for batch and real-time processing.

Develop and optimize solutions using Spark (PySpark) and SQL .

Ensure data pipelines are reliable, maintainable, and well-tested .

Work with stakeholders to understand business requirements and translate them into data-driven solutions .

Collaborate with cross-functional teams to ensure data quality, availability, and performance .

Stay updated with emerging data engineering tools and practices .

Must-Have Skills

Strong expertise in Spark (PySpark) .

Proficiency in SQL (query optimization, performance tuning, complex joins).

Hands-on experience in designing and architecting data pipelines .

Excellent communication and collaboration skills .

Good to Have

Experience with data streaming platforms (Kafka, Kinesis, etc.).

Proficiency in Python for data engineering tasks.

Exposure to Databricks and Azure cloud services.

Knowledge of HTAP systems, Debezium, 3PL / logistics domain (having most of this mentioned skills would be great)

Familiarity with orchestration frameworks such as Apache Airflow or Apache NiFi .