Data Engineer - Apache Spark

About the Role :

We are looking for an experienced Data Engineer with strong hands-on expertise in building, optimizing, and maintaining large-scale data processing systems. The ideal candidate will have deep technical knowledge in Apache Spark, data lakehouse technologies, and streaming platforms, along with solid programming experience and exposure to modern orchestration and deployment tools.

Key Responsibilities :

• Design, build, and optimize scalable data pipelines for batch and streaming data using Apache Spark and Hive.
• Develop and maintain robust ETL / ELT frameworks for data ingestion, transformation, and integration across multiple data sources.
• Implement data lakehouse architectures using Iceberg, Delta Lake, or Apache Hudi to support advanced analytics and reporting.
• Work with streaming technologies like Kafka and Solace for real-time data processing and ingestion.
• Collaborate with cross-functional teams including Data Scientists, Analysts, and Application Developers to deliver reliable data solutions.
• Automate and orchestrate workflows using Airflow or Control-M and integrate SQL transformations using DBT (preferred).
• Ensure high performance, scalability, and reliability of data systems while adhering to data governance and quality standards.
• Deploy, monitor, and manage data applications within containerized environments using Docker and Kubernetes.
• Optimize data storage, access, and retrieval from object stores such as S3 or MinIO.
• Contribute to design reviews, code reviews, and continuous improvement of data engineering best practices.

Required Skills and Qualifications :

Preferred Qualifications :

(ref : hirist.tech)