Grid Dynamics - Big Data Developer - Python / Scala

Job Title : Big Data : 7 to 10 : Hyderabad / Period : Immediate to 20 Days

About the Role :

We are looking for a highly skilled and experienced Big Data Developer to join our data engineering team. The ideal candidate will have a strong background in big data technologies, with hands-on experience in building scalable data pipelines and infrastructure. If you're passionate about working on large-scale distributed systems and cutting-edge open-source technologies, wed love to connect with you.

Key Responsibilities :

• Design, develop, and maintain scalable data ingestion, transformation, and enrichment pipelines.
• Work with Apache Spark (batch and streaming) for processing large volumes of data efficiently.
• Utilize Scala and Python to implement robust data engineering solutions.
• Manage and optimize HDFS and lead migration strategies across storage layers.
• Implement and manage table formats like Apache Iceberg, Delta Lake, or Apache Hudi for schema evolution, ACID compliance, and time travel features.
• Run Spark / Flink workloads on Kubernetes using tools like Spark-on-K8s operator or Flink-on-K8s.
• Leverage distributed object storage systems such as Ceph or AWS S3.
• Use Infrastructure-as-Code (Terraform, Helm) to provision and manage data infrastructure.

Required Skills & Experience :

Preferred Qualifications :

(ref : hirist.tech)