iKrux - Big Data Engineer - Python / Scala

We are seeking a highly skilled and motivated Big Data Engineer with strong experience in building, optimizing, and managing large-scale data processing pipelines. The ideal candidate will have hands-on expertise in Scala, Python, Apache Spark, Kafka, and / or Apache NiFi, and will work closely with cross-functional teams to design and implement scalable data solutions that drive business intelligence and analytics.

Key Responsibilities :

• Design, build, and maintain large-scale distributed data processing systems using Spark, Kafka, and / or NiFi.
• Write efficient and reusable code in Scala and Python for data transformation, ingestion, and enrichment.
• Collaborate with data scientists, analysts, and engineers to ensure data quality, consistency, and availability.
• Optimize data pipelines for performance, reliability, and scalability.
• Integrate diverse data sources, ensuring data governance and security compliance.
• Monitor and troubleshoot data workflows, resolving bottlenecks and issues.
• Participate in code reviews, architecture discussions, and best practice development.
• Stay updated on emerging big data technologies and frameworks to continuously improve system efficiency.

Required Skills and Experience :

(ref : hirist.tech)