Data Engineer - Big Data Technologies

About the Role :

We are seeking a highly skilled Data Engineer to design, build, and maintain scalable, high-performance data pipelines and systems that support advanced analytics and business intelligence.

You will work with cutting-edge big data technologies and frameworks to ingest, process, transform, and manage massive datasets.

The ideal candidate thrives in a fast-paced environment, is passionate about data innovation, and excels at bridging academic research with production-scale business solutions.

Key Responsibilities :

• Design, develop, and maintain robust, scalable big data pipelines using tools such as Apache Spark, Airflow, Bodo, Flume, Flink, and others for ingesting and processing terabytes of data efficiently.
• Architect and implement data warehouses and data marts utilizing technologies like Presto, Snowflake, Hadoop, and other cloud or on-premise data platforms.
• Build optimized data models and schemas to enable fast and reliable query performance.
• Develop and manage strategies for data ingestion, cleansing, transformation, and aggregation from heterogeneous sources including structured, semi-structured, and unstructured data.
• Understand and apply graph query languages such as GQL, Gremlin, Cypher for modeling and querying complex relationships.
• Build scalable graph-based system architectures to support business use cases involving relationship and network analysis.
• Apply graph data technologies for business impact, including data management, infrastructure optimization, budgeting, trade-offs, and workflow / project management.
• Write efficient and maintainable code in Python, Scala, or Rust, especially for supercomputing environments handling large-scale datasets (TB+).
• Develop quantitative analytics and business operation dashboards using tools like Tableau, Apache Superset, or similar visualization platforms.
• Stay abreast of developments in natural language processing (NLP) and large language models (LLMs) and incorporate these capabilities where relevant.
• Evaluate cutting-edge academic methods and prototype their application in production systems.
• Use advanced problem-solving skills combined with deep understanding of statistics, probability, algorithms, and mathematics to deliver innovative data solutions.
• Collaborate effectively with data scientists, software engineers, product managers, and business stakeholders to deliver high-impact projects.
• Operate efficiently in a dynamic, fast-moving development environment with changing priorities, tight deadlines, and limited resources.
• Demonstrate strong self-motivation and ability to work independently as well as within cross-functional teams.

Required Qualifications & Skills :

(ref : hirist.tech)