Position Overview :
We are looking for a skilled and detail-oriented Big Data Engineer to design, develop, and maintain scalable data pipelines and architectures.
The role involves working with large datasets, integrating diverse data sources, and ensuring data availability for analytics, machine learning, and business intelligence.
The ideal candidate will have strong expertise in big data technologies and the ability to collaborate with cross-functional teams to deliver data-driven solutions.
Key Responsibilities :
- Design, build, and optimize large-scale data pipelines for ingestion, transformation, and storage.
- Work with structured and unstructured data across multiple platforms and sources.
- Implement and maintain data lake and data warehouse solutions.
- Develop scalable ETL (Extract, Transform, Load) processes.
- Ensure data quality, security, and governance across systems.
- Collaborate with data scientists, analysts, and business teams to provide reliable data solutions.
- Monitor and optimize the performance of big data clusters and processing jobs.
- Research and integrate emerging big data tools and technologies.
- Troubleshoot and resolve issues related to data processing and storage.
Key Skills & Competencies :
Strong knowledge of big data frameworks (Hadoop, Spark, Flink, Kafka).Hands-on experience with cloud platforms (AWS, Azure, GCP).Proficiency in SQL, NoSQL databases (MongoDB, Cassandra, HBase).Strong programming skills in Python, Java, or Scala.Knowledge of ETL tools and data integration techniques.Familiarity with data warehousing solutions (Snowflake, Redshift, BigQuery).Problem-solving and analytical skills with attention to detail.Ability to work in cross-functional teams and manage multiple projects.Qualifications :
Bachelors or Masters degree in Computer Science, Data Engineering, Information Technology, or a related field.3 to 6 years of experience in data engineering or big data roles.Proven track record in handling large, complex datasets.Certifications in big data or cloud platforms (preferred but not mandatory).(ref : hirist.tech)