Roles and Responsibilities :
- Develop, Monitor, and Maintain data pipeline.
- Create and maintain optimal data pipeline architecture
- Assemble large, complex data sets that meet functional / non-functional business requirements.
- Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
- Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.
- Incident diagnosis, routing, evaluation & resolution
- Analyze the root cause of incidents.
- Create an incident closure report
Requirements :
BE Degree in computer science or equivalent from business schools, IITins, NIIT, IIM'sMin 3 years of Experience in Data managementExperience with data modeling, data warehousing, and building ETL pipelinesHands-on experience on Spark SQL and Spark streamingHands-on experience with Airflow or LuigiComfortable working with python and shell scriptsGood understanding of Hadoop EcosystemExperience with Data warehouses like RedShift and Databases like Postgres, MariaDB.Ability to implement webhooks if required.Immediate joining, need candidate within 15 or 30 days.
(ref : hirist.tech)