Role and responsibilities
- Strong technical, analytical, and problem-solving skills
- Strong organizational skills, with the ability to work autonomously as well as in a team-based environment
- Data pipeline template development
- Data pipeline Monitoring development & support (operations)
Technical skills requirements
The candidate must demonstrate proficiency in,
On-premise servers for data processing and extractionAbility to own and deliver on large, multi-faceted projectsFluency in complex SQL and experience with RDBMSs(Bigdata, Spark, PySpark, Scala, Python, NiFi, Hive, NoSql DBs)Experience designing and building big data pipelinesExperience working on large scale, distributed systemsExperience working on Azure Cloud services like (Azure Data Factory, Databricks, ADLS, Azure HD Insights, Azure Synapse, Azure Data warehouse)Strong hands-on experience of programming language like PySpark, Scala with Spark, Python.Exposure to various ETL and Business Intelligence toolsSolid grounding in Agile methodologiesExperience with git and other source control systemsStrong communication and presentation skillsNice-to-have skills
Certification in Hadoop / Big Data – Hortonworks / ClouderaAzure Cloud certificationUnix or Shell scriptingStrong delivery background across the delivery of high-value, business-facing technical projects in major organizationsExperience of managing client delivery teams, ideally coming from a Data Engineering / Data Science environment