Data Engineer - Senior

Although the role category specified in the GPP is Remote, the requirement is for Hybrid.

Key Responsibilities :

• Design and Automation : Deploy distributed systems for ingesting and transforming data from various sources (relational, event-based, unstructured).
• Data Quality and Integrity : Implement frameworks to monitor and troubleshoot data quality and integrity issues.
• Data Governance : Establish processes for managing metadata, access, and retention for internal and external users.
• Data Pipelines : Build reliable, efficient, scalable, and quality data pipelines with monitoring and alert mechanisms using ETL / ELT tools or scripting languages.
• Database Structure : Design and implement physical data models to optimize database performance through efficient indexing and table relationships.
• Optimization and Troubleshooting : Optimize, test, and troubleshoot data pipelines.
• Large Scale Solutions : Develop and operate large-scale data storage and processing solutions using distributed and cloud-based platforms (e.g., Data Lakes, Hadoop, Hbase, Cassandra, MongoDB, Accumulo, DynamoDB).
• Automation : Use modern tools and techniques to automate common, repeatable, and tedious data preparation and integration tasks.
• Infrastructure Renovation : Renovate data management infrastructure to drive automation in data integration and management.
• Agile Development : Ensure the success of critical analytics initiatives using agile development technologies such as DevOps, Scrum, Kanban.
• Team Development : Coach and develop less experienced team members.

External Qualifications and Competencies

Qualifications :

Competencies :

Additional Responsibilities Unique to this Position

Skills :

Preferred Experience :

Role :   Data Engineer

Industry Type :   Industrial Equipment / Machinery

Department :   Engineering - Software & QA

Employment Type :   Full Time, Permanent

Role Category :   Software Development

Education

UG :   B.Tech / B.E. in Any Specialization

PG :   Any Postgraduate

Skills Required

Pyspark, snowflake , Databricks, Python, Sql