Senior Data Engineer(Lead)

Responsibilities

• Around 6+ years of experience in data engineering or cloud computing data development
• Design, develop, and optimize ETL / ELT data pipelines using Apache Spark (PySpark or Scala), AWS Glue, and Azure Data Factory
• Work with structured and unstructured data to build scalable ingestion and transformation workflows across cloud platforms.
• Build data lake and data warehouse solutions using AWS S3, Azure Data Lake,AWS Glue
• Collaborate with data scientists, analysts, and application developers to support advanced analytics, reporting, and ML workflows
• Implement job orchestration, monitoring, and error handling for reliable pipeline execution.
• Maintenance and support of Data Pipeline.
• Data load monitoring
• Data Validation and quality checks
• Identify and optimize ingestion pipelines in consultation with Customer
• Data : L2 / L3 Support
• Investigate Glue job failures and restart
• Fix minor transformation logic or input data issues
• Resolve dependency failures in Glue workflows
• Tune job configurations
• Impact Analysis and RCA
• 8x5 support model

Qualifications

Education details

BTECH, BCA, MTECH,MCA,BSC,MSC

Required Skills

AWS Glue, ADF,ADB, Immediate Joiner

Show more

Show less

Skills Required

Azure Data Factory, Scala, Pyspark, AWS Glue, Apache Spark, Azure Data Lake, Aws S3