Senior Data Engineer

Greetings from TCS!

Job Title : Senior Data Engineer

Required Skillset : Python, Spark, Databricks, AWS (S3, Glue, AirFlow, Cloudwatch, Lambda)

Experience Range : 8+ years

Job Description

ROLE AND MISSION DESCRIPTION

Main tasks

Master Databricks tools (job creation, cluster, notebook) and be able to query efficiently with SQL

Maintain the platform in operational condition in production (analyze and correct incidents and defects)

Development of Python data ingestion and transformation jobs with Spark on large volumes of data

Provide a long-term vision, both operationally and in terms of data platform strategy

Support and promote best practices

Participate in technical and functional design workshops

Write and update technical documentation

Technical skills required in order of priority :

MUST

Python

Spark

Databricks

SQL

SHOULD

AWS (S3, Glue, AirFlow, Cloudwatch, Lambda, IAM)

COULD

Big DATA

WOULD

GIT

Methodologies : CI / CD avec Gitlab

JIRA / Confluence

Scrum

Thanks & Regards,

Ria Aarthi A.