Data Engineer with Pyspark

TCS Hiring !!!!

Role : Data Engineer with Pyspark

Exp : 4-7 Years

Location : Hyderabad / Chennai / Kolkata

JD

Technical / Functional Skills

• 5–6 years of total experience in data engineering or big data development
• 2–3 years hands-on experience with Apache Spark.
• Strong programming skills in PySpark, Python, and Scala.
• 2+ years of experience in Scala backend development.
• Proficient in Scala, both object oriented and functional programming concepts.
• Deep understanding and application of advanced functional programming concepts like category theory, monads, applicatives, and type classes
• Hands-On knowledge with Scala Typelevel libraries like Cats, Shapeless, and others used for building applications with strong typing and efficient concurrency.
• Solid understanding of data lakes, lakehouses, and Delta Lake concepts
• Experience in SQL development and performance tuning. Proficient in cloud services (e.g. S3, Glue, Lambda, EMR, Redshift, CloudWatch, IAM)
• Familiarity with Airflow, dbt, or similar orchestration tools is a plus.
• Experience in CI / CD tools like Jenkins, GitHub Actions, or Code Pipeline.
• Knowledge of data security, governance, and compliance frameworks.