Sr. Data Engineer

ADP is hiring Sr. Data Engineer (Sr. Member Technical)

Required Skills

Bachelor’s / Engineering degree in Computer Science or Information Technology.

Postgraduate degree or diploma in Computer Science / Data Science is an added advantage.

AWS, SQL, Databricks, and Apache Spark certifications are a plus.

Strong foundation in data analytics and engineering, with a minimum of 5–7 years of hands-on experience in project management, implementation, development, improvement, and support of data-related projects.

Strong expertise in programming languages such as Python and SQL .

Deep knowledge of big data processing and analytics (Spark, Spark optimization, Databricks, Big Data technologies).

Proficient in data modeling.

Good expertise in development and operations (DevOps, GIT).

Knowledge of data storage and architecture is preferred.

Experience with minimum 4–5 relevant project closures.

Roles & Responsibilities

Demonstrates strong capabilities in Python and SQL, designing functional Python scripts for data manipulation or ETL tasks; uses libraries like pandas, NumPy effectively.

Designs and optimizes complex SQL-based data models, and mentor’s others in query optimization and database design principles.

Independently builds and optimizes Spark jobs for complex ETL tasks and manages Databricks workflows for performance and reliability.

Designs scalable Spark workflows and mentors’ others in advanced optimization techniques (e.g., skew handling, memory tuning).

Develops and optimizes Databricks workflows; manages clusters and schedules jobs effectively.

Creates robust data models for analytics (e.g., star schemas); ensures scalability and performance.

Proficient in DevOps practices, utilizing CI / CD pipelines and GIT for deployment while maintaining high standards of code quality and reliability.

Innovates analytics methodologies for complex problems, mentors’ others in analytics best practices.

Demonstrates strong problem - solving and project ownership , independently resolving complex technical issues and driving projects from start to finish.

Effectively communicates complex ideas to both technical and non-technical stakeholders, ensuring clear understanding and delivering robust, scalable analytics solutions.

Develops robust analytics approaches ; ensures solutions are scalable and aligned with business objectives.

Optimizes pipeline performance with supervision and implements basic improvements (e.g., indexing, caching).

Effectively Participates in stakeholder meetings, contributes to discussions on project requirements and updates, and gathers basic requirements, while needing some supervision for advanced conflict resolution and establishing complex stakeholder relationships.

Actively contributes to Agile processes, project planning, and supports the development team through knowledge sharing and basic technical guidance.