Senior Data Engineer- (Python & AWS (Simplified JD)
Experience : 7–16 years
Location : Hyderabad
Job Summary
We are looking for a Senior Data Engineer who can build, maintain, and optimize large-scale data systems using AWS and Python . You will design data pipelines, ensure data quality, automate workflows, and support analytics / AI teams. The role needs strong technical expertise and leadership.
Key Responsibilities (Simplified)
- Build and manage scalable data pipelines using AWS Glue, Lambda, EMR, Step Functions, Redshift.
- Create and optimize data lakes and data warehouses on AWS (S3, Redshift, Athena).
- Develop ETL / ELT processes using Python (pandas, PySpark).
- Integrate data from various sources like RDBMS, APIs, Kafka / Kinesis.
- Design efficient data models and schemas.
- Ensure data quality, monitoring, and lineage using AWS tools.
- Implement data governance, security, and compliance (IAM, encryption).
- Work with Data Science, Analytics, Product, and DevOps teams.
- Set up CI / CD for data pipelines.
- Mentor junior engineers and review their code.
- Monitor system performance and handle troubleshooting.
Required Skills
Bachelor’s / Master’s degree in Computer Science or related field.5–10 years of experience in data engineering.Strong Python skills (pandas, PySpark, boto3, SQLAlchemy).Expertise with AWS services : Glue, Lambda, EMR, Step Functions, Redshift, Athena, S3, Kinesis, DynamoDB.Strong SQL and data modeling skills.Experience building data lakes, warehouses, and streaming solutions.Knowledge of ETL best practices.Experience with Git and CI / CD pipelines.Understanding of Docker / Kubernetes and DevOps concepts.Good communication and problem-solving skills.Nice-to-Have Skills
Spark / PySpark on EMR or Glue.Airflow, dbt, or Dagster.Kafka / Kinesis streaming.AWS Lake Formation, Glue Studio, DataBrew.SageMaker or QuickSight integration.AWS Certifications (optional)