Talent.com
Senior / Lead Data Engineer - AWS / Python

Senior / Lead Data Engineer - AWS / Python

Velodata Global Pvt LtdKerala
25 days ago
Job description

Job Title : Lead Data Engineer

Location : Kochi / Trivandrum

Experience : 10+ years

Notice Period : Immediate to 15 days

Primary Skills : AWS, Python

Job Overview :

Seeking an experienced Senior / Lead Data Engineer to design and implement scalable data ingestion frameworks, ensure data quality and validation, and build robust APIs for seamless data integration. The ideal candidate will have hands-on expertise with AWS-based big data pipelines, automation, and high-performance API development.

Key Responsibilities :

Data Ingestion Framework :

  • Architect and maintain ETL pipelines using AWS Glue, Lambda, EMR, ECS, EC2, Step Functions.
  • Optimize data ingestion for scalability, resiliency, and automation.

Data Quality & Validation :

  • Build automated validation and error-handling mechanisms.
  • Implement monitoring, logging, and alerting for data quality assurance.

    • API Development :

  • Design and develop secure RESTful APIs for internal / external data integration.
  • Maintain API documentation and ensure adherence to best practices.

    • Collaboration & Agile :

  • Partner with business stakeholders, data scientists, and operations.
  • Contribute to CI / CD pipelines (preferably GitLab) and Agile processes.

    • Required Qualifications :

  • Experience : Minimum 5+ years in data engineering / analytical platform development.
  • Programming : Strong in Python / PySpark / SQL for ETL and large-scale data handling.
  • AWS Expertise : Hands-on with AWS Glue, Lambda, Step Functions, S3, and data platforms (EMR, Apache Spark, Iceberg).
  • Database Exposure : DynamoDB, Aurora, Postgres, Redshift.
  • API Development : Proven REST API design & integration skills.

    • Preferred Qualifications :

  • Bachelors / Masters in Computer Science, Data Engineering, or related field.
  • Familiarity with AWS Kinesis, Firehose, SQS, and data lakehouse architectures.
  • Experience with data quality frameworks and API-driven integrations in multi-cluster setups.
  • Commitment to Information Security policies and best practices.

    • (ref : hirist.tech)

    Create a job alert for this search

    Data Engineer • Kerala