Talent.com
Lead Data Engineer - Python/SQL
Lead Data Engineer - Python/SQLTaylor and francis • Bangalore
Lead Data Engineer - Python / SQL

Lead Data Engineer - Python / SQL

Taylor and francis • Bangalore
24 days ago
Job description

Job Description :

Closing Date : Applications will close on 11th November 2025.

What you will be doing :

  • Function as part of an Agile engineering team in the role of Lead Data Engineer, responsible for orchestrating and upholding the highest levels of Engineering and Enterprise Data Warehouse Concepts and practices throughout the Software Development Life Cycle, collaborating closely with development, operations, and architecture teams.
  • Define delivery phases and perform project, task and individual management duties for team, setting activities, sub-activities, and milestones, while ensuring that development and delivery standards are rigorously enforced.
  • Manage and enhance our Continuous Integration, Delivery, and Deployment processes, teams, and users to ensure high-quality and efficient delivery.
  • Collaborate closely with Engineering Managers and Business Analysts to produce accurate delivery estimates and oversee the transition from analysis through design to deliver.
  • Design and implement scalable data solutions using AWS services.
  • Design and implement data models for warehouse solutions.
  • Create and maintain Infrastructure as Code using CloudFormation.
  • Implement comprehensive unit testing strategies.
  • Work closely with cross-functional teams to gather requirements.
  • Mentor junior engineers on AWS best practices and data engineering concepts.
  • Participate in code reviews and technical discussions.
  • Document technical solutions and maintain knowledge base.
  • Deploy and troubleshoot solutions as needed, adhering to best practices throughout the development lifecycle to identify and resolve issues prior to deployment.
  • Plan, design, and develop cloud-based applications, analytical services, and platforms.
  • Ensure adherence to company security guidelines and requirements.

What we are looking for :

  • An enthusiastic team member who fosters collaborative relationships, advocate for technical excellence, and adapt to evolving requirements within a dynamic team environment.
  • Demonstrate excellent communication, organizational skills, and a proactive approach to meeting deadlines and achieving project goals.
  • Proven experience as a Lead Data Engineer with expertise in Python and Spark.
  • Experience in CI / CD pipelines using GitHub and GitHub Actions to streamline development workflows.
  • Develop and maintain ETL pipelines using AWS Glue, Lambda, and EventBridge.
  • Manage data storage solutions across S3 and Redshift.
  • Implement security best practices using AWS IAM.
  • Experience with cloud computing services, especially AWS ECS, RDS, EFS, S3, Lambda, to build scalable and resilient applications.
  • Experience with database management including SQL (e.g., MySQL) and NoSQL (e.g., MongoDB), ensuring efficient data storage and retrieval.
  • Ensure code quality through SonarCloud integration.
  • Create and maintain Infrastructure as Code using CloudFormation.
  • In-depth knowledge in Test Driven Development (TDD) for Data Pipelines and Python.
  • Excellent container orchestration skills using Docker, ensuring efficient application deployment and management.
  • Strong understanding of Agile methodologies and implementations (Kanban, XP, Scrum).
  • Excellent problem-solving and troubleshooting skills.
  • Process-oriented with great documentation skills.
  • Bachelors degree in computer science or a related field, or equivalent work experience preferred.
  • What we offer in return :

  • 24 days annual leave.
  • 4 volunteering days annually.
  • Day off for your birthday.
  • Pension contributions.
  • Medical insurance for self and dependants; life cover and personal accident cover for self.
  • Seasonal social and charitable events.
  • Training and development.
  • Blended style and flexi working time.
  • Right tools for remote working.
  • Additional Information :

  • You must have the right to work and live in Bengaluru, India.
  • We believe that great things happen when people connect face-to-face.
  • That's why we work in-person with each other, or with customers and partners, three days a week or more.
  • When youre not spending time together in one of our offices or other workplaces like at conference you get the flexibility and support to work from home or remotely.
  • The successful candidate will be able to utilise our balanced working model.
  • We ask that candidates live within a commutable distance of our office in Bengaluru to ensure that as a balanced worker they can make it to our office location 3 days per week.
  • We want all our candidates to shine in our recruitment process.
  • Please let us know if there is anything we can do to ensure you are able to show us your best self.
  • This could include having the application form in a different format, more time for questions, or anything else please ask us, we are happy to be flexible! Please contact (ref : hirist.tech)
  • Create a job alert for this search

    Lead Data Engineer • Bangalore

    Related jobs
    Lead Data Engineer - SQL / Python

    Lead Data Engineer - SQL / Python

    Firstcareercentre • Bangalore
    Description : Job Summary : We are seeking a highly skilled Lead Data Engineer with strong expertise in SQL...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Platform Engineer (GCP)

    Lead Data Platform Engineer (GCP)

    Impetus • Bengaluru, Republic Of India, IN
    Lead Data Engineer – GCP (BigQuery - Composer - Python - PySpark).You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform.You will manage a te...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer - Python / Scala

    Lead Data Engineer - Python / Scala

    Syniverse Technologies Services India Pvt. Ltd. • Bangalore
    Description : The Lead Data Engineer is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems or build...Show more
    Last updated: 16 days ago • Promoted
    Lead Data Engineer - Python / SQL

    Lead Data Engineer - Python / SQL

    Marktine Technology Solutions Pvt Ltd • Bangalore
    Description : Relevant Experience : 7-10 years.Job description : - Lead the data engineering team by driving the design, build, test, a...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer - Python / Spark

    Lead Data Engineer - Python / Spark

    Success Pact Consulting Pvt Ltd • Bangalore
    Description : About the Role : We are looking for an experienced Lead Data Engineer to architect, build, and optimize...Show more
    Last updated: 15 days ago • Promoted
    Data Engineer - Python / SQL

    Data Engineer - Python / SQL

    Hashone Careers • Bangalore
    Description : We are seeking a skilled and motivated Data Engineer with 3 to 5 years of experience to join our growing team.The ideal candidate wi...Show more
    Last updated: 24 days ago • Promoted
    Lead Cloud Data Engineer

    Lead Cloud Data Engineer

    Tata Consultancy Services • Bengaluru, Republic Of India, IN
    Role- GCP Senior Data Engineer.Location- Hyderabad, Chennai, Bangalore, Kolkata, Mumbai.Data Integration, orchestration mechanism, ability to design BI solutions using Cloud Store, Big Query, Cloud...Show more
    Last updated: 19 days ago • Promoted
    Senior Data Engineer - Python / SQL / ETL

    Senior Data Engineer - Python / SQL / ETL

    TargetHR • Bangalore
    Description : - We are looking for an experienced Senior Data Engineer who can join us as part of the core team and who ...Show more
    Last updated: 15 days ago • Promoted
    Data Engineer - Python / SQL

    Data Engineer - Python / SQL

    Confidential • India, Bengaluru / Bangalore
    Data Pipeline Development : Design, build, and optimize scalable, secure, and reliable data pipelines to ingest, process, and transform large volumes of structured and unstructured data.Data Archit...Show more
    Last updated: 7 hours ago • Promoted • New!
    UsefulBI Corporation - Senior Data Engineer - Python / SQL

    UsefulBI Corporation - Senior Data Engineer - Python / SQL

    USEFULBI CORPORATION • Bangalore
    UsefulBI is looking for highly skilled candidates with expertise in generating power business insights from very large datasets, in this space where the primary aim would be to enable needle-moving...Show more
    Last updated: 15 days ago • Promoted
    Data Engineer - Python / SQL / Spark

    Data Engineer - Python / SQL / Spark

    SK HR Consultants • Bangalore
    Description : - Design, build and maintain scalable data pipelines for integrating diverse data sources.Implement and ma...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer - Python / SQL / ETL

    Lead Data Engineer - Python / SQL / ETL

    Scaling Theory Technologies Pvt Ltd • Bangalore
    Role Overview : We are seeking an experienced Lead Data Engineer to design, build, and optimize scalable data platforms within the Azure Databricks ecosystem.The ideal candida...Show more
    Last updated: 29 days ago • Promoted
    Senior Data & Analytics Engineer - Python / SQL

    Senior Data & Analytics Engineer - Python / SQL

    Blinkhealth • Bangalore
    What You'll Need : 5+ years hands-on experience with a combination of : - Python software development for data pipelines and automation.SQ...Show more
    Last updated: 30+ days ago • Promoted
    Lead GCP Data Engineer

    Lead GCP Data Engineer

    Impetus • Bengaluru, Karnataka, India
    Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark).You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform.You will manage a team of ...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer | Bangalore | Leading Big 4 Consulting Firm

    Data Engineer | Bangalore | Leading Big 4 Consulting Firm

    Acme Services • Bengaluru, Karnataka, India
    We are hiring sharp, hands-on Data Engineers to build scalable data solutions and drive performance across both traditional and cloud-based data platforms. Key Roles and Responsibilities.Strong comm...Show more
    Last updated: 7 days ago • Promoted
    Data Engineer - Python / SQL

    Data Engineer - Python / SQL

    Spigot Software • Bangalore
    Responsibilities : - Design, develop, and maintain scalable data pipelines.Collaborate with cross-functional teams to understand data requirements and deliver solutio...Show more
    Last updated: 22 days ago • Promoted
    Senior Data Engineer - Python / SQL / PySpark

    Senior Data Engineer - Python / SQL / PySpark

    AVIS BUDGET TECHNOLOGY INNOVATIONS PRIVATE LIMITED • Bangalore
    Description : Role : Senior Data Engineer Location : Bangalore Exp : 8-11 Years NP : ...Show more
    Last updated: 24 days ago • Promoted
    Data Engineer

    Data Engineer

    Ubique Systems • Bengaluru, India
    Primary skills : Python, SQL, data lakes, azure.Pipeline Development & Automation.Design, build, and maintain CI / CD pipelines to automate deployment of DQ rules and data services across environments...Show more
    Last updated: 30+ days ago • Promoted