Talent.com
Lead Data Engineer - SQL/Python
Lead Data Engineer - SQL/PythonFirstcareercentre • Bangalore
Lead Data Engineer - SQL / Python

Lead Data Engineer - SQL / Python

Firstcareercentre • Bangalore
30+ days ago
Job description

Description : Job Summary :

We are seeking a highly skilled Lead Data Engineer with strong expertise in SQL, Python, Big Data technologies, and Google BigQuery. The ideal candidate will design and build scalable data pipelines, lead data integration initiatives, and ensure optimal data quality and performance across large datasets. This role involves close collaboration with data analysts, scientists, and business teams to deliver reliable and high-performing data solutions.

Roles & Responsibilities :

  • Design, develop, and maintain scalable and reliable data pipelines for batch and real-time processing.
  • Lead and mentor a team of data engineers in best practices, performance tuning, and code quality.
  • Implement ETL / ELT workflows using Python, SQL, and modern orchestration tools (Airflow, Dataflow, etc.).
  • Optimize and manage large-scale datasets using BigQuery, Data Lake, or Data Warehouse architectures.
  • Collaborate with analytics, BI, and business teams to ensure data accuracy, consistency, and availability.
  • Work with Big Data tools (e.g., Hadoop, Spark, Kafka) for data transformation and aggregation.
  • Ensure proper data governance, lineage, and documentation of data flows.
  • Develop and maintain unit tests, monitoring, and data quality validation frameworks.
  • Integrate data from multiple sources (APIs, databases, cloud storage) into unified data systems.
  • Participate in architecture reviews and contribute to the overall data platform strategy.

Required Skills & Qualifications :

  • Bachelors or Masters degree in Computer Science, Information Technology, or related field.
  • 7+ years of experience in data engineering with at least 2+ years in a lead role.
  • Strong proficiency in SQL (complex queries, performance tuning, stored procedures).
  • Hands-on experience with Python for data processing and automation.
  • Expertise in Google BigQuery (partitioning, clustering, optimization, query design).
  • Strong experience with Big Data tools like Apache Spark, Hadoop, Hive, or Kafka.
  • Familiarity with ETL orchestration tools (Apache Airflow, Dataflow, Luigi, etc.).
  • Working knowledge of cloud platforms (GCP preferred; AWS or Azure is a plus).
  • Strong understanding of data modeling, warehousing, and schema design.
  • Experience with version control (Git) and CI / CD pipelines for data projects.
  • Preferred Skills (Good to Have) :

  • Experience with DBT or Alembic for data transformations and migrations.
  • Exposure to containerization tools (Docker, Kubernetes).
  • Knowledge of BI tools (Power BI, Looker, Tableau).
  • Familiarity with machine learning data preparation pipelines
  • (ref : hirist.tech)

    Create a job alert for this search

    Lead Data Engineer • Bangalore

    Related jobs
    Data Engineer

    Data Engineer

    NexionPro Services • Bangalore Urban, Karnataka, India
    PySpark-based data processing workflows.Collaborate with data architects, analysts, and cross-functional teams to understand business requirements and translate them into technical solutions.Optimi...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Platform Engineer (GCP)

    Lead Data Platform Engineer (GCP)

    Impetus • Bengaluru, Republic Of India, IN
    Lead Data Engineer – GCP (BigQuery - Composer - Python - PySpark).You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform.You will manage a te...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer - Python / SQL

    Lead Data Engineer - Python / SQL

    Marktine Technology Solutions Pvt Ltd • Bangalore
    Description : Relevant Experience : 7-10 years.Job description : - Lead the data engineering team by driving the design, build, test, a...Show more
    Last updated: 30+ days ago • Promoted
    AWS Lead Data engineer

    AWS Lead Data engineer

    Tata Consultancy Services • Bengaluru, India
    Job Summary : In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows ...Show more
    Last updated: 21 days ago • Promoted
    Data Engineer - Python / SQL

    Data Engineer - Python / SQL

    Hashone Careers • Bangalore
    Description : We are seeking a skilled and motivated Data Engineer with 3 to 5 years of experience to join our growing team.The ideal candidate wi...Show more
    Last updated: 25 days ago • Promoted
    Lead Data Engineer - Python / SQL / ETL

    Lead Data Engineer - Python / SQL / ETL

    Scaling Theory Technologies Pvt Ltd • Bangalore
    Role Overview : We are seeking an experienced Lead Data Engineer to design, build, and optimize scalable data platforms within the Azure Databricks ecosystem.The ideal candida...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer - SQL / Python

    Data Engineer - SQL / Python

    Ixly Technology • Bangalore
    Description : Must Have : Microsoft Fabrics, Python, Pyspark, SQL - Putting together large, intricate data...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data & Analytics Engineer - Python / SQL

    Senior Data & Analytics Engineer - Python / SQL

    Blinkhealth • Bangalore
    What You'll Need : 5+ years hands-on experience with a combination of : - Python software development for data pipelines and automation.SQ...Show more
    Last updated: 30+ days ago • Promoted
    Lead GCP Data Engineer

    Lead GCP Data Engineer

    Impetus • Bengaluru, Karnataka, India
    Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark).You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform.You will manage a team of ...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer | Bangalore | Leading Big 4 Consulting Firm

    Data Engineer | Bangalore | Leading Big 4 Consulting Firm

    Acme Services • Bengaluru, Karnataka, India
    We are hiring sharp, hands-on Data Engineers to build scalable data solutions and drive performance across both traditional and cloud-based data platforms. Key Roles and Responsibilities.Strong comm...Show more
    Last updated: 9 days ago • Promoted
    Lead Data Engineer - Python / SQL

    Lead Data Engineer - Python / SQL

    Taylor and francis • Bangalore
    Job Description : Closing Date : Applications will close on 11th November 2025.What you will be doing : ...Show more
    Last updated: 25 days ago • Promoted
    Senior Data Engineer - SQL / Python

    Senior Data Engineer - SQL / Python

    QUARKS TECHNOSOFT PRIVATE LIMITED • Bangalore
    Job Description : We are looking for a Senior Data Engineer with 6+ years of hands-on experience in designing and building scalable, cloud-...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer - SQL / Python

    Data Engineer - SQL / Python

    Acme Services • Bangalore
    Description : We are looking for sharp, hands-on Data Engineers with strong technical depth and a passion for...Show more
    Last updated: 3 days ago • Promoted
    Senior Data Engineer - Python / SQL / PySpark

    Senior Data Engineer - Python / SQL / PySpark

    AVIS BUDGET TECHNOLOGY INNOVATIONS PRIVATE LIMITED • Bangalore
    Description : Role : Senior Data Engineer Location : Bangalore Exp : 8-11 Years NP : ...Show more
    Last updated: 25 days ago • Promoted
    Senior Data Engineer - SQL / Python

    Senior Data Engineer - SQL / Python

    Elgebra • Bangalore
    We are looking for a highly skilled and experienced Senior Data Engineer to join our team on a contract basis.The ideal candidate will be a technical leader in developing, optimizing, and maintaini...Show more
    Last updated: 30+ days ago • Promoted
    Senior Lead Data Engineer

    Senior Lead Data Engineer

    Pi Square Technologies • Bangalore, Karnataka, India
    Apache airflow Airflow Python PySpark SQLServer SQL Scripting SQL DBT SnowSQL Snowflake Azure Databricks Databricks Azure DevOps DevOps Terraform Azure pipeline Docker Kubernete.Experience Range : 6...Show more
    Last updated: 14 days ago • Promoted
    Data Engineer

    Data Engineer

    Ubique Systems • Bengaluru, India
    Primary skills : Python, SQL, data lakes, azure.Pipeline Development & Automation.Design, build, and maintain CI / CD pipelines to automate deployment of DQ rules and data services across environments...Show more
    Last updated: 30+ days ago • Promoted
    Lead Azure Data Engineer

    Lead Azure Data Engineer

    IQVIA • Bengaluru, Karnataka, India
    Data Engineering, Python, Azure Databricks, Database, People management experience.Manages a team of professional-level employees. Develops and communicates plans and priorities to meet team perform...Show more
    Last updated: 16 days ago • Promoted