Talent.com
Lead Data Engineer - Python/SQL
Lead Data Engineer - Python/SQLMarktine Technology Solutions Pvt Ltd • Bangalore
Lead Data Engineer - Python / SQL

Lead Data Engineer - Python / SQL

Marktine Technology Solutions Pvt Ltd • Bangalore
30+ days ago
Job description

Description :

Relevant Experience : 7-10 years.

Job description :

  • Lead the data engineering team by driving the design, build, test, and launch of new data pipelines and models on the production data platform.
  • Define and implement the processes needed to achieve operational excellence in all areas, including agile development, and data solutions.
  • Oversee the design, development and maintenance of data infrastructure, including data warehouses, data lakes, data integration components supporting optimal extraction, transformation, and loading of data.
  • Leading the communication with vendors, clients, the leadership team, and other stakeholders to facilitate effective project management and provide ongoing support.
  • Collaborate with business owners for roadmap planning and prioritization, to deliver robust cloud-based data solutions for our customers.
  • Work closely with cross-functional teams to propel and execute necessary solution enhancements and provide support for existing solutions.
  • Define and enforce best practices for data engineering, data modeling, and data quality to ensure accuracy, reliability, reusability, security and consistency of data.
  • Evaluate and implement new technologies, tools, and frameworks to improve the performance and efficiency of the data engineering team.
  • Lead the planning, prioritization, and execution and tracking of data engineering projects, ensuring timely delivery and alignment with business objectives.
  • Establish and monitor key performance indicators (KPIs) to measure the performance and effectiveness of the data engineering team.
  • Stay current with industry trends and advancements in data engineering, Azure Cloud, and provide strategic guidance for adopting new technologies or :
  • 8-12 years of demonstrable experience in data engineering, analytics, data warehousing, data management, Data governance and Compliance Requirements.
  • Experience managing data engineers and guiding a team of engineers through project planning, execution, tracking and monitoring, and quality control stages.
  • Solid experience with cloud-based data tools and platforms.
  • Proficient in implementing efficient cost management strategies, particularly about storage and computational expenses.
  • Experience building processes supporting data transformation, data structures, metadata, security, governance, and workload management.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Expertise in designing and optimizing data models, RDBMS, NoSQL DBs, and data warehousing solutions.
  • Excellent leadership, communication, and interpersonal skills, with the ability to effectively collaborate with cross-functional teams.
  • Proven ability to drive innovation, make strategic decisions and lead complex data engineering initiatives to successful completion.

Skill Required : Must Haves :

  • SQL, Python, Py Spark, Spark SQL, Spark, Distributed Systems.
  • Databricks, ADLS Gen 2 Blob Storage, Azure DevOps, Azure Data Factory.
  • ETL, Building Data Pipelines, Data Warehousing, Datamart, Data Modelling and Governance.
  • Agile Practices, SDLC, DevOps practices, and CI / CD pipelines for data engineering workflows.
  • Solid understanding of the Microsoft Azure stack for large-scale data engineering developments and deployments is highly preferred.
  • Hands-on experience with Azure Databricks, including data ingestion, data transformation, workflow management and optimization, monitoring and troubleshooting Spark Jobs.
  • Ability to set up and manage Azure Data Lake Storage (ADLS) Gen 2 accounts, and familiarity with data lake architecture and best practices.
  • Familiarity with big data frameworks (e.g., Apache Spark).
  • Knowledge of Azure Key Vaults for securely storing and managing cryptographic keys, secrets, and certificates.
  • Good to Have :

  • Event Hubs for log management.
  • Workflow Orchestration.
  • Cosmos DB.
  • Power BI.
  • Professional Services Background.
  • Scala, Java.
  • (ref : hirist.tech)

    Create a job alert for this search

    Lead Data Engineer • Bangalore

    Related jobs
    Lead Data Engineer - SQL / Python

    Lead Data Engineer - SQL / Python

    Firstcareercentre • Bangalore
    Description : Job Summary : We are seeking a highly skilled Lead Data Engineer with strong expertise in SQL...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Platform Engineer (GCP)

    Lead Data Platform Engineer (GCP)

    Impetus • Bengaluru, Republic Of India, IN
    Lead Data Engineer – GCP (BigQuery - Composer - Python - PySpark).You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform.You will manage a te...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer - Python / Scala

    Lead Data Engineer - Python / Scala

    Syniverse Technologies Services India Pvt. Ltd. • Bangalore
    Description : The Lead Data Engineer is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems or build...Show more
    Last updated: 16 days ago • Promoted
    Lead Data Engineer - Python / Spark

    Lead Data Engineer - Python / Spark

    Success Pact Consulting Pvt Ltd • Bangalore
    Description : About the Role : We are looking for an experienced Lead Data Engineer to architect, build, and optimize...Show more
    Last updated: 15 days ago • Promoted
    Data Engineer - Python / SQL

    Data Engineer - Python / SQL

    Hashone Careers • Bangalore
    Description : We are seeking a skilled and motivated Data Engineer with 3 to 5 years of experience to join our growing team.The ideal candidate wi...Show more
    Last updated: 24 days ago • Promoted
    Lead Cloud Data Engineer

    Lead Cloud Data Engineer

    Tata Consultancy Services • Bengaluru, Republic Of India, IN
    Role- GCP Senior Data Engineer.Location- Hyderabad, Chennai, Bangalore, Kolkata, Mumbai.Data Integration, orchestration mechanism, ability to design BI solutions using Cloud Store, Big Query, Cloud...Show more
    Last updated: 19 days ago • Promoted
    Senior Data Engineer - Python / SQL / ETL

    Senior Data Engineer - Python / SQL / ETL

    TargetHR • Bangalore
    Description : - We are looking for an experienced Senior Data Engineer who can join us as part of the core team and who ...Show more
    Last updated: 15 days ago • Promoted
    Data Engineer - Python / SQL

    Data Engineer - Python / SQL

    Confidential • India, Bengaluru / Bangalore
    Data Pipeline Development : Design, build, and optimize scalable, secure, and reliable data pipelines to ingest, process, and transform large volumes of structured and unstructured data.Data Archit...Show more
    Last updated: 10 hours ago • Promoted • New!
    UsefulBI Corporation - Senior Data Engineer - Python / SQL

    UsefulBI Corporation - Senior Data Engineer - Python / SQL

    USEFULBI CORPORATION • Bangalore
    UsefulBI is looking for highly skilled candidates with expertise in generating power business insights from very large datasets, in this space where the primary aim would be to enable needle-moving...Show more
    Last updated: 15 days ago • Promoted
    Data Engineer - Python / SQL / Spark

    Data Engineer - Python / SQL / Spark

    SK HR Consultants • Bangalore
    Description : - Design, build and maintain scalable data pipelines for integrating diverse data sources.Implement and ma...Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer - Python / SQL / ETL

    Lead Data Engineer - Python / SQL / ETL

    Scaling Theory Technologies Pvt Ltd • Bangalore
    Role Overview : We are seeking an experienced Lead Data Engineer to design, build, and optimize scalable data platforms within the Azure Databricks ecosystem.The ideal candida...Show more
    Last updated: 29 days ago • Promoted
    Senior Data & Analytics Engineer - Python / SQL

    Senior Data & Analytics Engineer - Python / SQL

    Blinkhealth • Bangalore
    What You'll Need : 5+ years hands-on experience with a combination of : - Python software development for data pipelines and automation.SQ...Show more
    Last updated: 30+ days ago • Promoted
    Lead GCP Data Engineer

    Lead GCP Data Engineer

    Impetus • Bengaluru, Karnataka, India
    Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark).You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform.You will manage a team of ...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer | Bangalore | Leading Big 4 Consulting Firm

    Data Engineer | Bangalore | Leading Big 4 Consulting Firm

    Acme Services • Bengaluru, Karnataka, India
    We are hiring sharp, hands-on Data Engineers to build scalable data solutions and drive performance across both traditional and cloud-based data platforms. Key Roles and Responsibilities.Strong comm...Show more
    Last updated: 7 days ago • Promoted
    Data Engineer - Python / SQL

    Data Engineer - Python / SQL

    Spigot Software • Bangalore
    Responsibilities : - Design, develop, and maintain scalable data pipelines.Collaborate with cross-functional teams to understand data requirements and deliver solutio...Show more
    Last updated: 22 days ago • Promoted
    Lead Data Engineer - Python / SQL

    Lead Data Engineer - Python / SQL

    Taylor and francis • Bangalore
    Job Description : Closing Date : Applications will close on 11th November 2025.What you will be doing : ...Show more
    Last updated: 24 days ago • Promoted
    Senior Data Engineer - Python / SQL / PySpark

    Senior Data Engineer - Python / SQL / PySpark

    AVIS BUDGET TECHNOLOGY INNOVATIONS PRIVATE LIMITED • Bangalore
    Description : Role : Senior Data Engineer Location : Bangalore Exp : 8-11 Years NP : ...Show more
    Last updated: 24 days ago • Promoted
    Data Engineer

    Data Engineer

    Ubique Systems • Bengaluru, India
    Primary skills : Python, SQL, data lakes, azure.Pipeline Development & Automation.Design, build, and maintain CI / CD pipelines to automate deployment of DQ rules and data services across environments...Show more
    Last updated: 30+ days ago • Promoted