Talent.com
Lead Data Engineer - Python / SQL

Lead Data Engineer - Python / SQL

Marktine Technology Solutions Pvt LtdBangalore
8 days ago
Job description

Description :

Relevant Experience : 7-10 years.

Job description :

  • Lead the data engineering team by driving the design, build, test, and launch of new data pipelines and models on the production data platform.
  • Define and implement the processes needed to achieve operational excellence in all areas, including agile development, and data solutions.
  • Oversee the design, development and maintenance of data infrastructure, including data warehouses, data lakes, data integration components supporting optimal extraction, transformation, and loading of data.
  • Leading the communication with vendors, clients, the leadership team, and other stakeholders to facilitate effective project management and provide ongoing support.
  • Collaborate with business owners for roadmap planning and prioritization, to deliver robust cloud-based data solutions for our customers.
  • Work closely with cross-functional teams to propel and execute necessary solution enhancements and provide support for existing solutions.
  • Define and enforce best practices for data engineering, data modeling, and data quality to ensure accuracy, reliability, reusability, security and consistency of data.
  • Evaluate and implement new technologies, tools, and frameworks to improve the performance and efficiency of the data engineering team.
  • Lead the planning, prioritization, and execution and tracking of data engineering projects, ensuring timely delivery and alignment with business objectives.
  • Establish and monitor key performance indicators (KPIs) to measure the performance and effectiveness of the data engineering team.
  • Stay current with industry trends and advancements in data engineering, Azure Cloud, and provide strategic guidance for adopting new technologies or :
  • 8-12 years of demonstrable experience in data engineering, analytics, data warehousing, data management, Data governance and Compliance Requirements.
  • Experience managing data engineers and guiding a team of engineers through project planning, execution, tracking and monitoring, and quality control stages.
  • Solid experience with cloud-based data tools and platforms.
  • Proficient in implementing efficient cost management strategies, particularly about storage and computational expenses.
  • Experience building processes supporting data transformation, data structures, metadata, security, governance, and workload management.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Expertise in designing and optimizing data models, RDBMS, NoSQL DBs, and data warehousing solutions.
  • Excellent leadership, communication, and interpersonal skills, with the ability to effectively collaborate with cross-functional teams.
  • Proven ability to drive innovation, make strategic decisions and lead complex data engineering initiatives to successful completion.

Skill Required : Must Haves :

  • SQL, Python, Py Spark, Spark SQL, Spark, Distributed Systems.
  • Databricks, ADLS Gen 2 Blob Storage, Azure DevOps, Azure Data Factory.
  • ETL, Building Data Pipelines, Data Warehousing, Datamart, Data Modelling and Governance.
  • Agile Practices, SDLC, DevOps practices, and CI / CD pipelines for data engineering workflows.
  • Solid understanding of the Microsoft Azure stack for large-scale data engineering developments and deployments is highly preferred.
  • Hands-on experience with Azure Databricks, including data ingestion, data transformation, workflow management and optimization, monitoring and troubleshooting Spark Jobs.
  • Ability to set up and manage Azure Data Lake Storage (ADLS) Gen 2 accounts, and familiarity with data lake architecture and best practices.
  • Familiarity with big data frameworks (e.g., Apache Spark).
  • Knowledge of Azure Key Vaults for securely storing and managing cryptographic keys, secrets, and certificates.
  • Good to Have :

  • Event Hubs for log management.
  • Workflow Orchestration.
  • Cosmos DB.
  • Power BI.
  • Professional Services Background.
  • Scala, Java.
  • (ref : hirist.tech)

    Create a job alert for this search

    Lead Data Engineer • Bangalore

    Related jobs
    • Promoted
    Senior Analytics Support Engineer (GA4 | GTM | Looker | Tableau | SQL | AWS)

    Senior Analytics Support Engineer (GA4 | GTM | Looker | Tableau | SQL | AWS)

    Sky Systems, Inc. (SkySys)hosur, tamil nadu, in
    Senior Analytics Support Engineer.Full-Time Contract (40hrs / week).Hybrid (2x days onsite) in Bangalore, India.We’re looking for a skilled Analytics Support Engineer experienced in.GA4, GTM, Looker,...Show moreLast updated: 8 days ago
    • Promoted
    Senior Python Backend Developer (AI, Cloud & CI / CD)Agentic AI Engineer

    Senior Python Backend Developer (AI, Cloud & CI / CD)Agentic AI Engineer

    GenNow.AIhosur, tamil nadu, in
    Senior Python Backend Developer (4–8 years of experience).Python (FastAPI, Flask, or Django).Deploy and manage backend services on. GitHub Actions, GitLab CI, or Jenkins.Containerize applications us...Show moreLast updated: 10 days ago
    • Promoted
    Senior Azure Data Engineer (Architect) - Pan India

    Senior Azure Data Engineer (Architect) - Pan India

    Genpacthosur, tamil nadu, in
    We are seeking an experienced Azure Data & Analytics Architect with a strong background in delivery and excellent communication skills. The ideal candidate will have experience and a proven track re...Show moreLast updated: 10 days ago
    • Promoted
    AI - Lead Founding Engineer

    AI - Lead Founding Engineer

    BPMLinkshosur, tamil nadu, in
    Deep experience with LLMs, embeddings, vector search, knowledge graphs, and RAG / GraphRAG pipelines.Strong programming expertise in Python (LangChain, Haystack, FastAPI), and modern backend framewor...Show moreLast updated: 17 days ago
    • Promoted
    Platform Engineer – Python / Databricks / Notebooks / Kubernetes

    Platform Engineer – Python / Databricks / Notebooks / Kubernetes

    Syntasahosur, tamil nadu, in
    Platform Engineer – Python / Databricks / Notebooks / Kubernetes.Syntasa is seeking a high-caliber and dedicated.Syntasa Technologies India Private Limited. This position offers an exciting opportuni...Show moreLast updated: 7 days ago
    • Promoted
    PL / SQL Developer

    PL / SQL Developer

    Datacore Technologieshosur, tamil nadu, in
    Python experience with more than 6 years of experience in developing python applications, with working experience in database connectivity, multi-threading and multi-processing , file handling (xml...Show moreLast updated: 7 days ago
    • Promoted
    Big Data Developer(Hive, Pyspark)

    Big Data Developer(Hive, Pyspark)

    Tata Consultancy Serviceshosur, tamil nadu, in
    Big Data Developer(Hive, Pyspark).Years of Experience : 4 to 10 yrs.Date of Interview : 25th Oct, 2025.Mode of Interview : Virtual Drive( Cisco or WebEx). Hands-on experience of Hadoop, Python, PySpark...Show moreLast updated: 10 days ago
    • Promoted
    GCP Data Architect

    GCP Data Architect

    Tata Consultancy Serviceshosur, tamil nadu, in
    Job Title : - GCP Data Architect.Design and Implement Data Architectures : Architect and build scalable, end-to-end data solutions on GCP, encompassing data ingestion, transformation, storage, and co...Show moreLast updated: 18 days ago
    • Promoted
    Data Engineer(Azure / AWS, Python / Pyspark, SQL)

    Data Engineer(Azure / AWS, Python / Pyspark, SQL)

    Sail AnalyticsBengaluru, Karnataka, India
    Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift). Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago
    • Promoted
    Azure-ETL Engineer

    Azure-ETL Engineer

    Coforgehosur, tamil nadu, in
    The ideal candidate will be responsible for designing, implementing, and maintaining scalable data solutions that support business intelligence and analytics needs. Design, implement, and maintain d...Show moreLast updated: 8 days ago
    • Promoted
    Lead GCP Data Engineer

    Lead GCP Data Engineer

    ImpetusBengaluru, Karnataka, India
    Lead Data Engineer – GCP (BigQuery • Composer • Python • PySpark).You will lead the design, build and operation of large-scale data platforms on the Google Cloud Platform.You will manage a team of ...Show moreLast updated: 6 days ago
    • Promoted
    Python Developer / Data Engineer Intern

    Python Developer / Data Engineer Intern

    Kastackhosur, tamil nadu, in
    KaStack is an emerging AI startup dedicated to building real-time applications that solve real-world problems using cutting-edge artificial intelligence technologies. THIS IS AN UNPAID INTERNSHIP RO...Show moreLast updated: 7 days ago
    • Promoted
    Oracle PL / SQL -With Python and AWS(Data Engineer)

    Oracle PL / SQL -With Python and AWS(Data Engineer)

    Fidelity InternationalBengaluru, Karnataka, India
    Design, develop, and optimize complex SQL queries, stored procedures, and data models for Oracle-based systems.Create and maintain efficient data pipelines for extract, transform, and load (ETL) pr...Show moreLast updated: 3 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Veraxionhosur, tamil nadu, in
    Python, Spark, DBT, and AWS-native services.Agile environment to deliver scalable, secure, and high-performance data solutions. Python, DBT, and AWS services (Data Ops Live).Deliver end-to-end data ...Show moreLast updated: 8 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    USEReadyhosur, tamil nadu, in
    Job Title : Senior Databricks Engineer.As a Senior Databricks Engineer, you will be responsible for designing, developing, and optimizing our data architecture and pipelines on the Databricks Lakeho...Show moreLast updated: 29 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Sonata SoftwareBengaluru, Karnataka, India
    Experience building data pipelines.Highly proficient in python, Data frames.Experience with AWS Lambda, Events (SQS), API calls, Kinesis, DynamoDB. Highly proficient in SQL Query.Snowflake or experi...Show moreLast updated: 17 days ago
    • Promoted
    GCP Data Engineer

    GCP Data Engineer

    JRD Systemshosur, tamil nadu, in
    Collaborate with data architects, analysts, and business teams to define and maintain.Roadnet) and modernized data sources. Partner with governance and infrastructure teams to establish.Automate and...Show moreLast updated: 2 days ago
    • Promoted
    Data Engineer

    Data Engineer

    RAOS INFOSOFT JOIN PRIVATE LIMITEDBengaluru, Karnataka, India
    Looking to hire a data engineer with the following : .Born programmer, Strong in Python frameworks.Strong in SQL and RDBMS concepts. Should join within 1 week from the time interview is scheduled.Idea...Show moreLast updated: 8 days ago