Talent.com
This job offer is not available in your country.
Senior Data Engineer - Python / PySpark

Senior Data Engineer - Python / PySpark

Vtricks TechnologiesBangalore
15 days ago
Job description

Job Title : Senior Data Engineer - Azure | ADF | Databricks | PySpark | AWS

Location : Bangalore, Hyderabad, Chennai (Hybrid Mode)

Experience Required : 5+ Years (Budget around 25 LPA)

Employment Type : Full-Time

Industry : IT Services & Consulting / Data & Analytics

Notice Period : Immediate to 30 days preferred

Job Description :

We are looking for a Senior Data Engineer who is passionate about designing and developing scalable data pipelines, optimizing data architecture, and working with advanced big data tools and cloud platforms. This is a great opportunity to be a key player in transforming data into meaningful insights by leveraging modern data engineering practices on Azure, AWS, and Databricks.

You will be working with cross-functional teams including data scientists, analysts, and software engineers to deliver robust data solutions. The ideal candidate will be technically strong in Azure Data Factory, PySpark, Databricks, and AWS services, and will have experience in building end-to-end ETL workflows and driving business impact through data.

Key Responsibilities :

  • Design, build, and maintain scalable and reliable data pipelines and ETL workflows
  • Implement data ingestion and transformation using Azure Data Factory (ADF) and Azure Databricks (PySpark)
  • Work across multiple data platforms including Azure, AWS, Snowflake, and Redshift
  • Collaborate with data scientists and business teams to understand data needs and deliver solutions
  • Optimize data storage, processing, and retrieval for performance and cost-effectiveness
  • Develop data quality checks and monitoring frameworks for pipeline health
  • Ensure data governance, security, and compliance with industry standards
  • Lead code reviews, set data engineering standards, and mentor junior team members
  • Propose and evaluate new tools and technologies for continuous improvement

Must-Have Skills :

  • Strong programming skills in Python, SQL, or Scala
  • Azure Data Factory, Azure Databricks, Synapse Analytics
  • Hands-on with PySpark, Spark, Hadoop, Hive
  • Experience with cloud platforms (Azure preferred; AWS / GCP acceptable)
  • Data Warehousing : Snowflake, Redshift, BigQuery
  • Strong ETL / ELT pipeline development experience
  • Workflow orchestration tools such as Airflow, Prefect, or Luigi
  • Excellent problem-solving, debugging, and communication skills
  • Nice to Have :

  • Experience with real-time streaming tools (Kafka, Flink, Spark Streaming)
  • Exposure to data governance tools and regulations (GDPR, HIPAA)
  • Familiarity with ML model integration into data pipelines
  • Containerization and CI / CD exposure : Docker, Git, Kubernetes (basic)
  • Experience with Vector databases and unstructured data handling
  • Technical Environment :

  • Programming : Python, Scala, SQL
  • Big Data Tools : Spark, Hadoop, Hive
  • Cloud Platforms : Azure (ADF, Databricks, Synapse), AWS (S3, Glue, Lambda), GCP
  • Data Warehousing : Snowflake, Redshift, BigQuery
  • Databases : PostgreSQL, MySQL, MongoDB, Cassandra
  • Orchestration : Apache Airflow, Prefect, Luigi
  • Tools : Git, Docker, Azure DevOps, CI / CD pipelines
  • Soft Skills :

  • Strong analytical thinking and problem-solving abilities
  • Excellent verbal and written communication
  • Collaborative team player with leadership qualities
  • Self-motivated, organized, and able to manage multiple projects
  • Education & Certifications

  • Bachelor's or Master's Degree in Computer Science, IT, Engineering, or equivalent
  • Cloud certifications (e.g., Microsoft Azure Data Engineer, AWS Big Data) are a plus
  • Key Result Areas (KRAs) :

  • Timely delivery of high-performance data pipelines
  • Quality of data integration and governance compliance
  • Business team satisfaction and data readiness
  • Proactive optimization of data processing workloads
  • Key Performance Indicators (KPIs) :

  • Pipeline uptime and performance metrics
  • Reduction in overall data latency
  • Zero critical issues in production post-release
  • Stakeholder satisfaction score
  • Number of successful integrations and migrations
  • ref : hirist.tech)

    Create a job alert for this search

    Senior Data Engineer • Bangalore

    Related jobs
    • Promoted
    Senior Data Engineer - Python / PySpark

    Senior Data Engineer - Python / PySpark

    Antal InternationalBangalore
    We are looking for a skilled and driven Data Engineer to join our dynamic technology team.As a Data Engineer, you will be responsible for building and maintaining robust, scalable, and high-perform...Show moreLast updated: 27 days ago
    • Promoted
    • New!
    AWS Data Engineer - Python / PySpark

    AWS Data Engineer - Python / PySpark

    BLJ Tech GeeksBangalore
    Role Summary : We are looking for a skilled and detail-oriented Data Engineer to join our growing data team.The ideal candidate will be responsible for design...Show moreLast updated: 14 hours ago
    • Promoted
    AWS Data Engineer - Python / PySpark

    AWS Data Engineer - Python / PySpark

    BLJ TECH GEEKSBangalore
    Job Title : Data Engineer Company - PWC Location : Bangalore (Hybrid) Job Typ...Show moreLast updated: 30+ days ago
    • Promoted
    AWS Data Engineer - Python / PySpark

    AWS Data Engineer - Python / PySpark

    Megathil EndeavorsBangalore
    About the Role : We are looking for a highly skilled and motivated AWS Data Engineer to join our client team.The ideal candidate will have hands-on experience...Show moreLast updated: 10 days ago
    • Promoted
    Data Engineer - Python / PySpark

    Data Engineer - Python / PySpark

    DashhireBangalore
    Responsibilities : - Design, develop, and optimize ETL pipelines using Azure Data Factory (ADF).Build and manage data lakes and data warehouses usi...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer (Exp 5+, AWS, Python, Pyspark)

    Senior Data Engineer (Exp 5+, AWS, Python, Pyspark)

    Atyeti IncBengaluru, Karnataka, India
    Job Title : Senior Data Engineer.Shift Timing : From 11 : 00 AM onwards.Experience : (5+ in Python and PySpark).Education : Bachelor’s degree in Computer Science, Engineering, Statistics, Mathematics, or...Show moreLast updated: 11 days ago
    • Promoted
    Senior Data Engineer - Python / PySpark

    Senior Data Engineer - Python / PySpark

    Akira Consultancy Pvt. Ltd.Bangalore
    About Akira : AKIRA is a bootstrapped Data & Analytics Services company driven by technology excellence, consultative practices, and a focus on building technolog...Show moreLast updated: 4 days ago
    • Promoted
    Senior Data Engineer - PySpark / Python

    Senior Data Engineer - PySpark / Python

    Trigent Software Professional ServicesBangalore
    We're Hiring : Senior Data Engineer | 8+ Years | Remote (India) We're looking for a Senior Data Engineer with 8+ years of experience to join our dynamic dat...Show moreLast updated: 26 days ago
    • Promoted
    Infobell IT - Senior Data Engineer - Python / PySpark

    Infobell IT - Senior Data Engineer - Python / PySpark

    InfoBangalore
    Job Title : Senior Data Engineer Location : Bangalore (Work From Office) Experience Required : 5+ years<...Show moreLast updated: 28 days ago
    • Promoted
    AWS Data Engineer - ETL / PySpark / Python

    AWS Data Engineer - ETL / PySpark / Python

    ChangeleadersBangalore
    Position : AWS Data Engineer Experience : 5+ Yrs Location : Bangalore, Hyderabad, Chennai, Pune, Gurgaon <...Show moreLast updated: 15 days ago
    • Promoted
    Data Engineer - Python / PySpark

    Data Engineer - Python / PySpark

    Master Mind ConsultancyBangalore
    Primary Mandate Skills : - Python - Pyspark - Azure Key responsibilities : ...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Azure Data Engineer - Python / PySpark

    Senior Azure Data Engineer - Python / PySpark

    bellfast management private limitedBangalore
    Senior Azure Data : - As a Senior Azure Data Engineer you will leverage Azure technologies to drive data transformation, analytics, and machine le...Show moreLast updated: 11 days ago
    • Promoted
    AWS Data Engineer - Spark / PySpark / Python

    AWS Data Engineer - Spark / PySpark / Python

    Tekfortune IT India Pvt LtdBangalore
    Title : AWS Data Engineer (Contractual role) Experience : 6 - 18 Years Work Location : Pune / Bangalore Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer - Python / PySpark

    Data Engineer - Python / PySpark

    Tiyasha Software Private Limited.Bangalore
    Job Title : Data Engineer (Contract-to-Hire) Experience Level : 6 to 10 Years Location : Hybrid - 3 Days Onsite (Pune, Bangalore, Noida, Mumbai, Hyderabad)...Show moreLast updated: 30+ days ago
    • Promoted
    CDM Smith - Senior Data Engineer - Python / PySpark

    CDM Smith - Senior Data Engineer - Python / PySpark

    CDM SMITH GLOBAL SERVICES INDIA LLPBangalore
    Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer with deep expertise in Databricks to join our Digital Capital tea...Show moreLast updated: 17 days ago
    • Promoted
    UsefulBI Corporation - Data Engineer - Python / PySpark

    UsefulBI Corporation - Data Engineer - Python / PySpark

    Useful BI CorporationBangalore
    Location : Bangalore / Lucknow (8 days in a month-Mandatory WFO) Role Type : Full-time, Senior Level Show moreLast updated: 30+ days ago
    • Promoted
    Azure Data Engineer - Python / PySpark

    Azure Data Engineer - Python / PySpark

    AceNetBangalore
    About Us : AceNet Consulting is a fast-growing global business and technology consulting firm specializing in business strategy, digital transformation, technology co...Show moreLast updated: 29 days ago
    • Promoted
    Magna - Data Engineer - Python / PySpark

    Magna - Data Engineer - Python / PySpark

    Magna InternationalBangalore
    Key Responsibilities : - Design, develop, and maintain scalable data pipelines using Python and PySpark < / p...Show moreLast updated: 2 days ago