Talent.com
Senior Data Pipeline Specialist

Senior Data Pipeline Specialist

Rainier Softech Solutions Pvt LtdBengaluru, Republic Of India, IN
21 hours ago
Job description

Sr. Data Engineer (Databricks)

About the Role

We’re looking for a hands-on Data Engineer to build reliable, scalable data pipelines on Databricks. You’ll turn requirements into production-grade ELT / ETL jobs, Delta Lake tables, and reusable components that speed up our teams. In this role, you’ll implement and improve reference patterns, optimize Spark for performance and cost, apply best practices with Unity Catalog and workflow orchestration, and ship high-quality code others can build on. If you love solving data problems at scale and empowering teammates through clean, well-documented solutions, you’ll thrive here.

Core Qualifications :

  • Bachelor’s degree in computer science, Engineering, or a related field
  • 7+ years of experience in software / data engineering, including at least 2 years working with Databricks and Apache Spark
  • Strong proficiency in Python, SQL, and PySpark
  • Deep understanding of AWS and Azure Cloud service
  • Experience with Databricks Data LakeHouse, Databricks Workflows, and Databricks SQL, dbt
  • Solid grasp of data Lakehouse and warehousing architecture
  • Prior experience supporting AI / ML workflows, including training data pipelines and model deployment support
  • Familiarity with infrastructure-as-code tools like Terraform or CloudFormation
  • Strong analytical and troubleshooting skills in a fast-paced, agile environment
  • Excellent collaboration skills for interfacing with both technical and non-technical customer stakeholders
  • Clear communicator with strong documentation habits
  • Comfortable leading discussions, offering strategic input, and mentoring others

Key Responsibilities :

  • The ideal candidate will have a strong background in building scalable data pipelines, optimizing big data workflows, and integrating Databricks with cloud services
  • This role will play a pivotal part in enabling the customer’s data engineering and analytics initiatives—especially those tied to AI-driven solutions and projects—by implementing cloud-native architectures that fuel innovation and sustainability
  • Partner directly with the customer’s data engineering team to design and deliver scalable, cloud-based data solutions
  • Execute complex ad-hoc queries using Databricks SQL to explore large lakehouse datasets and uncover actionable insights
  • Leverage Databricks notebooks to develop robust data transformation workflows using PySpark and SQL
  • Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks
  • Build ETL / ELT workflows with AWS and Azure Services
  • Optimize Spark jobs for both performance and cost within the customer’s cloud infrastructure
  • Collaborate with data scientists, ML engineers, and business analysts to support AI and machine learning use cases, including data preparation, feature engineering, and model operationalization
  • Contribute to the development of AI-powered solutions that improve operational efficiency, route optimization, and predictive maintenance in the waste management domain
  • Implement CI / CD pipelines for Databricks jobs using GitHub Actions, Azure DevOps, or Jenkins
  • Ensure data quality, lineage, and compliance through tools like Unity Catalog, Delta Lake, and AWS Lake Formation
  • Troubleshoot and maintain production data pipelines
  • Provide mentorship and share best practices with both internal and customer teams
  • Create a job alert for this search

    Data Pipeline Specialist • Bengaluru, Republic Of India, IN

    Related jobs
    • Promoted
    GCP Data Pipeline Specialist

    GCP Data Pipeline Specialist

    Tata Consultancy ServicesBengaluru, Republic Of India, IN
    GCP Big Query, Dataproc, Pub Sub, Airflow.Expertise in Google Data Engineering services – Big Query, Dataproc etc.Strong experience in programming languages such as Python, PySpark data manipulatio...Show moreLast updated: 12 days ago
    • Promoted
    Principal Data Pipeline Specialist

    Principal Data Pipeline Specialist

    Societe Generale Global Solution CentreBengaluru, Republic Of India, IN
    With 7 - 10 years of experience, you will design and maintain scalable data pipelines, ensure compliance with global data regulations (including GDPR), and empower auditors / inspectors with high-qua...Show moreLast updated: 23 days ago
    • Promoted
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    LTIMindtreeBengaluru, Republic Of India, IN
    Seeking an experienced Senior Specialist with 7 to 11 years of expertise in Azure Data Factory and Azure SQL to design and manage scalable data integration solutions. Design develop and maintain sca...Show moreLast updated: 30+ days ago
    • Promoted
    Azure Data Pipeline Specialist

    Azure Data Pipeline Specialist

    Tata Consultancy ServicesBengaluru, Republic Of India, IN
    TCS is conducting Weekend Virtual drive for the role of.Years of experience : 7 to 10 yrs.Location : Bangalore, Chennai, Hyderabad, Pune. Date of Interview : 1st Nov, 2025.Time of Interview : 10 AM to 2...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    CargillBengaluru, Republic Of India, IN
    The Senior Professional, Data Engineering designs, builds, and maintains complex data systems that enable data analysis and reporting. This role includes a focus on Treasury-specific data solutions,...Show moreLast updated: 6 days ago
    • Promoted
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    BrillioBengaluru, Republic Of India, IN
    Brillio is a global digital technology company focused on delivering innovative solutions in data, AI, and cloud.With a strong presence across industries like banking, healthcare, and telecom, Bril...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Data Pipeline Developer

    Lead Data Pipeline Developer

    LiciousBengaluru, Republic Of India, IN
    We are seeking a highly skilled and experienced.SDE 3 – Data Platform Engineer.This role requires deep technical proficiency, architectural foresight, and strategic thinking to ensure our data syst...Show moreLast updated: 15 days ago
    • Promoted
    Data Pipeline Developer

    Data Pipeline Developer

    LTIMindtreeBengaluru, Republic Of India, IN
    Location : Any LTIMindtree location.Notice Period : Immediate to 90 Days.Extract transform and load ETL data from multiple sources ERP logistics compliance tools etc. Develop reusable data pipelines t...Show moreLast updated: 13 days ago
    • Promoted
    Data Pipeline Specialist

    Data Pipeline Specialist

    Tata Consultancy ServicesBengaluru, Republic Of India, IN
    Notice Period- • • • 30-45 DAYS • • •.AWS Snowflake, Python, Pyspark.Support pipeline buildout, implementation, and RFC deployments for new business use cases. Assignment group / project repo segregation ...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Data Pipeline Engineer

    Lead Data Pipeline Engineer

    ACL DigitalBengaluru, Republic Of India, IN
    Tech Lead design, build, and maintain the infrastructure for an organization's data, ensuring it is reliable, accessible, and optimized for use by data scientists and analysts.Key responsibilities ...Show moreLast updated: 13 days ago
    • Promoted
    Data Pipeline Specialist

    Data Pipeline Specialist

    TaggdBengaluru, Republic Of India, IN
    We are on the lookout for a skilled Datastage Developer to join our dynamic team! If you have a passion for data management and possess the ability to work with ETL processes, we want to connect wi...Show moreLast updated: 15 days ago
    • Promoted
    Data Pipeline Specialist

    Data Pipeline Specialist

    Grid DynamicsBengaluru, Republic Of India, IN
    Grid Dynamics (Nasdaq : GDYN) is a digital-native technology services provider that accelerates growth and bolsters competitive advantage for Fortune 1000 companies. Grid Dynamics provides digital tra...Show moreLast updated: 13 days ago
    • Promoted
    Data Mining Specialist

    Data Mining Specialist

    ACL DigitalBengaluru, Karnataka, India
    Hiring for Data Mining Specialist / Power Bi : Immediate Joiners : 9-12 Yrs : Bangalore Technical Skills : • Strong expertise in Power BI, including advanced knowledge of Power Query and DAX for build...Show moreLast updated: 5 days ago
    • Promoted
    • New!
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    Quess IT StaffingBengaluru, Republic Of India, IN
    We are looking for a highly skilled.The ideal candidate should have deep expertise in.Snowflake, PySpark, SQL, DBT, Airflow.Show moreLast updated: 11 hours ago
    • Promoted
    Cloud Infrastructure and Data Pipeline Specialist

    Cloud Infrastructure and Data Pipeline Specialist

    PriceEasy AIBengaluru, Republic Of India, IN
    Job Title : DevOps & Tech Support Engineer.MG Road, Bangalore (Work From Office – 5 Days / Week).PriceEasy is a fast-growing SaaS company focused on pricing intelligence, data automation, and large-sc...Show moreLast updated: 5 days ago
    • Promoted
    Senior AI and Data Pipeline Specialist

    Senior AI and Data Pipeline Specialist

    SCIENTEBengaluru, Republic Of India, IN
    Sciente Group is a multi-award winning, ISO9001 / Great Place to Work certified, Singapore headquartered technology focused organisation with business interest in Singapore (HQ), USA, Malaysia, Phili...Show moreLast updated: 29 days ago
    • Promoted
    Risk Data Pipeline

    Risk Data Pipeline

    ConfidentialBengaluru, Karnataka, India
    We are seeking a highly skilled.The ideal candidate will have strong programming expertise, hands-on experience with data engineering workflows, and a deep understanding of financial instruments an...Show moreLast updated: 13 days ago
    • Promoted
    Senior Data Pipeline Engineer

    Senior Data Pipeline Engineer

    InfosysBengaluru, Republic Of India, IN
    As a Software Developer you will work in a constantly evolving environment, due to technological advances and the strategic direction of the organization you work for. You will create, maintain, aud...Show moreLast updated: 30+ days ago