Talent.com
Principal Data Engineer
Principal Data EngineerRainier Softech Solutions Pvt Ltd • Bengaluru, Republic Of India, IN
No longer accepting applications
Principal Data Engineer

Principal Data Engineer

Rainier Softech Solutions Pvt Ltd • Bengaluru, Republic Of India, IN
15 days ago
Job description

Sr. Data Engineer (Databricks)

About the Role

We’re looking for a hands-on Data Engineer to build reliable, scalable data pipelines on Databricks. You’ll turn requirements into production-grade ELT / ETL jobs, Delta Lake tables, and reusable components that speed up our teams. In this role, you’ll implement and improve reference patterns, optimize Spark for performance and cost, apply best practices with Unity Catalog and workflow orchestration, and ship high-quality code others can build on. If you love solving data problems at scale and empowering teammates through clean, well-documented solutions, you’ll thrive here.

Core Qualifications :

  • Bachelor’s degree in computer science, Engineering, or a related field
  • 7+ years of experience in software / data engineering, including at least 2 years working with Databricks and Apache Spark
  • Strong proficiency in Python, SQL, and PySpark
  • Deep understanding of AWS and Azure Cloud service
  • Experience with Databricks Data LakeHouse, Databricks Workflows, and Databricks SQL, dbt
  • Solid grasp of data Lakehouse and warehousing architecture
  • Prior experience supporting AI / ML workflows, including training data pipelines and model deployment support
  • Familiarity with infrastructure-as-code tools like Terraform or CloudFormation
  • Strong analytical and troubleshooting skills in a fast-paced, agile environment
  • Excellent collaboration skills for interfacing with both technical and non-technical customer stakeholders
  • Clear communicator with strong documentation habits
  • Comfortable leading discussions, offering strategic input, and mentoring others

Key Responsibilities :

  • The ideal candidate will have a strong background in building scalable data pipelines, optimizing big data workflows, and integrating Databricks with cloud services
  • This role will play a pivotal part in enabling the customer’s data engineering and analytics initiatives—especially those tied to AI-driven solutions and projects—by implementing cloud-native architectures that fuel innovation and sustainability
  • Partner directly with the customer’s data engineering team to design and deliver scalable, cloud-based data solutions
  • Execute complex ad-hoc queries using Databricks SQL to explore large lakehouse datasets and uncover actionable insights
  • Leverage Databricks notebooks to develop robust data transformation workflows using PySpark and SQL
  • Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks
  • Build ETL / ELT workflows with AWS and Azure Services
  • Optimize Spark jobs for both performance and cost within the customer’s cloud infrastructure
  • Collaborate with data scientists, ML engineers, and business analysts to support AI and machine learning use cases, including data preparation, feature engineering, and model operationalization
  • Contribute to the development of AI-powered solutions that improve operational efficiency, route optimization, and predictive maintenance in the waste management domain
  • Implement CI / CD pipelines for Databricks jobs using GitHub Actions, Azure DevOps, or Jenkins
  • Ensure data quality, lineage, and compliance through tools like Unity Catalog, Delta Lake, and AWS Lake Formation
  • Troubleshoot and maintain production data pipelines
  • Provide mentorship and share best practices with both internal and customer teams
  • Create a job alert for this search

    Principal Data Engineer • Bengaluru, Republic Of India, IN

    Related jobs
    Principal Data Solutions Engineer

    Principal Data Solutions Engineer

    Dr. Martens plc • Bengaluru, Republic Of India, IN
    Working within a multi-disciplined team led by the.Senior Manager - Data Engineer.This is an exciting opportunity to contribute to a dynamic and innovative environment, where your work will directl...Show more
    Last updated: 2 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    Infosys • Bangalore, Bangalore (division), India
    As a Software Developer you will work in a constantly evolving environment, due to technological advances and the strategic direction of the organization you work for. You will create, maintain, aud...Show more
    Last updated: 5 hours ago • Promoted • New!
    Principal Engineer

    Principal Engineer

    Client of Solvecube • Bengaluru, Karnataka, India
    I hope you are enjoying a great day! I am from Solvecube HCM, an AI Based global consulting firm head quartered in Singapore. Our client is a Healthcare AI Startup and looking for a great tech leade...Show more
    Last updated: 9 days ago • Promoted
    Principal AI Engineer

    Principal AI Engineer

    Palo Alto Networks • Bengaluru, Karnataka, India
    At Palo Alto Networks® everything starts and ends with our mission : .Being the cybersecurity partner of choice, protecting our digital way of life. Our vision is a world where each day is safer and m...Show more
    Last updated: 1 day ago • Promoted
    Principal Engineer

    Principal Engineer

    Recro • Bengaluru, Karnataka, India
    We're Hiring : Principal Full Stack Software Engineer (Backend Focus) – Bangalore, India.Are you an expert ready to drive innovation in a globally established environment? Join our Product-based cli...Show more
    Last updated: 30+ days ago • Promoted
    Data Platform Engineer

    Data Platform Engineer

    Grizmo Labs • Bengaluru, Karnataka, India
    Design, build, and maintain scalable data platforms and pipelines to support the needs of the business.Ensure data quality and accuracy by implementing data validation and testing processes.Partici...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Platform Engineer

    Principal Data Platform Engineer

    Pure Storage • Bengaluru, Republic Of India, IN
    FlashBlade / / EXA is Pure Storage’s most advanced platform to date, engineered for the extreme performance, scale, and efficiency demands of modern AI and high-performance computing (HPC) infrastruct...Show more
    Last updated: 19 hours ago • Promoted • New!
    Principal Engineer, Data

    Principal Engineer, Data

    Baxter • Bengaluru, Karnataka, India
    At Baxter we believe every personregardless of who they are or where they are fromdeserves a chance to live a healthy life. It was our founding belief in 1931 and continues to be our guiding princip...Show more
    Last updated: 25 days ago • Promoted
    Principal Engineer, Data Analytics Engineering

    Principal Engineer, Data Analytics Engineering

    Western Digital • Bengaluru, Karnataka, India
    Position : Principal AI Engineer (Data Engineering & GenAI).We are seeking a Principal AI Engineer with deep expertise in data engineering Generative AI (GenAI) and AWS cloud platforms to archit...Show more
    Last updated: 4 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    PriceEasy AI • Bengaluru, Karnataka, India
    Join one of the most respected and cutting-edge companies in the pricing and analytics industry! PriceEasy, a VC-backed leader, empowers convenience retailers with actionable insights through advan...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    CodeMyMobile • Bengaluru, Karnataka, India
    Experience Required - 7 to 10 Years How to Apply : Are you a Data Engineer who cares about clean engineering, autonomy, and solving real data challenges? If this sounds like you, we’d love to conn...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Integration Engineer

    Principal Data Integration Engineer

    ANSR • Bengaluru, Karnataka, India
    ANSR is hiring for one of its clients.About 4flow : Headquartered in Berlin, Germany, 4flow provides consulting, software and services for logistics and supply chain management.More than 1300 team m...Show more
    Last updated: 21 hours ago • Promoted • New!
    Principal Data Engineer

    Principal Data Engineer

    MathCo • Bengaluru, Republic Of India, IN
    TheMathCompany or MathCo® is a global Enterprise AI and Analytics company trusted for data-driven decision-making by leading Fortune 500 and Global 2000 enterprises. Founded in 2016, MathCo builds c...Show more
    Last updated: 1 day ago • Promoted
    Principal Data Platform Engineer

    Principal Data Platform Engineer

    North Hires • Bengaluru, Republic Of India, IN
    North Hires is a leading consulting firm specializing in Custom Software Development, Recruitment, Sourcing, and Executive Search services. With a strong presence in the USA, UK, India, and EMEA, No...Show more
    Last updated: 19 hours ago • Promoted • New!
    Principal Data Engineer

    Principal Data Engineer

    ScaleneWorks • Bengaluru, Karnataka, India
    Quick Apply
    Being aware of departmental guidance on best practices, processes, templates, clauses and playbooks.Show more
    Last updated: 30+ days ago
    Principal Data Engineer

    Principal Data Engineer

    Tata Consultancy Services • Bengaluru, Karnataka, India
    Skill : Principal Data Engineer.Possesses hands-on technical experience working in AWS.The individual should possess a robust set of technical and soft skills and be an excellent AWS Data Engineer ...Show more
    Last updated: 17 days ago • Promoted
    Principal Data Platform Engineer

    Principal Data Platform Engineer

    Wayfair • Bengaluru, Republic Of India, IN
    Staff Engineer - Data Platforms- Software Engineering.Wayfair is seeking a passionate and driven Staff Engineer to join our. In this pivotal role, you will be instrumental in shaping and executing W...Show more
    Last updated: 21 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    Harman Becker Automotive Systems Manufacturing Kft • Bengaluru, Karnataka, India
    HARMANs engineers and designers are creative purposeful and agile.As part of this team youll combine your technical expertise with innovative ideas to help drive cutting-edge solutions in the car e...Show more
    Last updated: 16 days ago • Promoted