Talent.com
Principal Data Engineer
Principal Data EngineerRainier Softech Solutions Pvt Ltd • Bengaluru, Republic Of India, IN
Principal Data Engineer

Principal Data Engineer

Rainier Softech Solutions Pvt Ltd • Bengaluru, Republic Of India, IN
10 days ago
Job description

Sr. Data Engineer (Databricks)

About the Role

We’re looking for a hands-on Data Engineer to build reliable, scalable data pipelines on Databricks. You’ll turn requirements into production-grade ELT / ETL jobs, Delta Lake tables, and reusable components that speed up our teams. In this role, you’ll implement and improve reference patterns, optimize Spark for performance and cost, apply best practices with Unity Catalog and workflow orchestration, and ship high-quality code others can build on. If you love solving data problems at scale and empowering teammates through clean, well-documented solutions, you’ll thrive here.

Core Qualifications :

  • Bachelor’s degree in computer science, Engineering, or a related field
  • 7+ years of experience in software / data engineering, including at least 2 years working with Databricks and Apache Spark
  • Strong proficiency in Python, SQL, and PySpark
  • Deep understanding of AWS and Azure Cloud service
  • Experience with Databricks Data LakeHouse, Databricks Workflows, and Databricks SQL, dbt
  • Solid grasp of data Lakehouse and warehousing architecture
  • Prior experience supporting AI / ML workflows, including training data pipelines and model deployment support
  • Familiarity with infrastructure-as-code tools like Terraform or CloudFormation
  • Strong analytical and troubleshooting skills in a fast-paced, agile environment
  • Excellent collaboration skills for interfacing with both technical and non-technical customer stakeholders
  • Clear communicator with strong documentation habits
  • Comfortable leading discussions, offering strategic input, and mentoring others

Key Responsibilities :

  • The ideal candidate will have a strong background in building scalable data pipelines, optimizing big data workflows, and integrating Databricks with cloud services
  • This role will play a pivotal part in enabling the customer’s data engineering and analytics initiatives—especially those tied to AI-driven solutions and projects—by implementing cloud-native architectures that fuel innovation and sustainability
  • Partner directly with the customer’s data engineering team to design and deliver scalable, cloud-based data solutions
  • Execute complex ad-hoc queries using Databricks SQL to explore large lakehouse datasets and uncover actionable insights
  • Leverage Databricks notebooks to develop robust data transformation workflows using PySpark and SQL
  • Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks
  • Build ETL / ELT workflows with AWS and Azure Services
  • Optimize Spark jobs for both performance and cost within the customer’s cloud infrastructure
  • Collaborate with data scientists, ML engineers, and business analysts to support AI and machine learning use cases, including data preparation, feature engineering, and model operationalization
  • Contribute to the development of AI-powered solutions that improve operational efficiency, route optimization, and predictive maintenance in the waste management domain
  • Implement CI / CD pipelines for Databricks jobs using GitHub Actions, Azure DevOps, or Jenkins
  • Ensure data quality, lineage, and compliance through tools like Unity Catalog, Delta Lake, and AWS Lake Formation
  • Troubleshoot and maintain production data pipelines
  • Provide mentorship and share best practices with both internal and customer teams
  • Create a job alert for this search

    Principal Data Engineer • Bengaluru, Republic Of India, IN

    Related jobs
    Principle Data Engineer

    Principle Data Engineer

    KPI Partners • Bengaluru, Karnataka, India
    Mandatory skill : Snowflake, DBT, Airflow, Terraform, AWS / Azure.Notice period : immediate to 30 days.Location : Bangalore, Hyderabad, Pune. We are seeking a highly skilled Principal Data Engineer w...Show more
    Last updated: 7 days ago • Promoted
    Principal Engineer

    Principal Engineer

    Client of Solvecube • Bengaluru, Karnataka, India
    I hope you are enjoying a great day! I am from Solvecube HCM, an AI Based global consulting firm head quartered in Singapore. Our client is a Healthcare AI Startup and looking for a great tech leade...Show more
    Last updated: 3 days ago • Promoted
    Principal Data Engineer (AWS)

    Principal Data Engineer (AWS)

    Tata Consultancy Services • Bengaluru, Republic Of India, IN
    In this key leadership role, you will lead the development of foundational components for a Lakehouse architecture on AWS and drive the migration of existing data processing workflows to the new La...Show more
    Last updated: 22 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    Sigmoid • Bengaluru, Republic Of India, IN
    Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations ...Show more
    Last updated: 6 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    Go Digit General Insurance • Bengaluru, Republic Of India, IN
    Designation : Senior Data Engineer.Digit’s mission is to ‘Make Insurance, Simple’.We are backed by Fairfax- one of the largest global investment firms. We have also been ranked as 'LinkedIn top 5 sta...Show more
    Last updated: 22 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    MakeMyTrip • Bengaluru, India
    Role : Principal Data Engineer.Years of Experience : 8-10 Years.We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products...Show more
    Last updated: 30+ days ago • Promoted
    Principal Engineer

    Principal Engineer

    AIM • Bengaluru, Karnataka, India
    MachineHack is a rapidly growing platform focused on democratizing data science and machine learning through competitive hackathons, tutorials, and community engagement. We are seeking a highly skil...Show more
    Last updated: 3 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    PriceEasy AI • Bengaluru, Karnataka, India
    Join one of the most respected and cutting-edge companies in the pricing and analytics industry! PriceEasy, a VC-backed leader, empowers convenience retailers with actionable insights through advan...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Solutions Engineer

    Principal Data Solutions Engineer

    Arcana • Bengaluru, Republic Of India, IN
    Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Senior Data Engineer.Embark on a journey where your deep-rooted expertise in computer science fundamentals, a...Show more
    Last updated: 6 days ago • Promoted
    Principal Data Innovation Engineer

    Principal Data Innovation Engineer

    PriceEasy AI • Bengaluru, Republic Of India, IN
    Join one of the most respected and cutting-edge companies in the pricing and analytics industry! PriceEasy, a VC-backed leader, empowers convenience retailers with actionable insights through advan...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    ANSR • Bengaluru, Republic Of India, IN
    ANSR is hiring for one of its clients.Headquartered in Berlin, Germany, 4flow provides consulting, software and services for logistics and supply chain management. More than 1300 team members levera...Show more
    Last updated: 30+ days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    ACL Digital • Bengaluru, Republic Of India, IN
    We are looking for professionals with expertise in Data Engineering.Data Engineer - Looking for 6+years experience -Tech Lead – Data Engineer - looking for 10+ years of experience.If this role matc...Show more
    Last updated: 13 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    ScaleneWorks • Bengaluru, Karnataka, India
    Quick Apply
    Being aware of departmental guidance on best practices, processes, templates, clauses and playbooks.Show more
    Last updated: 30+ days ago
    Principal Data Engineer

    Principal Data Engineer

    Dexian • Bengaluru, Republic Of India, IN
    Analyze existing HR applications and workflows related to scheduling, shift management, and location tracking.Identify process gaps, inefficiencies, and automation opportunities.Rewrite and optimiz...Show more
    Last updated: 1 day ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    CodeMyMobile • Bangalore, IN
    Experience Required - 7 to 10 Years.Are you a Data Engineer who cares about clean engineering, autonomy, and solving real data challenges? If this sounds like you, we’d love to connect!.Email your ...Show more
    Last updated: 24 days ago • Promoted
    Principal Data Platform Engineer

    Principal Data Platform Engineer

    Wayfair • Bengaluru, Republic Of India, IN
    Staff Engineer - Data Platforms- Software Engineering.Wayfair is seeking a passionate and driven Staff Engineer to join our. In this pivotal role, you will be instrumental in shaping and executing W...Show more
    Last updated: 15 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    Tata Consultancy Services • Bengaluru, Karnataka, India
    Skill : Principal Data Engineer.Possesses hands-on technical experience working in AWS.The individual should possess a robust set of technical and soft skills and be an excellent AWS Data Engineer ...Show more
    Last updated: 11 days ago • Promoted
    Principal Data Engineer

    Principal Data Engineer

    Withum • Bengaluru, Republic Of India, IN
    Withum is a place where talent thrives - where who you are matters.It’s a place of endless opportunities for growth.A place where entrepreneurial energy plus inclusive teamwork equals exponential r...Show more
    Last updated: 30+ days ago • Promoted