This job offer is not available in your country.
Senior Data Platform Engineer - Azure Databricks
Antal InternationalBengaluru, Karnataka, India16 hours ago
Job descriptionLead the expansion and utilization of Databricks Unity Catalog for granular data governance, including row / column-level security, data lineage tracking, and metadata tagging. Conduct regular security and compliance checks to ensure the platform adheres to internal policies and industry standards. Build and maintain resilient CI / CD pipelines using GitHub Actions to automate infrastructure (Terraform) and application deployments. Implement and manage a comprehensive observability stack (logging, monitoring, tracing, alerting) to ensure high availability and proactive issue resolution. Apply FinOps principles to monitor, analyze, and optimize platform costs. Manage day-to-day platform administration, including onboarding projects, managing clusters, and configuring jobs. Develop and integrate user-friendly interfaces (e.g., via ServiceNow or Backstage) to enable self-service for access requests and resource provisioning. Create and maintain clear documentation, including architecture diagrams, runbooks, and best-practice guides. Coach and train other colleagues, championing a "platform-as-a-product" mindset focused on delivering value to internal users. Bachelor's or Master's degree in Computer Science, Mathematics, Engineering, or a related discipline. 5+ years of professional experience in a platform engineering, DevOps, or SRE role with a focus on data-intensive platforms. Deep expertise in Microsoft Azure, including ADLSGen2, Private Endpoints, Azure Key Vault, Azure Log Analytics, and EntraID. Extensive hands-on experience with Azure Databricks administration, operations, security, and performance tuning. Strong proficiency with Terraform and a solid understanding of GitOps principles. Advanced skills in shell scripting and experience with a scripting language like Python. Demonstrable experience building CI / CD pipelines with GitHub Actions. Familiarity with Databricks Asset Bundles is a plus. Proficient with Linux administration and troubleshooting. Practical experience implementing data governance solutions for data lakes, especially with Unity Catalog. Solid understanding of Kubernetes (k8s) concepts and architecture. Understanding of MLOps principles (e.g., feature stores, model deployment strategies) to support ML engineering teams. Experience integrating with or building plugins for platforms like Backstage or ServiceNow. Excellent communication and collaboration skills to work effectively with technical teams and business stakeholders. Strong analytical thinking, problem-solving abilities. Demonstrated high level of initiative, self-motivation, and a proactive, self-starter mindset, with a strong drive to independently identify and solve challenges. A focus on treating the platform as a product, prioritizing the user experience and delivering tangible value to internal customers. A passion for continuous learning, innovation, knowledge sharing, and driving excellence in data engineering. Ability to work effectively in a cross-functional, fast-paced environment.
Job Description
What we offer :
Our Client, believes in rewarding our employees for their hard work. We offer competitive salaries, company pensions and performance related benefits. Our people can also take advantage of our extensive flexible benefits package and more.
How we work :
Our people are key to our success. Our core objective is to provide them with a supportive and entrepreneurial work environment that fosters collaboration. This allows our people to take responsibility and make optimal use of their skills. Together, we want to shape the future of energy.
About Us
We're a leading European energy company committed to powering the energy transition with speed, innovation, and resilience. Our Platform Engineering team is pivotal, providing the secure, scalable infrastructure, platforms, and tooling that enable rapid, high-quality software delivery for both our Advanced Solution Delivery team and Trading Citizen Developers across the trading organization.
Position Summary
As the (Senior) Data Platform Engineer (Azure Databricks) , you will be responsible for building, maintaining, and evolving our Azure Databricks platform, ensuring it is secure, reliable, scalable, and cost-efficient. This role is pivotal in empowering our Data- & AIOps team by providing a robust, automated, and well-governed data environment. The ideal candidate will drive a "platform-as-a-product" mindset, continuously improving our architecture, operations, and user experience.
Your Responsibilities
Platform Architecture & Engineering :
- Design and implement architectural improvements, including workspace migrations and robust disaster recovery (DR) strategies.
- Enhance and maintain the platform's security posture, focusing on data governance, data and compute isolation at logical and physical levels.
- Drive the evolution of the platform by researching, prototyping, and integrating new tools and features to enhance performance and scalability.
Data Governance & Security :
Automation & Operations (DevOps & GitOps) :
User Enablement & Self-Service :
Your profile
Essential Qualifications :
Preferred Qualifications :
Soft Skills and Cultural Fit :
Check Your Resume for Match
Upload your resume and our tool will compare it to the requirements for this job like recruiters do.
Create a job alert for this search
Azure Data Engineer • Bengaluru, Karnataka, India
Related jobs
- Promoted
Senior Data Engineer
IGT Solutionshosur, tamil nadu, in Primary Skills : Advanced SQL Server, ETL development using SSIS, Azure Data Factory, and Databricks, Power BI, Data warehousing, Data quality and validation, General Microsoft technology stack.We a...Show moreLast updated: 28 days ago
- Promoted
Lead Azure Data Engineer
RandomTreesBengaluru, IN We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies.
If you're passionate about building r...Show moreLast updated: 29 days ago
- Promoted
Azure Data Engineer
Tata Consultancy Servicesbangalore, karnataka, in Job Title : Azure Data Engineer.Required Skillset : Azure Databricks(Strong),scala(strong), Pyspark(Good to have).Azure Databricks using Scala, PySpark.
Software Development Life Cycle experience.Work...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
EXLhosur, tamil nadu, in We are seeking a highly skilled.You will work closely with cross-functional teams to provide thought leadership deliver robust, secure, and high-performance data solutions that empower analytics an...Show moreLast updated: 29 days ago
- Promoted
Senior Data Engineer
Publicis ProductionBengaluru, Karnataka, India We are seeking a proactive and self-motivated Senior Data Engineer with a proven track record in building scalable cloud-based data solutions across multiple cloud platforms to support our work in ...Show moreLast updated: 24 days ago
- Promoted
Senior Data Engineer
Otomeyt AIhosur, tamil nadu, in We are seeking a highly skilled 7+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 21 days ago
- Promoted
Senior Data Engineer
Deltacubeshosur, tamil nadu, in Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect.
Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 18 days ago
- Promoted
Senior Generative AI Engineer (Databricks Data Lake)
Ampstekhosur, tamil nadu, in Title : Senior Generative AI Engineer (Databricks Data Lake).We are seeking an experienced Senior Generative AI Engineer with a strong background in Databricks and data lake architectures.This indiv...Show moreLast updated: 3 days ago
- Promoted
Senior Data Engineer
Prudent Technologies and Consulting, Inc.hosur, tamil nadu, in Snowflake Instance Management : Set up, configure, and manage between three to four new Snowflake data warehouse instances per quarter, tailored to support emerging products and analytics requiremen...Show moreLast updated: 30+ days ago
- Promoted
Senior Cloud Engineer AWS
Matrix USAhosur, tamil nadu, in We are seeking an experienced AWS Developer proficient in Python and PySpark to design, develop, and maintain scalable, serverless data processing and workflow automation solutions on AWS.The ideal...Show moreLast updated: 8 days ago
- Promoted
Data Platform Engineer
Intuitive.Cloudhosur, tamil nadu, in With the reputation of being a.Digital Transformation challenges across following Intuitive Superpowers : .Application & Database Modernization.
Platform Engineering (IaC / EaC, DevSecOps & SRE).Cloud N...Show moreLast updated: 8 days ago
- Promoted
Senior Data Engineer
SAIVA AIhosur, tamil nadu, in We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 11 days ago
- Promoted
Senior Data Engineer
MastekBengaluru, Karnataka, India Role : Data Engineer – Databricks.We are seeking a highly skilled Data Engineer to design, develop, and maintain data pipelines that extract data from Oracle Symphony via APIs, process and store it...Show moreLast updated: 22 days ago
- Promoted
Senior Data Engineer
JRD SystemsBengaluru, Karnataka, India This role is critical to both the development of new data products and the modernization of existing platforms.The ideal candidate is a seasoned data engineer with hands-on experience designing, bu...Show moreLast updated: 23 days ago
- Promoted
Senior Data Engineer
Evolenthosur, tamil nadu, in Lead the design, development, and deployment of complex data solutions, ensuring scalability and maintainability.Architect and implement high-performance data pipelines and data warehousing solutio...Show moreLast updated: 26 days ago
- Promoted
Senior Data Engineer
CEShosur, tamil nadu, in As part of our data engineering team, you’ll build and maintain.This is a hands-on role where you’ll design and optimize modern data infrastructure, ensuring reliability, scalability, and performan...Show moreLast updated: 8 days ago
- Promoted
Senior Azure Data Engineer
Programmers.iohosur, tamil nadu, in Job Title : Senior Azure Data Engineer.Skills Required : Azure Synapse, Azure Data Factory, Azure Databricks, and ADLS Gen2, PySpark, SQL,Microsoft Fabrics.
We are seeking a highly skilled and experie...Show moreLast updated: 30+ days ago
- Promoted
Senior AWS Data Engineer
TEKsystemsbangalore, karnataka, in Your skills & Your Experience Experience.Analytics and Data Warehousing.Experience in building end-to-end solutions with the Big data platform, Spark or scala programming.Solid experience in ETL pi...Show moreLast updated: 9 days ago
- Promoted
Lead Data Engineer
Eucloid Data Solutionshosur, tamil nadu, in Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 11 days ago
- Promoted
Senior Data Engineer
BrightEdgehosur, tamil nadu, in BrightEdge About Us BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results.
Our platform processes massive amounts of da...Show moreLast updated: 29 days ago