This job offer is not available in your country.

Azure Databricks Engineer - Spark / Python

Armour Corporate ServicesHyderabad

30+ days ago

Job description

Job Overview :

Position Title : Databricks Engineer

Location : Noida / Hyderabad

P&L Responsibility : No

Key Stakeholders : Data Scientists; Business Analysts; Machine Learning Engineers; Project Sponsors

Reports To : Director of Data Engineering

Direct Reports : None (Individual Contributor)

Nature of Work : Hybrid design, develop and optimise scalable big data solutions on the Databricks Unified Analytics Platform, enabling advanced AI / ML workflows

About the Client :

This is a high-growth data & AI solutions firm partnering with mid-market and large enterprises in finance, healthcare and retail. With a lean, agile team of data engineers and machine learning specialists, they deliver end-to-end analytics - from data ingestion and lakehouse architecture to production-grade AI deployments. A strong focus on continuous upskilling, cross-functional innovation workshops and mentorship drives a culture of growth and thought leadership.

Why Join This Organisation :

Lead the design of enterprise-scale data lakehouse and AI / ML solutions for marquee clients
Thrive in a lean, agile environment that prioritises innovation, learning and collaboration
Shape the next generation of predictive analytics and streaming data pipelines
Accelerate your career through mentorship programs and exposure to cutting-edge tooling

Position Overview :

As a Databricks Engineer, youll be the technical cornerstone for building high-performance, cost-efficient data architectures on Databricks. Collaborating with data scientists, ML engineers and business stakeholders, youll architect robust ETL workflows and real-time analytics solutions that drive strategic decision-making.

You will tackle challenges across distributed data processing, performance tuning and governance - promoting best practices in Apache Spark, security and compliance. Your contributions will ensure data integrity, reliability and scalability across large-scale enterprise environments.

Key Responsibilities :

Design, implement and optimize batch and streaming pipelines in Databricks

Develop scalable ETL workflows using Apache Spark for large datasets

Architect and tune Delta Lake-based data lakehouse solutions for performance and cost efficiency

Enforce data governance, security policies and compliance standards

Integrate advanced AI / ML models into production pipelines alongside data scientists

Automate deployments via CI / CD tools and infrastructure-as-code (e.g., Terraform)

Monitor and troubleshoot cluster performance, job failures and data quality issues

Education & Experience Requirements :

Bachelors or Masters in Computer Science, Data Engineering or related field

5+ years hands-on experience with Databricks and Apache Spark in enterprise settings

Proven expertise in SQL and Python or Scala for data processing

Experience with cloud platforms (AWS, Azure or GCP) and Delta Lake architecture

Familiarity with CI / CD frameworks, DevOps best practices and infrastructure-as-code

Solid understanding of data security, compliance and governance

Essential Skills & Competencies :

Technical / Functional : Advanced knowledge of Databricks, Apache Spark, Delta Lake, cloud data services

Leadership : Cross-functional collaboration, influencing architectural decisions, mentoring peers

Behavioural : Proactive problem-solver, detail-oriented, adaptable in fast-paced agile teams, strong communicator

(ref : hirist.tech)

Create a job alert for this search

Azure Engineer • Hyderabad