Talent.com
This job offer is not available in your country.
Technical Architect

Technical Architect

ConfidentialPune
30+ days ago
Job description

The Senior Tech Lead - Databricks leads the design, development, and implementation of advanced data solutions. Has To have extensive experience in Databricks, cloud platforms, and data engineering, with a proven ability to lead teams and deliver complex projects.

Responsibilities :

  • Lead the design and implementation of Databricks-based data solutions.
  • Architect and optimize data pipelines for batch and streaming data.
  • Provide technical leadership and mentorship to a team of data engineers.
  • Collaborate with stakeholders to define project requirements and deliverables.
  • Ensure best practices in data security, governance, and compliance.
  • Troubleshoot and resolve complex technical issues in Databricks environments.
  • Stay updated on the latest Databricks features and industry trends.

Key Technical Skills & Responsibilities

  • Experience in data engineering using Databricks or Apache Spark-based platforms.
  • Proven track record of building and optimizing ETL / ELT pipelines for batch and streaming data ingestion.
  • Hands-on experience with Azure services such as Azure Data Factory, Azure Data Lake Storage, Azure Databricks, Azure Synapse Analytics, or Azure SQL Data Warehouse.
  • Proficiency in programming languages such as Python, Scala, SQL for data processing and transformation.
  • Expertise in Spark (PySpark, Spark SQL, or Scala) and Databricks notebooks for large-scale data processing.
  • Familiarity with Delta Lake, Delta Live Tables, and medallion architecture for data lakehouse implementations.
  • Experience with orchestration tools like Azure Data Factory or Databricks Jobs for scheduling and automation.
  • Design and implement the Azure key vault and scoped credentials.
  • Knowledge of Git for source control and CI / CD integration for Databricks workflows, cost optimization, performance tuning.
  • Familiarity with Unity Catalog, RBAC, or enterprise-level Databricks setups.
  • Ability to create reusable components, templates, and documentation to standardize data engineering workflows is a plus.
  • Ability to define best practices, support multiple projects, and sometimes mentor junior engineers is a plus.
  • Must have experience of working with streaming data sources and Kafka (preferred)

    • Eligibility Criteria :

  • Bachelor s degree in Computer Science, Data Engineering, or a related field
  • Extensive experience with Databricks, Delta Lake, PySpark, and SQL
  • Databricks certification (e.g., Certified Data Engineer Professional)
  • Experience with machine learning and AI integration in Databricks
  • Strong understanding of cloud platforms (AWS, Azure, or GCP)
  • Proven leadership experience in managing technical teams
  • Excellent problem-solving and communication skills

    • Our Offering

  • Global cutting-edge IT projects that shape the future of digital and have a positive impact on environment.
  • Wellbeing programs & work-life balance - integration and passion sharing events.
  • Attractive Salary and Company Initiative Benefits
  • Courses and conferences
  • Attractive Salary
  • Hybrid work culture

    • Skills Required

    Data Security, Machine Learning, Databricks, Data Processing, Sql, Python

    Create a job alert for this search

    Technical Architect • Pune