Talent.com
This job offer is not available in your country.
Tech Lead - Databricks Job

Tech Lead - Databricks Job

YASH TechnologiesIndore, India
1 day ago
Job description

Job Description

Platform Design & Architecture

  • Lead the design and deployment of large-scale data solutions on Databricks.
  • Establish best practices for notebook development, job orchestration, and cluster management.
  • Stay updated with the latest Databricks features and implement improvements.

Data Engineering & Pipeline Development

  • Architect and develop ETL / ELT pipelines using PySpark, Spark SQL, and Delta Lake.
  • Optimize data workflows for performance, scalability, and cost-efficiency.
  • Integrate structured and unstructured data from various sources.

    • Cloud Integration

  • Leverage cloud services (AWS S3, Lambda, Glue, etc.) for data processing.
  • Ensure secure and compliant integration between Databricks and cloud platforms.

    • Data Governance & Security

  • Implement data governance policies using Unity Catalog or similar tools.
  • Ensure data lineage, cataloging, and access control compliance.

    • DevOps & Automation

  • Collaborate with DevOps teams to automate deployments using CI / CD pipelines (Azure DevOps, Git).
  • Manage infrastructure as code and monitor workloads for performance bottlenecks.

    • Team Collaboration

  • Mentor junior developers and conduct code reviews.
  • Work closely with data scientists, analysts, and business stakeholders.
  • Communicate technical concepts effectively to both technical and non-technical audiences.

    • Required Technical / Functional Competencies

    Domain / Industry Knowledge :

  • Basic knowledge of customer's business processes- relevant technology platform or product.
  • Able to prepare process maps, workflows, business cases and simple business models in line with customer requirements with assistance from SME and apply industry standards / practices in implementation with guidance from experienced team members.

    • Requirement Gathering and Analysis :

  • Working knowledge of requirement management processes and requirement analysis processes, tools & methodologies.
  • Able to analyse the impact of change requested / enhancement / defect fix and identify dependencies or interrelationships among requirements & transition requirements for engagement.

    • Product / Technology Knowledge :

  • Working knowledge of technology product / platform standards and specifications.
  • Able to implement code or configure / customize products and provide inputs in design and architecture adhering to industry standards / practices in implementation.
  • Analyze various frameworks / tools, review the code and provide feedback on improvement opportunities.

    • Architecture tools and frameworks :

  • Working knowledge of architecture Industry tools & frameworks.
  • Able to identify pros / cons of available tools & frameworks in market and use those as per Customer requirement and explore new tools / framework for implementation.

    • Architecture concepts and principles :

  • Working knowledge of architectural elements, SDLC, methodologies.
  • Able to provides architectural design / documentation at an application or function capability level and implement architectural patterns in solution & engagements and communicates architecture direction to the business.

    • Analytics Solution Design :

  • Knowledge of statistical & machine learning techniques like classification, linear regression modelling, clustering & decision trees.
  • Able to identify the cause of errors and their potential solutions.

    • Tools & Platform Knowledge :

  • Familiar with wide range of mainstream commercial & open-source data science / analytics software tools, their constraints, advantages, disadvantages, and areas of application.

    • Required Behavioral Competencies

    Accountability :

  • Takes responsibility for and ensures accuracy of own work, as well as the work and deadlines of the team.

    • Collaboration :

  • Shares information within team, participates in team activities, asks questions to understand other points of view.

    • Agility :

  • Demonstrates readiness for change, asking questions and determining how changes could impact own work.

    • Customer Focus :

  • Identifies trends and patterns emerging from customer preferences and works towards customizing / refining existing services to exceed customer needs and expectations.

    • Communication :

  • Targets communications for the appropriate audience, clearly articulating and presenting his / her position or decision.

    • Drives Results :

  • Sets realistic stretch goals for self & others to achieve and exceed defined goals / targets.

    • Resolves Conflict :

  • Displays sensitivity in interactions and strives to understand others’ views and concerns.

    • Certifications

    Mandatory At YASH, you are empowered to create a career that will take you to where you want to go while working in an inclusive team environment. We leverage career-oriented skilling models and optimize our collective intelligence aided with technology for continuous learning, unlearning, and relearning at a rapid pace and scale. Our Hyperlearning workplace is grounded upon four principles

  • Flexible work arrangements, Free spirit, and emotional positivity
  • Agile self-determination, trust, transparency, and open collaboration
  • All Support needed for the realization of business goals,
  • Stable employment with a great atmosphere and ethical corporate culture
    • Create a job alert for this search

    Tech Lead • Indore, India