Talent.com
This job offer is not available in your country.
Azure Databricks Engineer - Python / Spark

Azure Databricks Engineer - Python / Spark

ConfidentialPune
30+ days ago
Job description

Key Responsibilities :

  • Design and implement scalable data pipelines using Azure Data Bricks.
  • Develop ETL processes to efficiently extract, transform, and load data.
  • Collaborate with data scientists and analysts to define and refine data requirements.
  • Optimize Spark jobs for performance and efficiency.
  • Monitor and troubleshoot production workflows and jobs.
  • Implement data quality checks and validation processes.
  • Create and maintain technical documentation related to data architecture.
  • Conduct code reviews to ensure best practices are followed.
  • Work on integrating data from various sources including databases, APIs, and third-party services.
  • Utilize SQL and Python for data manipulation and analysis.
  • Collaborate with DevOps teams to deploy and maintain data solutions.
  • Stay updated with the latest trends and updates in Azure Data Bricks and related technologies.
  • Facilitate data visualization initiatives for better data-driven insights.
  • Provide training and support to team members on data tools and practices.
  • Participate in cross-functional projects to enhance data sharing and access.

Qualifications :

Essential Qualifications :

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • Minimum of 6 years of experience in data engineering or a related domain.
  • Strong expertise in Azure Data Bricks and data lake concepts.
  • Proficiency with SQL, Python, and Spark.
  • Solid understanding of data warehousing concepts.
  • Experience with ETL tools and frameworks.
  • Familiarity with cloud platforms such as Azure, AWS, or Google Cloud.
  • Excellent problem-solving and analytical skills.
  • Ability to work collaboratively in a diverse team environment.
  • Experience with data visualization tools such as Power BI or Tableau.
  • Strong communication skills with the ability to convey technical concepts to non-technical stakeholders.
  • Knowledge of data governance and data quality best practices.
  • Hands-on experience with big data technologies and frameworks.

    • Desirable Qualifications :

  • A relevant certification in Azure is a plus.
  • Ability to adapt to changing technologies and evolving business requirements.

    • Skills Required

    Azure Data Bricks, Sql, Python, Spark, Data Warehousing, Data Integration

    Create a job alert for this search

    Azure Engineer • Pune