Talent.com
This job offer is not available in your country.
Data Engineer - Python / Apache Spark

Data Engineer - Python / Apache Spark

BOHIYAANAM TECHNOLOGY LLPHyderabad
12 days ago
Job description

Description :

Key Responsibilities :

  • Design, develop, and deploy end-to-end data engineering solutions using Databricks, Apache Spark, PySpark, Python, and SQL.
  • Build scalable and efficient ETL / ELT pipelines for data ingestion, transformation, and integration from various sources.
  • Work with data warehousing solutions and ensure high-performance data processing and storage.
  • Engage directly with clients to gather requirements, propose solutions, and manage delivery expectations.
  • Collaborate with data scientists, analysts, and other engineers to support ML model deployment, MLOps, and advanced analytics use cases.
  • Implement data modeling, data architecture, and adhere to best practices in modern data platform design.
  • Utilize CI / CD tools such as GitHub, Azure DevOps, and Pipelines for version control, testing, and production deployment.
  • Manage technical project delivery, including scope, timelines, and technical documentation.
  • Troubleshoot performance issues and optimize distributed computing processes using Apache Spark.
  • Continuously learn and apply new features and tools to enhance existing solutions and architectures.

Required Skills & Experience :

  • 4+ years of experience in data engineering, data platforms, and data analytics.
  • 3+ years hands-on experience with Databricks, Apache Spark, PySpark, Python, and SQL.
  • Experience working in client-facing roles or consulting environments.
  • Strong understanding of data modeling, ETL processes, and cloud data architecture principles.
  • Working knowledge of at least two cloud platforms (AWS, Azure, GCP), with deep expertise in one.
  • Solid knowledge of Spark runtime internals and distributed computing.
  • Experience with CI / CD and deployment pipelines GitHub, Azure DevOps, etc.
  • Familiarity with MLOps concepts and production model support.

    • Preferred Qualifications :

  • Databricks Data Engineer Associate or Professional Certification (preferred or must be willing to obtain).
  • Experience with whiteboarding, architecture documentation, and delivering technical presentations.
  • Ability to manage technical conflict, communicate effectively, and handle multiple client engagements.

    • (ref : hirist.tech)

    Create a job alert for this search

    Data Engineer • Hyderabad