Talent.com
This job offer is not available in your country.
Databricks Developer - Data Engineering

Databricks Developer - Data Engineering

SMARTWORK IT SERVICESAhmedabad
30+ days ago
Job description

Databricks SQL Engineer (Pharma Domain) - Contract (6 Months)

Location : Mumbai / Pune / Bangalore / Chennai / Ahmedabad / Noida, India (Chennai Most Preferred)

Experience : 5-10 Years

Shift Timings : Day shift IST, starting from 12 : 00 PM IST

Work Mode : Contract (Fixed Term Contract)

Duration : 6 Months

Job Summary :

We are seeking an experienced and highly skilled Databricks SQL Engineer with a strong understanding of the Pharmaceutical or Life Sciences domain to join our offshore Data Engineering team on a 6-month contract. In this role, you will be pivotal in designing, building, and optimizing efficient, scalable SQL-based data models and pipelines within the Databricks ecosystem, leveraging Databricks SQL, Spark SQL, and Delta Lake. You will be responsible for transforming raw data into valuable analytical insights, directly supporting critical decision-making processes across various pharma-related business functions. Your domain expertise will be crucial in understanding and addressing the specific data challenges and requirements of the pharmaceutical industry.

Responsibilities :

  • Data Modeling & Schema Design : Design and implement efficient and scalable data models within Databricks SQL and Delta Lake, optimized for analytical workloads specific to the pharmaceutical domain (e.g., clinical trial data, drug development data, sales & marketing data, supply chain data, regulatory data).
  • ETL / ELT Pipeline Development : Develop and maintain robust ETL / ELT pipelines using Databricks SQL and Spark SQL to ingest, transform, cleanse, and load data from various source systems relevant to the pharma industry (e.g., transactional databases, data lakes, external data providers, APIs).
  • Performance Optimization : Optimize Databricks SQL queries and Spark SQL jobs for performance and scalability to ensure efficient data processing and query execution on large datasets.
  • Data Quality & Governance : Implement data quality checks and validation rules within the data pipelines to ensure the accuracy, consistency, and reliability of data, adhering to pharma-specific data governance standards.
  • Delta Lake Implementation & Management : Leverage the features of Delta Lake for data reliability, versioning, and ACID properties within the Databricks environment.
  • Collaboration with Data Scientists & Analysts : Collaborate closely with Data Scientists and Business Analysts to understand their analytical requirements and translate them into efficient SQL-based data solutions.
  • Pharma Domain Expertise : Apply your understanding of pharmaceutical or life sciences business processes, data nuances, and regulatory requirements to design relevant and insightful data models and pipelines.
  • Documentation : Create and maintain clear and comprehensive technical documentation for data models, ETL / ELT pipelines, and data transformations.
  • Troubleshooting & Support : Identify, troubleshoot, and resolve data-related issues and performance bottlenecks within the Databricks environment.
  • Adherence to Best Practices : Follow data engineering best practices for coding standards, data governance, security, and performance.
  • Integration with Downstream Systems : Ensure seamless integration of processed data with downstream analytical tools and reporting platforms used within the pharma organization.

Technical Skills & Qualifications :

  • 5-10 years of relevant experience in Data Engineering.
  • Strong proficiency in Databricks SQL and Spark SQL for data manipulation, transformation, and querying.
  • Hands-on experience with Delta Lake and its features (ACID transactions, time travel, schema evolution).
  • Solid understanding of data warehousing concepts, dimensional modeling, and schema design principles.
  • Experience in building and optimizing ETL / ELT pipelines for large-scale data processing.
  • Strong SQL skills with experience in writing complex queries and performance tuning.
  • Proven experience working with large datasets and distributed computing environments (Spark).
  • Understanding of data quality principles and implementation of data validation processes.
  • Excellent analytical and problem-solving skills.
  • Strong communication (written and verbal) and collaboration skills.
  • Willingness to work in the Day shift IST, starting from 12 : 00 PM IST.
  • Preferred Skills :

  • Strong understanding of the Pharmaceutical or Life Sciences domain, including familiarity with relevant data types (clinical trial data, drug discovery data, sales data, supply chain data).
  • Experience with other data engineering tools and technologies within the Azure Databricks ecosystem (Databricks Workflows, Databricks Delta Live Tables).
  • Experience with data integration tools beyond SQL.
  • Familiarity with cloud data warehousing solutions (Azure Synapse Analytics, Snowflake).
  • Experience with scripting languages like Python for data engineering tasks.
  • Knowledge of data governance frameworks and tools.
  • Experience with visualization tools (Power BI, Tableau) and their integration with Databricks.
  • Understanding of data security and compliance requirements within the pharmaceutical industry.
  • ref : hirist.tech)

    Create a job alert for this search

    Data Engineering • Ahmedabad

    Related jobs
    • Promoted
    Manager - Data Engineering

    Manager - Data Engineering

    Core Minds Tech SOlutionsAhmedabad
    Job Description : In this role, you will : - Manage and develop team of data analysts with low to moderate complexity...Show moreLast updated: 30+ days ago
    Postgres Developer (Offshore, India only)

    Postgres Developer (Offshore, India only)

    DATAMAXISIndia
    The developer will be responsible for developing in Postgres platform hosted in Azure as well as in Azure Databricks.Good data Engineering, data modeling, SQL knowledge is a must with Postgres and ...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer (Databricks)

    Data Engineer (Databricks)

    Marx Enterprise Transformation ArchitectureAhmedabad, IN
    Azure Data Engineer – Databricks & Data Fabric.Data Engineering, 3+ years on Azure stack.The ideal candidate will be responsible for building modern data pipelines, optimizing data flows, and enabl...Show moreLast updated: 10 days ago
    • Promoted
    Data Architect

    Data Architect

    BL ConsultantsAhmedabad, IN
    Currently we have an open position with our client - Its a IT Consulting Firm.We're seeking a skilled Databricks Architect with a strong background in data engineering and AI / ML.Your expertise will...Show moreLast updated: 1 day ago
    • Promoted
    Kanerika Software - Data Engineer - Azure Data Factory / Databricks

    Kanerika Software - Data Engineer - Azure Data Factory / Databricks

    KANERIKA SOFTWARE PRIVATE LIMITEDAhmedabad
    What You Will Do : Following are high level responsibilities that you will play but not limited to : - Develop and maintain data pipelines using Azure Data...Show moreLast updated: 28 days ago
    • Promoted
    Infrastructure Consultant

    Infrastructure Consultant

    BounteousAhmedabad, IN
    Senior Cloud Infrastructure Engineer for Data Platform.JD for DevOps requirement in Data Lake team.Job Title : Senior Cloud Infrastructure Engineer for Data Platform- Periscope.The ideal candidate ...Show moreLast updated: 1 day ago
    Systems Integration Senior Specialist

    Systems Integration Senior Specialist

    NTT Data Vertex Software Inc.INDIA
    Systems Integration Senior Specialist.Diverse Data Engineering activities on Azure Data platform : .Build & Deliver Data pipeline connecting various enterprise data sources both RDBMS, NoSQL & APIs.D...Show moreLast updated: 30+ days ago
    • Promoted
    Azure Data Engineer

    Azure Data Engineer

    RandomTreesAhmedabad, IN
    We are seeking a skilled and experienced.The ideal candidate will have a strong background in building and optimizing data pipelines and data sets, utilizing Azure Data Factory, Databricks, PySpark...Show moreLast updated: 25 days ago
    Senior Data Engineer - Databricks

    Senior Data Engineer - Databricks

    SRIJAN TECHNOLOGIESINDIA
    See all the jobs at Srijan Technologies PVT LTD here : .Senior Data Engineer - Databricks.We turn customer challenges into growth opportunities. Material is a global strategy partner to the worlds mos...Show moreLast updated: 30+ days ago
    Sr. Data Analytics Engineer

    Sr. Data Analytics Engineer

    Ajmera Infotech Inc.Ahmedabad, Gujarat, India
    Data Analytics Engineer Power mission-critical decisions with governed insights.NYSE-listed clients driving decisions that can t afford to fail. Our 120-engineer team specializes in highly reg...Show moreLast updated: 3 days ago
    • Promoted
    Databricks SQL Engineer

    Databricks SQL Engineer

    UPNOTCH CONSULTINNGAhmedabad
    Position : Databricks SQL Engineer Location : Mumbai / Pune / Chennai / Overview / Job Summary : We are loo...Show moreLast updated: 30+ days ago
    • Promoted
    Data Analytics and Business Intelligence Engineer (Databricks Specialist)

    Data Analytics and Business Intelligence Engineer (Databricks Specialist)

    Vriba SolutionsAhmedabad, IN
    Job Title : Data Analytics and Business Intelligence Engineer (Databricks Specialist).We are seeking a highly skilled and analytical. Data Analytics and Business Intelligence (BI) Engineer.The ideal ...Show moreLast updated: 17 days ago
    Sr. Product Manager, Developer Ecosystem

    Sr. Product Manager, Developer Ecosystem

    DatabricksINDIA
    You will be the Product Manager in the Developer Ecosystem team at Databricks in Amsterdam.You will be responsible for enhancing the customer developer experience across the inner and outer develop...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Snowflake Developer & Infrastructure Integration Expert

    Senior Snowflake Developer & Infrastructure Integration Expert

    iVedha Inc.Ahmedabad, IN
    We're Hiring : Senior Snowflake Developer & Infrastructure Integration Expert (Remote).We’re looking for a seasoned Snowflake Developer and Infrastructure Expert to take charge of full lifecycle imp...Show moreLast updated: 30+ days ago
    • Promoted
    Cloud Administrator

    Cloud Administrator

    Numeric TechnologiesAhmedabad, IN
    We are seeking an experienced Databricks Administrator with 7+ years of expertise in managing and optimizing Databricks environments. The ideal candidate should have hands-on experience with Azure / A...Show moreLast updated: 10 days ago
    • Promoted
    Data Engineering Lead

    Data Engineering Lead

    Techs To SuitAhmedabad
    We are looking for Candidates currently in Hyderabad, Indore or Ahmedabad only not the ones who are willing to relocate Job Summary : < / p...Show moreLast updated: 10 days ago
    • Promoted
    Data Engineer - AWS - Databricks, AWS EMR, Redshift, and various database management systems - 5+ Relevant Years - Immediate Joiners

    Data Engineer - AWS - Databricks, AWS EMR, Redshift, and various database management systems - 5+ Relevant Years - Immediate Joiners

    Blue Cloud Softech Solutions LimitedAhmedabad, IN
    Data Engineering, including more than 5 years of hands-on expertise with Databricks, AWS EMR, Redshift, and various database management systems. Advanced SQL RDBMS design and query building skills (...Show moreLast updated: 1 day ago
    • Promoted
    Data Engineer

    Data Engineer

    CozzeraAhmedabad, IN
    We are seeking a highly skilled and experienced.The ideal candidate should have.Design, develop, and maintain robust, scalable, and efficient data pipelines and ETL / ELT processes.Integrate data fro...Show moreLast updated: 1 day ago