Full Stack Developer

Job Role :

We are seeking a skilled and passionate Data Engineer to join our team. In this role, you will be responsible for designing, implementing, and optimizing data pipelines to extract, transform, and load (ETL) large datasets across a range of systems and platforms. You will be working with cutting-edge tools like Databricks and Azure Data Factory (ADF), leveraging your expertise in Python, SQL, and ETL processes to ensure the scalability and reliability of our data infrastructure.

Roles & Responsibilities :

• ETL Pipeline Development : Build and maintain scalable and efficient ETL pipelines to process large datasets using Azure Data Factory (ADF) and Databricks.
• Data Integration : Integrate data from multiple sources, ensuring data accuracy, consistency, and availability for downstream analytics and reporting.
• Python Scripting : Write and optimize Python code to automate ETL workflows and data transformations.
• SQL Database Management : Design and query SQL databases for data extraction, transformation, and storage.
• Collaboration : Work closely with data scientists, analysts, and other engineering teams to meet data requirements and optimize pipeline performance.
• Performance Tuning : Continuously monitor and optimize data pipelines for performance, ensuring that they scale as needed.
• Documentation : Create and maintain detailed documentation for data pipelines, architecture, and processes.

Experience Required :

Skills Required

Database Management, Sql Database, Data Extraction, Automation, Python, Analytics