Azure Big Data Engineer

ConfidentialBengaluru / Bangalore

30+ days ago

Job description

Mandatory Skills : Azure Skills Like

Azure Databricks
Azure Synapse
Azure Functions
Azure Data Lake
Azure Integration Services
Azure API Management
CI / CD and PowerShell Scripts
ADF
ETL Pipelines
Unity Catalog
Azure SQL Database
SQL Coding
Storage Account, Azure Data Explorer, Spark Structured Streaming, ADF, Netezza SQL queries, Data lineage, Spark jobs.

Role & Responsibilities / Job Description :

Overall 4 to 8 years of experience with a minimum 4+ years of relevant professional work experience.

Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure.

Work together with data analysts to understand the needs for data and create effective data workflows.

Create and maintain data storage solutions including Azure Databricks , Azure Functions, Azure Data Lake, Azure Synapse , Azure Integration Services, Azure API Management, CI / CD and PowerShell Scripts. ADF, ETL Pipelines, Unity Catalog, Azure SQL Database, SQL Coding, Azure SQL Database, Azure Data Lake.

Utilizing Azure Data Factory to create and maintain ETL (Extract, Transform, Load) operations using ADF pipelines.

Hands-On Experience working on Data Bricks for implementing Transformations and Delta Lake.

Hands-On Experience working on Serverless SQL Pool, Dedicated SQL Pool.

Use ADF pipelines to orchestrate the end to end data transformation including the execution of DataBricks notebooks.

Should have experience working on Medallion Architecture.

Experience working on CI / CD pipelines using Azure DevOps.

Attaching both ADF and ADB to DevOps.

Creating and managing Azure infrastructure across the landscape using Bicep.

Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.

Improve the scalability, efficiency, and cost-effectiveness of data pipelines.

Monitoring and resolving data pipeline problems will guarantee consistency and availability of the data.

Good to have exposure on Power BI, Power Automate.

Task description :

We are looking for a Data Engineer to join our growing team of analytics experts.

The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams.

The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.

Should have good hands on implementing Incremental / Delta loads.

Develop and Support activities of development in Projects and Automation solutions for customers

Deliverables Expected

The following shall be the deliverables as per project documented quality process :

Updated requirements document / Detailed Bug Analysis report

Concept document (wherever applicable)

Design document (for all features)

Test specification document

Source code

Review report

Test report

Filled review checklists

Traceability matrix

User manuals and related artifacts

Filled pre-delivery checklist (PDC)

Release Note & release mails

Skills Required

Adf

Create a job alert for this search

Azure Data Engineer • Bengaluru / Bangalore

Related jobs

Promoted

Azure Data Engineer

Tata Consultancy ServicesGreater Bengaluru Area, India

TCS has always been in the spotlight for being adept in the next big technologies.What we can offer you is a space to explore varied technologies and quench your techie soul.Required Skill : Azure D...Show moreLast updated: 30+ days ago

Promoted

Azure Data Engineer-Technical Lead

SigmoidBengaluru, Karnataka, India

Kindly find the Job Description Below.Job Title : Technical Lead-Azure Data Engineer.Years of Experience : 10+ years of experience. We are looking for a detailed oriented self-starter to assist our eng...Show moreLast updated: 1 day ago

Promoted

Data Engineer

Networth CorpBengaluru, Karnataka, India

Working along with architects for planning, designing, integrating, and implementing complex solutions in Azure Cloud computing services. End to end development lifecycle of complex large-scale and ...Show moreLast updated: 30+ days ago

Promoted

Data engineer - AWS - Inperson drive

Live ConnectionsGreater Bengaluru Area, India

Show moreLast updated: 1 day ago

Promoted

Data Bricks Engineer

Relevance LabGreater Bengaluru Area, India

SQL skills and hands-on experience using.AI-assisted development tools).The ideal candidate will be responsible for designing, developing, and optimizing data pipelines, supporting analytics worklo...Show moreLast updated: 1 day ago

Promoted

Data Engineer

VIDA Digital IdentityGreater Bengaluru Area, India

At VIDA we’re building the future of digital identity.As a Data Engineer you’ll work across the stack—from platform and infrastructure to data pipelines and end-user tooling.You’ll help us moderniz...Show moreLast updated: 1 day ago

Promoted

Lead Azure Data Engineer

IQVIABengaluru, Karnataka, India

Data Engineering, Python, Azure Databricks, Database, People management experience.Manages a team of professional-level employees. Develops and communicates plans and priorities to meet team perform...Show moreLast updated: 11 days ago

Promoted

Azure Data Engineer

Capgemini InventBengaluru, Karnataka, India

At Capgemini Invent, we believe difference drives change.As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to...Show moreLast updated: 1 day ago

Promoted

Senior Data Engineer -Azure Databricks

Tredence Inc.Bengaluru, Karnataka, India

Loc : Bangalore / Chennai / Pune / Kolkata / Gurugram.Azure Databricks, Pyspark, SQL.Bachelor's and / or masters degree in computer science or equivalent experience. IT experience and experience in Data wareho...Show moreLast updated: 11 days ago

Promoted

AWS Data Engineer

Tata Consultancy Serviceshosur, tamil nadu, in

TCS is Hiring AWS Data Engineer Bangalore location.Strong hands-on experience in Python programming and PySpark.Experience using AWS services (RedShift, Glue, EMR, S3 & Lambda).Experience working w...Show moreLast updated: 27 days ago

Promoted

Senior Data Engineer

Live Connectionshosur, tamil nadu, in

Hiring : Data Engineer | 9-12 Years Experience | Remote (India).Python, SQL, Big Data, Cloud, Snowflake.We are looking for an experienced. The ideal candidate will have deep experience in.Python, SQL...Show moreLast updated: 30+ days ago

Promoted

Azure Data Engineer

Rainier Softech Solutions Pvt LtdBengaluru, Karnataka, India

We’re looking for a hands-on Data Engineer to build reliable, scalable data pipelines on Databricks.You’ll turn requirements into production-grade ELT / ETL jobs, Delta Lake tables, and reusable comp...Show moreLast updated: 4 days ago

Promoted

Senior Azure Data Engineer

Tata Consultancy ServicesGreater Bengaluru Area, India

Job Title : Senior Azure Data Engineer.Required Skillset : Azure data factory, data bricks, data lake, automation and performance optimization of ETL. Good Knowledge of Data Brick lakehouse and Azure ...Show moreLast updated: 30+ days ago

Promoted
New!

Data Engineer

Tata Consultancy ServicesGreater Bengaluru Area, India

Previous experience as a data engineer or in a similar role.Must have experience with MS Azure services such as Data Lake Storage, Data Factory, Databricks, HDInsight, Azure SQL Database, Azure Syn...Show moreLast updated: 22 hours ago

Promoted

(Senior) Azure Data Engineer - Full remote - contractor in USD

All European CareersBangalore, IN

Remote

For an international project in Chennai, we are urgently looking for a Full Remote Senior Azure Data Engineer, who will build data pipeline for enterprise search applications using ADF and Databric...Show moreLast updated: 4 days ago

Promoted

Senior Data Engineer

ArcanaGreater Bengaluru Area, India

Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Senior Data Engineer.Embark on a journey where your deep-rooted expertise in computer science fundamentals, a...Show moreLast updated: 1 day ago

Promoted

AWS Data Engineer

COVET IT INChosur, tamil nadu, in

Please go through the below requirements and let me know your interest and forward your resume along with your contact information to raja@covetitinc. We are seeking a highly skilled AWS Data Engine...Show moreLast updated: 1 day ago

Promoted

Azure Data Engineer

LTIMindtreeBengaluru, Karnataka, India

Role Senior Data Engineer 8 years of experience.Build reusable utilities templates and automation pipelines.Design scalable data engineering frameworks standards and best practices.Provide architec...Show moreLast updated: 30+ days ago