Talent.com
This job offer is not available in your country.
Data Engineering

Data Engineering

ScaleneWorksBengaluru, Karnataka, India
30+ days ago
Job type
  • Quick Apply
Job description

Job Title : Middleware Engineer

Position : Data Engineer

Experience : 5-6yrs

Category : IT Infrastructure

Main location : India, Karnataka, Bangalore

Employment Type : Full Time

Qualification : Bachelor's degree in Computer Science or related field or higher.

Roles and Responsibilities

Data Engineer - 5-6 years experience.

Responsibilities

Design, develop, and maintain data architectures, pipelines, and workflows for the collection, processing, storage, and retrieval of large volumes of structured and unstructured data from multiple sources.

Collaborate with cross-functional teams to identify and prioritize data engineering requirements and to develop and deploy data-driven solutions to address business challenges.

Build and maintain scalable data storage and retrieval systems (e.g., data lakes, data warehouses, databases), fault-tolerant, and high-performance data platforms on cloud infrastructure such as AWS, Azure, or Google Cloud Platform.

Develop and maintain ETL workflows, data pipelines, and data transformation processes to prepare data for machine learning and AI applications.

Implement and optimize distributed computing frameworks such as Hadoop, Spark, or Flink to support high-performance and scalable processing of large data sets.

Build and maintain monitoring, alerting, and logging systems to ensure the availability, reliability, and performance of data pipelines and data platforms.

Collaborate with Data Scientists and Machine Learning Engineers to deploy models on production environments and ensure their scalability, reliability, and accuracy.

Requirements :

Bachelor s or master s degree in computer science, engineering, or related field.

At least 5-6 years of experience in data engineering, with a strong background in machine learning, cloud computing and big data technologies.

Experience with at least one major cloud platform (AWS, Azure, GCP).

Proficiency in programming languages like Python, Java, and SQL.

Experience with distributed computing technologies such as Hadoop, Spark, and Kafka.

Familiarity with database technologies such as SQL, NoSQL, NewSQL.

Experience with data warehousing and ETL tools such as Redshift, Snowflake, or Airflow.

Strong problem-solving and analytical skills.

Excellent communication and teamwork skills.

Preferred qualification :

Experience with DevOps practices and tools such as Docker, Kubernetes, or Ansible, Terraform.

Experience with data visualization tools such as Tableau, Superset, Power BI, or Plotly, D3.js.

Experience with stream processing frameworks such as Kafka, Pulsar or Kinesis.

Experience with data governance, data security, and compliance.

Experience with software engineering best practices and methodologies such as Agile or Scrum.

Must Have Skills

data engineer with expertise in machine learning, cloud computing , and big data technologies.

Data Engineering Experince on multiple clouds one of them , preferably GCP

data lakes, data warehouses, databases

ETL workflows, data pipelines,data platforms

Hadoop, Spark, or Flink

Hadoop, Spark, and Kafka

SQL, NoSQL, NewSQL

Redshift, Snowflake, or Airflow

Create a job alert for this search

Data Engineering • Bengaluru, Karnataka, India

Related jobs
Director, Data Engineering

Director, Data Engineering

VisaBangalore, Karnataka, India
Commercial Money Movement Solutions charter is to capture new sources of money movement through card and non-card flows including Visa Business Solutions Government Solutions and Visa Direct which ...Show moreLast updated: 2 days ago
  • Promoted
Snowflake Developer - Data Engineering

Snowflake Developer - Data Engineering

Pro Integrate ConsultingBangalore
What You'll Do : - Snowflake Development : Design, develop, and implement robust, scalable, and high-performance data warehousing solutions on the Snowfl...Show moreLast updated: 30+ days ago
Data Engineering Lead

Data Engineering Lead

Maximus Human Resources PvtLtdBangalore, KA, India
Quick Apply
Experience writing high quality, maintainable SQL / Python on large datasets.Experience with Amazon Redshift, S3, EC2 and other AWS technologies Experience working with ETL tools like Alteryx, Ma...Show moreLast updated: 6 days ago
  • Promoted
Solution Architect - Data Engineering

Solution Architect - Data Engineering

ConsultBae India Private LimitedBangalore
Job Description : We are seeking a highly skilled and innovative Solution Architect to guide the architecture and design of scalable, effic...Show moreLast updated: 25 days ago
Data Engineering Lead / ETL

Data Engineering Lead / ETL

KarixBengaluru, Karnataka, India
Responsible for developing, deploying, monitoring and maintaining ETL Jobs and all data engineering and pipeline activities. Having good knowledge of DB activities and support in DB solutions.Having...Show moreLast updated: 12 days ago
  • Promoted
Solution Architect - Data Engineering

Solution Architect - Data Engineering

NasugroupBangalore
Job Description : - Design and implement end-to-end data solutions for BFS clients, covering data engineering and analytics involving modern data sta...Show moreLast updated: 30+ days ago
Manager Data Engineering

Manager Data Engineering

Tredence Inc.Bengaluru, Karnataka, India
This position requires someone with good problem solving, business understanding and client presence.Overall professional experience of the candidate should be above 10 years.A minimum of 6 years o...Show moreLast updated: 12 days ago
  • Promoted
MSBI Developer - Data Engineering

MSBI Developer - Data Engineering

Akira Consultancy Pvt. Ltd.Bangalore
About Akira : AKIRA is a bootstrapped Data & Analytics Services company driven by technology excellence,consultative practices, and a focus on building technology...Show moreLast updated: 4 days ago
  • Promoted
Engineering Manager - Data Architecture

Engineering Manager - Data Architecture

Freelancer RecruiterBangalore
Mandatory requirements : - Apache Kafka - Elastic search - Spark Spring - NoSql - Swift Show moreLast updated: 30+ days ago
  • Promoted
Data Engineer_ Engineering Management

Data Engineer_ Engineering Management

HashedIn by DeloitteBengaluru, Karnataka, India
Location : Bangalore, Hyderabad, Chennai, Mumbai, Pune, Kolkata, Gurgaon.Notice Period : 0 to Max 45 Days.Expertise in Spark (Scala / Python), Kafka, and cloud-native big data services (GCP, AWS, Azu...Show moreLast updated: 10 days ago
  • Promoted
Associate Director - Data Engineering

Associate Director - Data Engineering

AffineBengaluru, Karnataka, India
Bangalore - HSR Layout (Hybrid).As the Director / Associate Director of Data Engineering, you will be responsible for leading and managing the data engineering team to design, develop, and maintain r...Show moreLast updated: 14 days ago
  • Promoted
GoKwik - Data Engineering Manager

GoKwik - Data Engineering Manager

GOKWIK COMMERCE SOLUTIONS PRIVATE LIMITEDBangalore
About the Role : We are seeking a highly skilled and motivated Data Engineering Manager to lead our growing team of data engineers.This rol...Show moreLast updated: 7 days ago
Director - Data Engineering

Director - Data Engineering

Blend360Bengaluru, Karnataka, India
We are seeking a strategic Director of Data & AI Engineering to lead the growth and evolution of our data engineering function. This role will play a pivotal part in designing scalable platforms...Show moreLast updated: 27 days ago
Engineering Manager (Data Platform)

Engineering Manager (Data Platform)

RazorpayBengaluru, Karnataka, India
We are looking for an experienced.Your team will power advanced analytics, AI / ML, and decision-making across the organization. You will play a critical role in shaping Razorpay’s data architecture, ...Show moreLast updated: 12 days ago
  • Promoted
Technical Lead - Data Engineering

Technical Lead - Data Engineering

Coders Brain Technology Private LimitedBangalore
Job Title : SSE Kafka, Python, and Azure Databricks (Healthcare Data Project) Experience : 5 to 8 years Location : B...Show moreLast updated: 18 days ago
  • Promoted
Data Engineering Manager

Data Engineering Manager

Rocket LearningBengaluru, Karnataka, India
Compensation : 28-30 LPA based on experience.The ideal candidate will have hands-on expertise in.AWS, Data Lakes, Python, and big data technologies. This role is critical in ensuring data-driven deci...Show moreLast updated: 17 days ago
Data Engineering leader

Data Engineering leader

SPIROGreater Bengaluru Area, India
Job Title : Head of Data Engineering.Department : Data Engineering & Analytics.Reporting To : Chief Technology Officer (CTO). Spiro is seeking an exceptional Head of Data Engineering to lead, design, a...Show moreLast updated: 1 day ago
Senior Manager- Data Engineering

Senior Manager- Data Engineering

QuinceBengaluru, Karnataka, India
Quince was started to challenge the existing idea that nice things should cost a lot.Our mission was simple : create an item of equal or greater quality than the leading luxury brands and sell them ...Show moreLast updated: 1 day ago