AWS Databricks + Mongo DB ( Full-time at a Fortune 500 tech MNC )
HARPvijayapura, rajasthan, in1 day ago
Job descriptionApache Spark , PySpark , Spark SQL Databricks Python (for Data Engineering use cases) Relational Databases : Oracle, SQL Server NoSQL : MongoDB Big Data Architecture and scalable pipeline design Java or Scala (good to have) AWS IAM : Role and policy creation Docker : Image creation and container management Camunda : Workflow orchestration and process management Kubernetes : Container orchestration (good to have)
JOB DESCRIPTION / REQUIREMENT
Role Overview :
We are looking for a Senior Data Engineer with 5+ years of experience in Big Data technologies . The ideal candidate will have strong hands-on experience in Spark , PySpark , and Databricks , and be capable of building scalable and reliable data pipelines. Knowledge of DevOps practices and containerization tools is a plus.
Primary Responsibilities :
- Develop and maintain scalable data pipelines for large-scale data processing.
- Translate business requirements into technical specifications and efficient ETL code.
- Work independently to develop and optimize Spark / PySpark pipelines.
- Write complex business logic using PySpark and Spark SQL.
- Understand and manage Spark clusters and parallel data processing.
- Develop unit tests for ETL components to ensure data integrity and quality.
- Build modular, reusable code functions to streamline development.
- Work with Athena : create tables, indexes, and write complex SQL queries.
- Integrate data from relational databases (Oracle, SQL Server) and NoSQL databases (e.g., MongoDB).
Technical Skills Required (Primary) :
Secondary / Desirable Skills :
Create a job alert for this search
Db • vijayapura, rajasthan, in
Related jobs
- Promoted
Data Engineer(Azure / AWS, Python / Pyspark, SQL)
Sail Analyticsvijayapura, rajasthan, in Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift).
Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer
Tata Consultancy Servicesvijayapura, rajasthan, in Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 11 days ago
- Promoted
Azure databrick ( Full-time at a Fortune 500 tech MNC )
HARPvijayapura, rajasthan, in Minimum 8 to 11 years of hands-on experience with Databricks platform and Azure Data Factory.Expertise in building and optimizing ELT / ETL pipelines using Databricks and Azure Data Factory.Strong kn...Show moreLast updated: 1 day ago
- Promoted
AWS Database Eng
Tata Consultancy Servicesvijayapura, rajasthan, in Desired Competencies (Technical / Behavioral Competency).Should have expertise in creating data warehouses in AWS utilizing the following tools : EC2, S3, EMR, Athena, Sagemaker, Aurora and Snowflake....Show moreLast updated: 3 days ago
- Promoted
Senior AWS DevOps Engineer (ECS / CI-CD)
graph8vijayapura, rajasthan, in Your mission : build and maintain a secure, automated, and observable AWS foundation so engineers can ship faster, safer, and cheaper.
You’ll be the owner of deployment velocity, system uptime, and c...Show moreLast updated: 1 day ago
- Promoted
Senior Java AWS Engineer
Applicantzvijayapura, rajasthan, in THIS IS A LONG TERM CONTRACT POSITION WITH ONE OF THE LARGEST, GLOBAL, TECHNOLOGY LEADER.Candidate should be very good in Java and Spring.
Must worked on complex system and ready to onboard quickly....Show moreLast updated: 2 days ago
- Promoted
Cloud Engineer (AWS, Azure)
BPMLinksvijayapura, rajasthan, in Job Description – Cloud Engineer (AWS, Azure, Snowflake) : .We are looking for a skilled Cloud Engineer to take ownership of the existing multi-cloud infrastructure and data integration setup.Manage ...Show moreLast updated: 2 days ago
- Promoted
AWS Data Engineer
TerraGiGvijayapura, rajasthan, in Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
Writing reusable, testable, and efficient code.Integration of data storage ...Show moreLast updated: 14 days ago
- Promoted
Senior / Lead Engineer - DevOps (AWS / Azure / GCP)
QBurstvijayapura, rajasthan, in We are seeking an experienced and versatile DevOps Engineer.The ideal candidate will have hands-on experience with CI / CD pipelines, Kubernetes, Linux systems, monitoring / logging tools, and Infrastr...Show moreLast updated: 17 days ago
- Promoted
Full Stack Engineer
Applicantzvijayapura, rajasthan, in THIS IS A 5-MONTH CONTRACT POSITION WITH ONE OF THE LARGEST, GLOBAL, TECHNOLOGY LEADERS.In this role, you will migrate legacy systems to.
AWS Cloud Services (EC2, RDS, S3, Elasticsearch).Microservic...Show moreLast updated: 30+ days ago
- Promoted
AWS Cloud Engineer
Proglitevijayapura, rajasthan, in Infrastructure & System Administration : .Deploy, manage, and optimize EC2 instances across dev, test, and production environments.
Perform system administration and troubleshooting for Linux and Wind...Show moreLast updated: 30+ days ago
- Promoted
AWS Cloud Engineer (Terraform Experts)
Saradysol Tech Ventures Private Limitedvijayapura, rajasthan, in AWS Cloud Engineer (Terraform Experts).Saradysol Tech Ventures Private Limited.Saradysol Tech Ventures Private Limited is a leading technology solutions provider, dedicated to delivering exceptiona...Show moreLast updated: 1 day ago
- Promoted
Senior AWS Administrator
CYAN360vijayapura, rajasthan, in Job title : AWS Infrastructure Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.ETL / ELT processes using AWS services such as AWS Glue, AWS Lambda and AWS Step Functions.
Ensure data quality, security, ...Show moreLast updated: 15 days ago
- Promoted
AWS Data Engineer
Insight Globalvijayapura, rajasthan, in We are seeking a highly skilled AWS Data Engineer to join our team and help design, build, and maintain scalable data pipelines and infrastructure in the cloud.
The ideal candidate will have experie...Show moreLast updated: 1 day ago
- Promoted
AWS Data Engineer / Snowflake Data Engineer
Numeric Technologiesvijayapura, rajasthan, in Please apply only if you are comfortable to work in rotational shift.Apply only if you are an immediate to 15 days joiner.
Work Mode - Monthly once to office in Bangalore.Years of experience - 2 to ...Show moreLast updated: 2 days ago
- Promoted
Senior Data Engineer (AWS / Databricks)
Accoladevijayapura, rajasthan, in The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
- Promoted
DevOps Engineer
IntraEdgevijayapura, rajasthan, in Seeking a skilled DevOps Engineer with strong expertise in Amazon Web Services (AWS) to join the engineering team.In this role, you will design, implement, and maintain infrastructure that enables ...Show moreLast updated: 30+ days ago
- Promoted
Python Developer
Infogainvijayapura, rajasthan, in Python Developer with 3–5 years of experience, specializing in AWS Lambda and API development.Design, develop, and deploy serverless applications using AWS Lambda.
Build and maintain RESTful APIs.Wr...Show moreLast updated: 24 days ago