AWS Data Engineer
TerraGiGhosur, tamil nadu, in14 hours ago
Job descriptionDemonstrated experience in change management processes, including understanding of governance frameworks and preparation of supporting artefacts required for approvals. Strong clarity on the path to production, with hands-on involvement in deployments, testing cycles, and obtaining business sign-offs. Proven track record in technical solution design, with the ability to provide architectural guidance and support implementation strategies. Experience in at least developing and delivering end-to-end Proof of Concept (POC) solutions covering the below : Basic proficiency in Databricks, including creating jobs and configuring clusters. Exposure to connecting external data sources (e.g., Amazon S3) to Databricks. Understanding of Unity Catalog and its role in data governance. Familiarity with notebook orchestration and implementing modular code structures to enhance scalability and maintainability. Candidates must have actual hands-on work experience , not just home projects or academic exercises. Profiles should clearly state how much experience they have in each skill area , as this helps streamline the interview process. Candidates must know their CV / profile inside out , including all projects and responsibilities listed. Any ambiguity or lack of clarity on the candidate’s part can lead to immediate rejection, as we value accuracy and ownership. They should be able to confidently explain their past experience, challenges handled, and technical contributions.
Role
AWS Data Engineer
Experience
8+ years
Location
Remote
Time Zone
UK
Duration
2 months (Extendable)
Job Description
- Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
- Writing reusable, testable, and efficient code
- Integration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark scripts.
- Need to ensure overall build delivery quality is good and on-time delivery is done at all times.
- Should be able to handle meetings with customers with ease.
- Need to have excellent communication skills to interact with the customer.
- Be a team player and willing to work in an onsite-offshore model, mentor other folks in the team (onsite as well as offshore)
- 5+ years of experience in programming with python. Strong proficiency in python
- Familiarity with functional programming concepts
- 3+ years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR
- Experience in building pipelines and data lake for large enterprises on AWS
- Good understanding of Spark’s Dataframe and API
- Experience in configuring EMR clusters on AWS
- Experience in dealing with AWS S3 object storage from Spark.
- Experience in troubleshooting spark jobs. Knowledge of monitoring spark jobs using Spark UI
- Performance tuning of Spark jobs.
- Understanding fundamental design principles behind business processes
Process Knowledge and Expertise :
Databricks-Specific Skills :
Important Pointers :
Create a job alert for this search
Aws Data Engineer • hosur, tamil nadu, in
Related jobs
- Promoted
Data Engineer(Azure / AWS, Python / Pyspark, SQL)
Sail Analyticshosur, tamil nadu, in Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift).
Bachelor's degree or equivalent...Show moreLast updated: 19 days ago
- Promoted
AWS Data Engineer
L&T Technology ServicesBengaluru, Karnataka, India Graduate with 8-10 years of experience working as a AWS Data Engineer.Excellent knowledge of PySpark, JavaScript, AWS - RDS, and Redshift.Show moreLast updated: 30+ days ago
- Promoted
Senior AWS Data Engineer
CYAN360Bangalore, IN Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •.
Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show moreLast updated: 30+ days ago
- Promoted
- New!
Data Engineer
BrothersTechhosur, tamil nadu, in Remote (Night Shift - Flexible).Must have excellent English written and verbal communication skills.The ideal candidate should be experienced in data engineering on.
Snowflake Azure -ADF, Microsoft ...Show moreLast updated: 6 hours ago
- Promoted
Senior Data Engineer (AWS / Databricks)
Accoladehosur, tamil nadu, in The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 21 days ago
- Promoted
Data Engineer
Manuh Technologieshosur, tamil nadu, in S3, Glue, Redshift, EMR, Lambda).Develop automation scripts and tools using.Collaborate with data analysts, data scientists, and business stakeholders to ensure data availability and reliability.Tr...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer
Tata Consultancy ServicesBengaluru, Karnataka, India Bangalore or Bhubaneswar or Hyderabad.Bachelor's degree in Engineering , Technology computer science, statistics, biostatistics, mathematics, biology or other health related field or equivalent exp...Show moreLast updated: 30+ days ago
- Promoted
Python AWS Data Engineer
Digitrix Software LLPBangalore, IN Mandatory skills - python , pyspark, who can write codes , any cloud exp - aws / gcp / azure.Python, AWS Python (core language skill) Backend, Pandas, PySpark (DataFrame API), interacting with AW...Show moreLast updated: 30+ days ago
- Promoted
Agentic Data Engineer (AWS)
Sustainability Economics.aiBengaluru, Karnataka, India AI, enabling profitable energy transitions while powering end-to-end AI infrastructure.By integrating AI-driven cloud solutions with sustainable energy, we create scalable, intelligent ecosystems t...Show moreLast updated: 30+ days ago
- Promoted
2921 - AWS Data Engineer
EXLBangalore Urban, Karnataka, India Job Title – Data Warehouse Developer.We are looking for a Data Warehouse Developer with strong expertise in SQL and a deep understanding of Data Warehouse (DWH) concepts.
The role involves designing...Show moreLast updated: 11 days ago
- Promoted
Senior Data Engineer
Manuh Technologieshosur, tamil nadu, in Develop automation scripts and tools using.Collaborate with data analysts, data scientists, and business stakeholders to ensure data availability and reliability.
Troubleshoot and resolve data-relat...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Impetushosur, tamil nadu, in Strong ability to write clean and efficient code.Good understanding of Spark SQL for distributed data processing.Experience with large datasets and structured data manipulation.Ability to write que...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer (Pyspark)
Tata Consultancy ServicesBengaluru, Karnataka, India Bangalore,Chennai,Hyderabad,Pune,Kochi,Bhubaneshawar,Kolkata.AWS Lambda, Python, Boto3 ,Pyspark, Glue.Strong experience in Python to package, deploy and monitor data science apps.Knowledge in Pytho...Show moreLast updated: 21 days ago
- Promoted
Data Engineer
V2Softhosur, tamil nadu, in Experience to deal with large stream volumes.Implementation experience ( atleast couple of projects ) in MSK / Flink / Spark / Scala.
Very good knowledge on atleast 3 out of the 4 technologies.Desirable e...Show moreLast updated: 11 days ago
- Promoted
AWS Data Engineer
CoforgeBengaluru, Karnataka, India AWS Services, ETL, ETL Integration, CodePipeline, Jenkins, Glue, EMR, Athena, ECS, EKS, Kubernetes, CloudWatch, Prometheus, Grafana, Python, Shell, or PowerShell.
We are looking for an experienced A...Show moreLast updated: 11 days ago
- Promoted
Data Engineer
Talogyhosur, tamil nadu, in This opportunity is ideal for a determined and proactive individual who has a wide range of skills in a variety of database administration, reporting and dashboarding disciplines.This role requires...Show moreLast updated: 30+ days ago
- Promoted
AI Data Engineer (AWS / Snowflake)
Entechhosur, tamil nadu, in AI Data Engineer (AWS / Snowflake).Entech Consulting is supporting our client, an AI-driven company that helps global organizations reduce the cost and time-to-market for promotional assets through i...Show moreLast updated: 2 days ago
- Promoted
- New!
AWS Data Engineer
TerraGiGBangalore, IN Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
Writing reusable, testable, and efficient code.Integration of data storage ...Show moreLast updated: 18 hours ago
- Promoted
Lead Data Engineer
ITQube LTDhosur, tamil nadu, in Looking for a Data Engineer to join our engineering will contribute directly to the design, automation, and optimization of our data processes, primarily developing solutions in Python within the A...Show moreLast updated: 21 days ago
- Promoted
- New!
Senior Data Engineer
Nityo Infotechhosur, tamil nadu, in Profile name- Senior Data Infrastructure Engineer - Apache Spark & Kubernetes.Minimum 5 years of experience in Data Engineering or Infrastructure roles.
Proven hands-on experience with Apache Spark ...Show moreLast updated: 14 hours ago