Aws Data Engineer
TerraGiGDombivli, Republic Of India, IN12 days ago
Job descriptionDemonstrated experience in change management processes, including understanding of governance frameworks and preparation of supporting artefacts required for approvals. Strong clarity on the path to production, with hands-on involvement in deployments, testing cycles, and obtaining business sign-offs. Proven track record in technical solution design, with the ability to provide architectural guidance and support implementation strategies. Experience in at least developing and delivering end-to-end Proof of Concept (POC) solutions covering the below : Basic proficiency in Databricks, including creating jobs and configuring clusters. Exposure to connecting external data sources (e.G., Amazon S3) to Databricks. Understanding of Unity Catalog and its role in data governance. Familiarity with notebook orchestration and implementing modular code structures to enhance scalability and maintainability. Candidates must have actual hands-on work experience , not just home projects or academic exercises. Profiles should clearly state how much experience they have in each skill area , as this helps streamline the interview process. Candidates must know their CV / profile inside out , including all projects and responsibilities listed. Any ambiguity or lack of clarity on the candidate’s part can lead to immediate rejection, as we value accuracy and ownership. They should be able to confidently explain their past experience, challenges handled, and technical contributions.
Role
AWS Data Engineer
Experience
8+ years
Location
Remote
Time Zone
UK
Duration
2 months (Extendable)
Job Description
- Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
- Writing reusable, testable, and efficient code
- Integration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark scripts.
- Need to ensure overall build delivery quality is good and on-time delivery is done at all times.
- Should be able to handle meetings with customers with ease.
- Need to have excellent communication skills to interact with the customer.
- Be a team player and willing to work in an onsite-offshore model, mentor other folks in the team (onsite as well as offshore)
- 5+ years of experience in programming with python. Strong proficiency in python
- Familiarity with functional programming concepts
- 3+ years of hands-on experience in developing ETL data pipelines using pySpark on AWS EMR
- Experience in building pipelines and data lake for large enterprises on AWS
- Good understanding of Spark’s Dataframe and API
- Experience in configuring EMR clusters on AWS
- Experience in dealing with AWS S3 object storage from Spark.
- Experience in troubleshooting spark jobs. Knowledge of monitoring spark jobs using Spark UI
- Performance tuning of Spark jobs.
- Understanding fundamental design principles behind business processes
Process Knowledge and Expertise :
Databricks-Specific Skills :
Important Pointers :
Create a job alert for this search
Aws Data Engineer • Dombivli, Republic Of India, IN
Related jobs
- Promoted
Senior Data Engineer (AWS / Databricks)
Accolademumbai, maharashtra, in The multifamily real estate industry is undergoing a massive transformation, and Accolade is at the forefront.We are building the industry's first AI-native Operations Centralization Platform, desi...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer - Snowflake / AWS
ResourcetreeMumbai About the Role : A "Senior Data Engineer" is mid-level professional leading the design, build and evolution of the inhouse data platforms.You lead the const...Show moreLast updated: 28 days ago
- Promoted
Collinson Group - Lead Data Engineer - AWS Platform
Collinson Loyalty & Benefits Private Limited”Mumbai Description : - Design, develop and maintain data pipelines for collecting, transforming and loading data into various data stores.
Build and maintain data warehousing...Show moreLast updated: 23 days ago
- Promoted
Data Engineer(Azure / AWS, Python / Pyspark, SQL)
Sail Analyticsmumbai city, maharashtra, in Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift).
Bachelor's degree or equivalent...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer
TerraGiGMumbai, IN Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
Writing reusable, testable, and efficient code.Integration of data storage ...Show moreLast updated: 12 days ago
- Promoted
Data Engineer
Recrodombivli, maharashtra, in Data Pipeline Engineering : Design, build, and maintain ingestion, transformation, and storage pipelines using Azure Data Factory, Synapse Analytics, and Data Lake.
AI Data Enablement : Collaborate wi...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Envuthane, maharashtra, in At Envu, we partner with our customers to design world-class, forward-thinking innovations that protect and enhance the health of environments around the world.
We offer dedicated services in : Profe...Show moreLast updated: 30+ days ago
- Promoted
Aws Data Engineer
TerraGiGThāne, Republic Of India, IN Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR.
Writing reusable, testable, and efficient code.Integration of data storage ...Show moreLast updated: 12 days ago
- Promoted
AI / ML & Data Engineer
Mindfire SolutionsKalyan-Dombivli, IN We are looking for an experienced AI / ML & Data Engineer to design, develop, and deploy scalable machine learning models and data infrastructure on AWS.
You will work closely with cross-functional te...Show moreLast updated: 14 days ago
- Promoted
Senior AWS Data Engineer_Exp : 5+ Years
Atyeti Incthane, maharashtra, in Required Skills & Qualifications.Bachelor’s or Master’s degree in Computer Science or equivalent experience.Application Developer or in similar software engineering roles.Python; strong SQL and clo...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
thinkbridgenavi mumbai, maharashtra, in We are a global digital product development firm that helps growth-stage companies gain the technology sophistication and maturity of leading modern digital businesses.
We differentiate ourselves by...Show moreLast updated: 30+ days ago
- Promoted
Aws Data Engineer
Tata Consultancy ServicesDombivli, Republic Of India, IN Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 9 days ago
- Promoted
AWS Data Engineer
Tata Consultancy Servicesmumbai city, maharashtra, in Aws data engineer having strong experience of Python.Technical / Behavioral Competency.Proficient in Python, with experience in deploying Python packages and OOP, Experience in ingesting data from di...Show moreLast updated: 9 days ago
- Promoted
AWS-DATA Engineer
BDO Indiamumbai, maharashtra, in BDO India Services Private Limited is the India member firm of BDO International.BDO India offers strategic, operational, accounting, tax & regulatory advisory and assistance for both domestic and ...Show moreLast updated: 1 day ago
- Promoted
Senior Data Engineer with AWS_ Exp : 6+ Years
Atyeti IncMumbai, IN Required Skills & Qualifications.Bachelor’s or Master’s degree in Computer Science or equivalent experience.Application Developer or in similar software engineering roles.Python; strong SQL and clo...Show moreLast updated: 1 day ago
- Promoted
Data Engineer
Insight Globaldombivli, maharashtra, in GCP DATA ENGINEER - Contract (Long term).Data Engineer with hands-on support for Google Looker.Strong experience in data modeling and building data marts.
Proficiency in ETL / ELT pipeline development...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Engineer
ConfidentialNavi Mumbai, Mumbai, India Holcim is the leading partner for sustainable construction, creating value across the built environment from infrastructure and industry to buildings.
We offer high-value end-to-end Building Materia...Show moreLast updated: 5 days ago
- Promoted
AWS Data Engineer - Contractual Position
ConfidentialMumbai Work extensively with AWS services including S3, Lambda, Glue, API Gateway, and SQS.Design and implement data engineering solutions on AWS, utilizing Python, PySpark, and SQL for processing and tra...Show moreLast updated: 5 days ago