EY - GDS Consulting - AI And DATA - AWS / Azure Databricks - Senior
ConfidentialThiruvananthapuram / Trivandrum, India19 days ago
Job descriptionDevelop and maintain scalable data pipelines, in line with ETL principles, and build out new integrations, using AWS native technologies, to support continuing increases in data source, volume, and complexity. Define data requirements, gather, and mine data, while validating the efficiency of data tools in the Big Data Environment. Lead the evaluation, implementation and deployment of emerging tools and processes to improve productivity. Implement processes and systems to provide accurate and available data to key stakeholders, downstream systems, and business processes. Partner with Business Analysts and Solution Architects to develop technical architectures for strategic enterprise projects and initiatives. Coordinate with Data Scientists to understand data requirements, and design solutions that enable advanced analytics, machine learning, and predictive modelling. Mentor and coach junior Data Engineers on data standards and practices, promoting the values of learning and growth. Foster a culture of sharing, re-use, design for scale stability, and operational efficiency of data and analytical solutions. Bachelor's degree in engineering, Computer Science, Data Science, or related field 6+ years of experience in software development, data engineering, ETL, and analytics reporting development Experience in building and maintaining data and system integrations using dimensional data modelling and optimized ETL pipelines. Proven track record of designing and implementing complex data solutions Demonstrated understanding and experience using : Python & PySpark : Ability to write efficient scripts for data transformation, and pipeline orchestration, knowledge of Spark or any distributing frameworks Knowledge of data modelling and data architecture principles Familiarity with event-driven architectures , API -based data sources , data quality validation, archival strategies, and incremental loading. Distributed Data Framework (e.g., Spark). Good understanding of Spark architecture Cloud platform services (AWS / Azure). AWS services (AWS Glue, Lambda, S3 , Stepfunctions) Proficiency in SQL and relational database management systems Optimize and tune database performance, including indexing, query optimization, and storage management. CICD - Terraform or CloudFormation, Jenkins ,Bitbucket : For infrastructure-as-code and deployment automation. Experience with database design concepts : schema definition, normalization, relationships, indexing, constraints. Knowledge of Data lakes, Data warehouses Databricks / Delta Lakehouse architecture Code management platforms like Github / Gitlab / etc., Understanding of database architecture, Data modelling concepts and administration. Handson experience of Spark Structured Streaming for building real-time ETL pipelines. Utilizes the principles of continuous integration and delivery to automate the deployment of code changes to elevate environments, fostering enhanced code quality, test coverage, and automation of resilient test cases. Proficient in programming languages (e.g., SQL, Python, Pyspark) to design, develop, maintain, and optimize data architecture / pipelines that fit business goals. Strong organizational skills with the ability to work multiple projects simultaneously and operate as a leading member across globally distributed teams to deliver high-quality services and solutions. Excellent written and verbal communication skills, including storytelling and interacting effectively with multifunctional teams and other strategic partners Strong problem solving and troubleshooting skills Ability to work in a fast-paced environment and adapt to changing business priorities Master's degree in engineering specialized in Computer Science, Data Science, or related field Experience working with pharma clients, preferably in commercial pharma, marketing and medical affairs Demonstrated understanding and experience using : Job orchestration tools like Tidal / Airflow / or similar Knowledge on No SQL Databricks Certified Data Engineer Associate AWS Certified Data Engineer - Associate
At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the best version of you. And we're counting on your unique voice and perspective to help EY become even better, too. Join us and build an exceptional experience for yourself, and a better working world for all.
Job Description For Senior Data Engineer
Rank – Senior
- Objectives and Purpose
- The Senior Data Engineer ingests, builds, and supports large-scale data architectures that serve multiple downstream systems and business users. This individual supports the Data Engineer Leads and partners with Visualization on data quality and troubleshooting needs.
- The Senior Data Engineer will :
- Clean, aggregate, and organize data from disparate sources and transfer it to data warehouses.
- Support development testing and maintenance of data pipelines and platforms, to enable data quality to be utilized within business dashboards and tools.
- Create, maintain, and support the data platform and infrastructure that enables the analytics front-end; this includes the testing, maintenance, construction, and development of architectures such as high-volume, large-scale data processing and databases with proper verification and validation processes.
Your Key Responsibilities
To qualify for the role, you must have the following :
Essential Skillsets
Desired Skillsets
EY | Building a better working world
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.
Enabled by data and technology, diverse EY teams in over 150 countries provide trust through assurance and help clients grow, transform and operate.
Working across assurance, consulting, law, strategy, tax and transactions, EY teams ask better questions to find new answers for the complex issues facing our world today.
Skills Required
Pyspark, AWS Glue, data warehouses , Data Architecture, Sql, Spark, Databricks, Data Modelling, Python
Create a job alert for this search
Ey Gds Consulting Ai • Thiruvananthapuram / Trivandrum, India
Related jobs)
- Promoted
Associate Director, Data Architecture (Snowflake • Databricks • AWS • CPG / FMCG • Enterprise Data)
Sky Systems, Inc. (SkySys)Kollam, IN Associate Director, Data Architecture.Full-Time Contract (40hrs / week).Months+ (with a possibility of Contract-to-Hire).
Marketing, Product, Finance, Supply Chain, and global business teams.This role...Show moreLast updated: 2 days ago
- Promoted
Senior Data Engineer
TEKsystems Global Services in IndiaThiruvananthapuram, IN Develop and manage data pipelines using ADF and Azure data bricks.Expert knowledge in data warehousing, data modeling and ETL.
Good to have datastage exposure.Expertise in on-prem to azure cloud mig...Show moreLast updated: 1 day ago
- Promoted
Senior DataOps Engineer (AWS)
MSBC GroupKollam, IN Join us as a Senior DataOps Engineer (AWS)—Drive High-Performance Data Systems for Financial Services.Lead the E-Comms data pipeline within Compass’s Application Simplification workstream : design, ...Show moreLast updated: 1 day ago
- Promoted
Senior Data Architect- Snowflake
USEReadyThiruvananthapuram, IN USEReady is a data and analytics firm that provides the strategies, tools, capability, and capacity that businesses need to turn their data into a competitive advantage.
USEReady partners with cloud...Show moreLast updated: 1 day ago
- Promoted
Senior AI Engineer
eTeamKollam, IN We’re building AI products that are transforming the.Our mission is to revolutionize how organizations discover, develop, and engage talent through intelligent, human-centered AI solutions.We combi...Show moreLast updated: 3 days ago
- Promoted
AI Data Engineer
Peak Trust Global Real EstateThiruvananthapuram, IN This role requires strong technical skills across Python, automation, ML tooling, and analytical reporting.Key Responsibilities (Technical).
Build automated data collection workflows using tools suc...Show moreLast updated: 4 days ago
- Promoted
Senior AWS Data Engineer
CYAN360Thiruvananthapuram, IN Position : Senior AWS Data Engineer.Work Timings : 2 : 30 PM to 11 : 30 PM IST.Need someone who can join immediately or in 15 days • • •.
Design, develop, and deploy end-to-end data pipelines on AWS cloud in...Show moreLast updated: 30+ days ago
- Promoted
Databricks Gen AI Engineer
SyrenThiruvananthapuram, IN Model Serving, Vector Search, and embedding workflows.Clustering, Unity Catalog, Delta Lake).OpenAI / Azure OpenAI), and GenAI app patterns (RAG / Agents).
Proficiency in SQL, Spark performance tuning...Show moreLast updated: 1 day ago
- Promoted
Senior Data Engineer
FornaxKollam, IN If solving business challenges drives you.Fornax is a team of cross-functional individuals who solve critial business challenges using core concept of analytics, critical thinking.We are seeking a ...Show moreLast updated: 1 day ago
- Promoted
Senior Consultant - CPI
valanticThiruvananthapuram, IN We are looking for an experienced.SAP Cloud Platform Integration (CPI).This role requires a deep understanding of APIs, system connectivity, and modern integration practices.Design, build, and deli...Show moreLast updated: 16 days ago
- Promoted
Senior AI Engineer
MindBrainThiruvananthapuram, IN We are looking for an experienced AI Engineer with strong hands-on expertise in building and deploying autonomous agent systems.
The ideal candidate will contribute to designing, coding, and optimiz...Show moreLast updated: 30+ days ago
- Promoted
Azure Data Engineer
Tech MahindraThiruvananthapuram, IN Expertise in CI-CD, Pyspark, SQL.Expertise in ADF, ADLS & ADB (Mandatory).Experience in "Azure Services / Key Vault".Experience in "GIT / Jenkins / Azure DevOps".Show moreLast updated: 2 days ago
- Promoted
Senior Data Engineer
SAIVA AIKollam, IN We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
AWS Data Architect
ACL DigitalThiruvananthapuram, IN AWS (S3, Redshift, Glue, Lake Formation, IAM).Proficient in data modeling, performance tuning, and security best practices.
.AWS Certified Solutions Architect preferred.Show moreLast updated: 8 days ago
- Promoted
Senior AI Engineer
Calyx GlobalKollam, IN Calyx Global is a ClimateTech startup founded by Donna Lee and Duncan van Bergen, backed by Sequoia, Susquehanna (SIG), New Climate Ventures, and others.
We provide ratings and research on the globa...Show moreLast updated: 3 days ago
- Promoted
Sr. Azure Data Architect & Presales Solution
Programmers.ioKollam, IN Job Title : Azure Data Architect.Location : Hyderabad, Pune, Jaipur.Experience required : 12+ years.We are seeking a highly experienced.
The ideal candidate should bring strong expertise in SQL, ETL / EL...Show moreLast updated: 3 days ago
- Promoted
Senior Solutions Architect (Data)
Hillview Consulting SolutionsKollam, IN If candidate is in Mumbai this would be onsite in Andheri East, Mumbai, Maharashtra.We’re looking for a senior, hands-on.
You’ll own architecture for ETL / ELT, data warehousing, analytics pipelines, ...Show moreLast updated: 3 days ago
- Promoted
AI Strategy Consultant
Tata Consultancy ServicesThiruvananthapuram, IN Experience in developing and implementing AI strategy and AI solution frameworks.Consulting experience in technology strategy and digital transformation.
Proficient in navigating the AI implementati...Show moreLast updated: 1 day ago