No longer accepting applications
Associate Staff Engineer, Big Data Engineer
ConfidentialBengaluru / Bangalore, Chennai30+ days ago
Job descriptionWriting and reviewing great quality code. Understanding the client s business use cases and technical requirements and be able to convert them in to technical design which elegantly meets the requirements Mapping decisions with requirements and be able to translate the same to developers Identifying different solutions and being able to narrow down the best option that meets the client s requirements. Defining guidelines and benchmarks for NFR considerations during project implementation Writing and reviewing design documents explaining overall architecture, framework, and high-level design of the application for the developers Reviewing architecture and design on various aspects like extensibility, scalability, security, design patterns, user experience, NFRs, etc., and ensure that all relevant best practices are followed. Developing and designing the overall solution for defined functional and non-functional requirements; and defining technologies, patterns, and frameworks to materialize it Understanding and relating technology integration scenarios and applying these learnings in projects. Resolving issues that are raised during code / review, through exhaustive systematic analysis of the root cause, and being able to justify the decision taken Carrying out POCs to make sure that suggested design / technologies meet the requirements.
Job description
- Total experience 5+years.
- Hands on experience in Data Engineering, Data Lakes, Data Mesh, or Data Warehousing / ETL environments.
- Strong working knowledge in Python, SQL, Airflow and PySpark.
- Hands-on experience implementing projects applying SDLC practices.
- Hands on experience in building data pipelines and building data frameworks for unit testing, data lineage tracking, and automation.
- Experience with building and maintaining a cloud system.
- Familiarity with databases like DB2 and Teradata.
- Strong working knowledge in Apache Spark, Apache Kafka, Hadoop and MapReduce.
- Strong troubleshooting skills and ability to design for scalability and flexibility.
- Expertise in Spanner for high-availability, scalable database solutions.
- Knowledge of data governance and security practices in cloud-based environments.
- Problem-solving mindset with the ability to tackle complex data engineering challenges
- Familiar with containerization technologies (Docker / Kubernetes).
- Excellent communication and collaboration skills.
RESPONSIBILITIES :
Skills Required
Python, Sql, Db2, IT, Sdlc
Create a job alert for this search
Big Data Engineer • Bengaluru / Bangalore, Chennai
Related jobs
- Promoted
Senior Data Engineer
PoshmarkChennai, Tamil Nadu, India Senior Software Engineer - Data Engineering.Big Data team is a central player in the Poshmark organization.Our goal is to democratize data, support exploding business, build data and ML pipelines t...Show moreLast updated: 9 days ago
- Promoted
Databricks Data Engineer
Tata Consultancy ServicesChennai, Tamil Nadu, India The Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.
Location - Kolkata / Chennai / Trivan...Show moreLast updated: 17 days ago
- Promoted
Big Data Engineer - Hadoop / Spark
ImpacteersChennai Position Overview : We are looking for a skilled and detail-oriented Big Data Engineer to design, develop, and maintain scalable data pipelines and architectures.The ...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
Veraxionchennai, India Python, Spark, DBT, and AWS-native services.Agile environment to deliver scalable, secure, and high-performance data solutions.
Python, DBT, and AWS services (Data Ops Live).Deliver end-to-end data ...Show moreLast updated: 6 days ago
- Promoted
Data Engineer
Tata Consultancy ServicesChennai, Tamil Nadu, India Location- ONLY Chennai Exp- 5 TO 8 Skill- Azure Senior Data Engineers Requires skills-Azure Data Platform - Databricks,AzureSQL and ADLS.Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
EXLChennai, India Job Responsibilities Collaborate with project stakeholders (client) to identify product and technical requirements.Develop, implement, and tune large-scale distributed systems and pipelines th...Show moreLast updated: 30+ days ago
- Promoted
Data Engineer
KaarTechChennai, Tamil Nadu, India Data Pipeline Development : Design, develop, and maintain scalable data pipelines to support various data analytics and reporting needs.
Data Integration : Integrate data from multiple sources and ens...Show moreLast updated: 25 days ago
- Promoted
Senior Data Engineer
Saransh Incchennai, India Data Engineer (Contract-to-Hire | Remote).Full-Time / Fixed-Term Contract (12 months, extendable based on performance).We are seeking a highly skilled.
The ideal candidate will be hands-on, detail-o...Show moreLast updated: 10 days ago
- Promoted
Big Data Engineer - Hadoop / Hive
Unison consulting pte ltdChennai Responsibilities : - Design, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databases.Build and maintain scalabl...Show moreLast updated: 30+ days ago
- Promoted
Big Data Engineer
Tata Consultancy ServicesChennai, Tamil Nadu, India Chennai, Bangalore, Hyderabad, Mumbai, Kolkata, Pune, Indore, Ahmedabad, Kochi, Gurgaon.HDFS, Hive, Spark, Sqoop, Flume, Oozie, Unix Script, Autosys.
Hands-on experience in Big data, Hadoop.Hands-on...Show moreLast updated: 8 days ago
- Promoted
Senior Data Engineer - Snowflake / AWS
ResourcetreeChennai About the Role : A "Senior Data Engineer" is mid-level professional leading the design, build and evolution of the inhouse data platforms.You lead the const...Show moreLast updated: 22 days ago
- Promoted
Reveille Technologies - Big Data Engineer
Reveille TechnologiesChennai Description : We are looking for highly skilled and motivated Big Data Engineers / Leads to join our team and drive the development of our next-generation data plat...Show moreLast updated: 24 days ago
- Promoted
Data Engineer - Big Data Technologies
Zyoin GroupChennai The core responsibilities for the job include the following : Data Pipeline Development : - Design, d...Show moreLast updated: 30+ days ago
- Promoted
Staff Engineer, Machine Learning
PoshmarkChennai, Tamil Nadu, India Staff Engineer Machine Learning.Big Data team is a central player in the Poshmark organization.Our goal is to democratize data, support exploding business, build data and ML pipelines to fuel exist...Show moreLast updated: 6 days ago
- Promoted
Senior Data Engineer
SAIVA AIChennai, IN We are building the future of healthcare analytics.Join us to design, build, and scale robust data pipelines that power nationwide analytics and support our machine learning systems.Our goal : pipel...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Engineer
Invoria Global SolutionsChennai, Tamil Nadu, India Invoria Global Solutions (IGS).This opportunity is being offered.Senior Power BI Developer (8–12 Years Experience).PAN India — Delhi / Bangalore / Chennai / Hyderabad (Hybrid).We are looking for an...Show moreLast updated: 14 days ago
- Promoted
- New!
Lead Data Engineer
JMAN GroupChennai, India JMAN Group is a fast-growing data engineering & data science consultancy.We work primarily with Private Equity Funds and their Portfolio Companies to create commercial value using Data & Artificial...Show moreLast updated: 22 hours ago
- Promoted
Staff / Senior Data Engineer
HG Insightsmount, India At HG Insights we provides AI-powered revenue growth intelligence and technology intelligence data to help B2B companies refine their go-to-market (GTM) strategies by analyzing market size, identif...Show moreLast updated: 10 days ago