Big Data SpecialistAtomic North • Pune, Republic Of India, IN
Big Data Specialist
Atomic North • Pune, Republic Of India, IN
12 days ago
Job descriptionWe are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance and reliability. Design, develop, and maintain data pipelines using PySpark and related Big Data technologies. Work with HDFS, Hive, Sqoop, and other tools in the Hadoop ecosystem. Write efficient HiveQL and SQL queries to handle large-scale datasets. Perform performance tuning and optimization of distributed data systems. Collaborate with cross-functional teams in an Agile environment to deliver high-quality solutions. Manage and schedule workflows using Apache Airflow or Oozie. Troubleshoot and resolve issues in data pipelines to ensure reliability and accuracy. Proven experience in Big Data Engineering with a focus on PySpark. Strong knowledge of HDFS, Hive, Sqoop, and related tools. Proficiency in SQL / HiveQL for large datasets. Expertise in performance tuning and optimization of distributed systems. Familiarity with Agile methodology and collaborative team practices. Experience with workflow orchestration tools (Airflow / Oozie). Strong problem-solving, analytical, and communication skills. Knowledge of data modeling and data warehousing concepts. Exposure to DevOps practices and CI / CD pipelines for data engineering. Experience with other Big Data frameworks such as Spark Streaming or Kafka.
Big Data Engineer
Location : Pune / Nagpur (WFO)
Experience Required : 8+ Years
- Long term onsite to Abu Dhabi
Job Summary :
Key Responsibilities :
Required Skills :
Good to Have :
Create a job alert for this search
Data Big Data • Pune, Republic Of India, IN
Related jobs
Data Pipeline Specialist
Atomic North • Pune, Republic Of India, IN
Long term onsite to Abu Dhabi • •.We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems.
The ideal candidate will be responsible for designing, d...Show more
Last updated: 12 days ago • Promoted
Big Data Developer
Tata Consultancy Services • Pune, Republic Of India, IN
Role : Big Data Streaming Ingestion Developer.Required Technical Skill Set : Hortonworks suite of tools including Kafka, Scala, Hbase and Phoenix.
Location of Requirement : Chennai / Mumbai / Pune.Hands on...Show more
Last updated: 30+ days ago • Promoted
Cloud Data Pipeline Specialist
ACL Digital • Republic Of India, IN
We have urgent openings for AWS Data Engineer.The ideal candidate will have a strong background in designing, deploying, and maintaining data pipelines using industry-leading tools such as.Familiar...Show more
Last updated: 9 days ago • Promoted
Big Data Integration Specialist
Tata Consultancy Services • Republic Of India, IN
Locations : Hyderabad, Kolkata, Noida, Chennai, Bangalore.Designing and building Databricks data pipelines is required.Experience in transforming the data using Azure Databricks & Delta lake.Hands-o...Show more
Last updated: 8 days ago • Promoted
Big Data Analytics Specialist
SGS & Co • Chennai, Republic Of India, IN
Position Title : Senior Data Engineer.Experience Required : 8 to 12 Years.We are looking for a highly skilled and experienced Data Engineer with strong expertise in.
The ideal candidate will play a ke...Show more
Last updated: 22 days ago • Promoted
Big Data Ingestion Specialist
Tata Consultancy Services • Chennai, Republic Of India, IN
Desired Skills -Technical / Behavioral.Extensive experience in Bigdata Projects particularly Data Ingestion and Transformation Projects.
Total experience of 6 Years or Above with relevant Experience a...Show more
Last updated: 4 days ago • Promoted
Cloud Data Pipeline Specialist
HCLTech • Chennai, Republic Of India, IN
Develop and maintain data models and transformations using DBT.Load and manage data in Snowflake, including staging and curated layers.
Implement data ingestion processes from various sources using ...Show more
Last updated: 4 days ago • Promoted
Data Visualization Specialist
Tata Consultancy Services • Pune, Republic Of India, IN
Skill-Data Analyst - Big Query.Proficiency in SQL, especially in the context of Google BigQuery and the Google Cloud Platform (GCP).
Knowledge of Python or R, particularly for more advanced analyses...Show more
Last updated: 30+ days ago • Promoted
Big Data Developer
Brillio • India
Design, develop, and maintain robust data pipelines in Hadoop and related ecosystems, ensuring data reliability, scalability, and performance.
Implement data ETL processes for batch and streaming an...Show more
Last updated: 11 hours ago • Promoted • New!
BigData Scala + Quantexa
LTIMindtree • India
Secondary skill : Apache Hive, HDFS, Oozie, Impala, Parquet.Location : Chennai / Mumbai / Pune / Bengaluru.Show more
Last updated: 2 days ago • Promoted
Cloud Data Pipeline Specialist
Capgemini Invent • Pune, Republic Of India, IN
At Capgemini Invent, we believe difference drives change.As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to...Show more
Last updated: 6 days ago • Promoted
Big Data Ingestion Engineer
Tata Consultancy Services • Pune, Republic Of India, IN
Role : Big Data Streaming Ingestion Developer.Required Technical Skill Set : Hortonworks suite of tools including Kafka, Scala, Hbase and Phoenix.
Location of Requirement : Chennai / Mumbai / Pune.Hands on...Show more
Last updated: 21 days ago • Promoted
Big Data Specialist
Atomic North • India
Long term onsite to Abu Dhabi • •.We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems.
The ideal candidate will be responsible for designing, d...Show more
Last updated: 2 days ago • Promoted
Data Engineer (Big Data)
Tata Consultancy Services • Chennai, Republic Of India, IN
Desired Skills -Technical / Behavioral.Extensive experience in Bigdata Projects particularly Data Ingestion and Transformation Projects.
Total experience of 6 Years or Above with relevant Experience a...Show more
Last updated: 4 days ago • Promoted
GCP Big Data Engineer
Impetus • Republic Of India, IN
Provides technical leadership in Big Data space (Hadoop Stack like Spark, M / R, HDFS, Hive, HBase etc) and contributes to open source Big Data technologies.
Must have : Operating knowledge of cloud co...Show more
Last updated: 30+ days ago • Promoted
Big Data Solutions Engineer
HCLTech • Chennai, Republic Of India, IN
Senior Data Engineer – Vendor Experience : 6–12 Years.We are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernete...Show more
Last updated: 9 days ago • Promoted
Senior Big Data Solutions Engineer
InfoCepts • Pune, Republic Of India, IN
Key Result Areas and Activities : .Study existing technology landscape and understand current data integration frameworks and do impact assessment for the requirements.
Should be able to design comple...Show more
Last updated: 22 days ago • Promoted
Big Data Platform Specialist
LTIMindtree • Republic Of India, IN
Job Location : Bangalore, Hyderabad, Chennai, Mumbai, Pune.Notice Period : Immediate to 15 days.Please do not apply Notice Period with more than 15 days of Notice Period.
Mandatory Skills : Build Cloud...Show more
Last updated: 13 days ago • Promoted