This job offer is not available in your country.
Big Data Developer
IntraEdgeChennai, Tamil Nadu, India30+ days ago
Job descriptionBachelor’s or Master’s degree in Computer Science, Engineering, or a related field. Minimum 5 years of hands-on experience in Big Data engineering roles. Strong proficiency in Python for data processing and scripting. Experience with Apache Spark , Hadoop ecosystem (Hive, HDFS, HBase), Kafka. Solid understanding of data warehousing concepts and data modeling. Hands-on experience with SQL and NoSQL databases. Familiarity with cloud data platforms like AWS (EMR, S3, Glue) , Azure , or Google Cloud . Good understanding of CI / CD, containerization (Docker, Kubernetes), and version control (Git). Experience with Airflow or other workflow orchestration tools. Knowledge of real-time data processing (e.g., Spark Streaming, Flink). Exposure to data governance, data lineage, and catalog tools. Opportunity to work on cutting-edge Big Data solutions with global clients. Collaborative and innovation-driven work culture. Competitive compensation and career growth opportunities. Flexible work environment with hybrid (based on role).
Job Title : Senior Big Data Engineer (Python)
Experience : 5+ Years
Locations : Chennai, Bangalore, Gurugram
Employment Type : Full-Time
Notice Period : Immediate to 30 Days Preferred
Job Summary
We are seeking an experienced Big Data Engineer with strong expertise in Python to design, build, and manage large-scale data pipelines and analytics solutions. The ideal candidate will have hands-on experience working with Big Data technologies and cloud platforms, and a passion for writing efficient, scalable, and maintainable code.
Key Responsibilities
- Design, develop, and maintain scalable big data pipelines using Python and other data processing tools.
- Work with distributed data processing frameworks like Spark, Hadoop, Hive, or similar.
- Implement ETL processes for structured and unstructured data from various sources.
- Collaborate with data scientists, analysts, and other engineering teams to understand data needs.
- Optimize data workflows and ensure data quality, integrity, and security.
- Deploy solutions on cloud platforms (AWS / GCP / Azure) and automate data workflows.
- Monitor data pipelines, troubleshoot issues, and ensure high availability and performance.
Required Skills & Qualifications
Preferred Skills (Nice to Have)
Why Join Us?
Create a job alert for this search
Big Data Developer • Chennai, Tamil Nadu, India
Related jobs
- Promoted
Senior Data Engineer
Eucloid Data SolutionsChennai, Tamil Nadu, India Eucloid is looking for a skilled Data Engineer to join our team and contribute to the design, development, and optimization of data frameworks pipelines using Python, Elasticsearch, SQL DB and Queu...Show moreLast updated: 8 days ago
- Promoted
Big Data Specialist
Brilliochennai, tamil nadu, in We are seeking a highly skilled Big Data Engineer to join our team.The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distribute...Show moreLast updated: 26 days ago
- Promoted
Senior Big Data Developer - ETL
MOURITechChennai Looking for a Senior Big Data Developer to design, build, and optimize data pipelines and analytics solutions.Candidates with prior exposure to the Ericsson Wallet Platform will be given preference...Show moreLast updated: 1 day ago
- Promoted
Big Data Developer(Chennai)
Tata Consultancy ServicesChennai, Tamil Nadu, India Greetings from Tata Consultancy Services!!!.TCS has been a great pioneer in feeding the fire of young techies like you.We are a global leader in the technology arena and there’s nothing that can st...Show moreLast updated: 20 days ago
- Promoted
- New!
Big Data with Quantexa-Pan India
Tata Consultancy ServicesChennai, IN We are looking for Big data developer with Quantexa as mandatory skill.Experience Range- 5 to 12 years.Data Engineer - Big Data Hadoop, Spark and Scala.
Spark, Scala, Hadoop, HIVE, Oozie, Shell Scri...Show moreLast updated: 14 hours ago
- Promoted
Senior Data Engineer
DeltacubesChennai, IN Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect.
Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 15 days ago
- Promoted
Senior Data Engineer
InfogainChennai, IN Big Data Engineer (Lead) : As a Big Data Engineer (Lead), you will be responsible for leading a team of big data engineers.
You will work closely with clients and team members to understand their req...Show moreLast updated: 30+ days ago
- Promoted
Big Data Engineer - Hadoop / Spark
ImpacteersChennai Position Overview : We are looking for a skilled and detail-oriented Big Data Engineer to design, develop, and maintain scalable data pipelines and architectures.The ...Show moreLast updated: 13 days ago
- Promoted
Big Data Developer
Tata Consultancy ServicesChennai, Tamil Nadu, India JAVA,SCALA using Maven, Apache Spark Frameworks and Unix Shell scripting.Should be comfortable with Unix File system as well as HDFS commands.
Should have worked on query languages such as Oracle SQ...Show moreLast updated: 30+ days ago
- Promoted
Senior Big Data Engineer
VeltrisChennai, IN Veltris is a Digital Product Engineering Services partner committed to driving technology-enabled transformation across enterprises, businesses, and industries.
We specialize in delivering next-gene...Show moreLast updated: 14 days ago
- Promoted
Senior Data Engineer
InfoBeansChennai, Tamil Nadu, India Azure Data Factory (Azure Data Factory, Fabric Data Factory, SQL, Matillion, Python.Chennai, Bangalore, Hyderabad, Indore.
Experienced Data Professional with strong SQL skills, focusing on data inte...Show moreLast updated: 26 days ago
- Promoted
Data Engineer
AceolutionChennai, IN We are looking for a freelancer to engage with us for 20-40 hours per week.Kindly find the JD below for your reference.Design, develop, and maintain scalable data pipelines and workflows.Work exten...Show moreLast updated: 8 days ago
- Promoted
Senior Data Engineer
BrightEdgeChennai, IN BrightEdge About Us BrightEdge is a leading SEO and content performance marketing platform that transforms online content into tangible business results.
Our platform processes massive amounts of da...Show moreLast updated: 30+ days ago
- Promoted
Senior Data Owner
iVoyantChennai, IN One of my leading MNC clients is looking for an experienced Senior Data Owner to oversee the Data & Analytics pillar of their FinCrime Platform.
This is a high-visibility role where you’ll be part o...Show moreLast updated: 6 days ago
- Promoted
Senior Data Engineer
Tata Consultancy ServicesChennai, Tamil Nadu, India Greetings from TCS Human Resources Team!!!.Skill : Snowflake Data Engineer.Required Technical Skill Set : Snowflake, dbt, FiveTran, Snowpipe, AWS services : Managed Airflow (MWAA), S3, Lambda.Minimum ...Show moreLast updated: 26 days ago
- Promoted
Scala Big Data Lead Engineer - 7 YoE - Immediate Joiner - Any UST Location
USTchennai, tamil nadu, in CTC, notice period, and current location details to.Apache Hadoop, Airflow, Kubernetes, and Containers.Troubleshoot Hadoop log files and work with.
Required Skills & Qualifications : .Scala, Spark, Py...Show moreLast updated: 30+ days ago
- Promoted
- New!
Big Data Engineer - Hadoop / Hive
Unison consulting pte ltdChennai Responsibilities : - Design, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databases.Build and maintain scalabl...Show moreLast updated: 4 hours ago
- Promoted
Data Engineer
ImpetusChennai, Tamil Nadu, India Bigdata, Pyspark, Python ,Hadoop / HDFS; Spark;.Develops and maintains scalable data pipelines to support continuing increases in data volume and complexity.
Collaborates with analytics and business...Show moreLast updated: 24 days ago