Talent.com
This job offer is not available in your country.
Big Data Developer

Big Data Developer

IntraEdgeChennai, Tamil Nadu, India
30+ days ago
Job description

Job Title : Senior Big Data Engineer (Python)

Experience : 5+ Years

Locations : Chennai, Bangalore, Gurugram

Employment Type : Full-Time

Notice Period : Immediate to 30 Days Preferred

Job Summary

We are seeking an experienced Big Data Engineer with strong expertise in Python to design, build, and manage large-scale data pipelines and analytics solutions. The ideal candidate will have hands-on experience working with Big Data technologies and cloud platforms, and a passion for writing efficient, scalable, and maintainable code.

Key Responsibilities

  • Design, develop, and maintain scalable big data pipelines using Python and other data processing tools.
  • Work with distributed data processing frameworks like Spark, Hadoop, Hive, or similar.
  • Implement ETL processes for structured and unstructured data from various sources.
  • Collaborate with data scientists, analysts, and other engineering teams to understand data needs.
  • Optimize data workflows and ensure data quality, integrity, and security.
  • Deploy solutions on cloud platforms (AWS / GCP / Azure) and automate data workflows.
  • Monitor data pipelines, troubleshoot issues, and ensure high availability and performance.

Required Skills & Qualifications

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • Minimum 5 years of hands-on experience in Big Data engineering roles.
  • Strong proficiency in Python for data processing and scripting.
  • Experience with Apache Spark , Hadoop ecosystem (Hive, HDFS, HBase), Kafka.
  • Solid understanding of data warehousing concepts and data modeling.
  • Hands-on experience with SQL and NoSQL databases.
  • Familiarity with cloud data platforms like AWS (EMR, S3, Glue) , Azure , or Google Cloud .
  • Good understanding of CI / CD, containerization (Docker, Kubernetes), and version control (Git).

    • Preferred Skills (Nice to Have)

  • Experience with Airflow or other workflow orchestration tools.
  • Knowledge of real-time data processing (e.g., Spark Streaming, Flink).
  • Exposure to data governance, data lineage, and catalog tools.

    • Why Join Us?

  • Opportunity to work on cutting-edge Big Data solutions with global clients.
  • Collaborative and innovation-driven work culture.
  • Competitive compensation and career growth opportunities.
  • Flexible work environment with hybrid (based on role).
    • Create a job alert for this search

    Big Data Developer • Chennai, Tamil Nadu, India