Talent.com
Big Data Engineer
Big Data EngineeriDWTeam • Hyderabad, Republic Of India, IN
Big Data Engineer

Big Data Engineer

iDWTeam • Hyderabad, Republic Of India, IN
1 day ago
Job description

We are seeking an experienced Databricks Engineer to join our data team. The ideal candidate will have strong expertise in Databricks, Spark, and cloud platforms (Azure / AWS / GCP), along with a solid understanding of data engineering best practices. You will work closely with cross-functional teams to build scalable data pipelines, optimize big-data workloads, and support advanced analytics initiatives.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark .
  • Build and optimize ETL / ELT workflows for large-scale datasets.
  • Develop Delta Lake tables and manage data quality, reliability, and performance.
  • Integrate data from various sources (APIs, databases, cloud storage).
  • Collaborate with data scientists, analysts, and business stakeholders to support analytics and ML workloads.
  • Implement best practices for CI / CD, version control, and automation.
  • Monitor platform performance and troubleshoot issues in Databricks clusters and jobs.

Required Skills & Experience

  • 3+ years of experience as a Data Engineer or similar role.
  • Hands-on experience with Databricks (Jobs, Workflows, Delta Lake, Unity Catalog, Notebooks).
  • Strong proficiency in Apache Spark (PySpark / Scala) .
  • Experience with at least one cloud platform : Azure, AWS, or GCP .
  • Strong SQL knowledge for data modelling, transformations, and performance tuning.
  • Experience with CI / CD tools (Git, Azure DevOps, Jenkins, etc.).
  • Solid understanding of data warehousing and data-lake architecture.
  • Preferred Qualifications

  • Databricks certification(s) (Data Engineer Associate / Professional).
  • Experience with MLflow or supporting data science workloads.
  • Knowledge of streaming technologies (Structured Streaming / Kafka).
  • Experience with orchestration tools (Airflow, ADF, AWS Glue, etc.).
  • What We Offer

  • Competitive compensation and benefits.
  • Opportunity to work with modern cloud and big-data technologies.
  • Collaborative, growth-oriented team environment.
  • Career development and certification support.
  • Create a job alert for this search

    Big Data Engineer • Hyderabad, Republic Of India, IN

    Related jobs
    Big Data Solutions Engineer

    Big Data Solutions Engineer

    Tata Consultancy Services • Hyderabad, Republic Of India, IN
    Thanks for Applying with TCS for Data Engineer.You will get the interview confirmation call from my Team, Kindly do Available for the. Telephonic interview call for managerial round.Virtual Intervie...Show more
    Last updated: 30+ days ago • Promoted
    Big Data Engineer

    Big Data Engineer

    Saaki Argus & Averil Consulting • Hyderabad, Telangana, India
    Quick Apply
    Notice Period : Immediate Joiner or 30 Day.Hands-on experience on technologies like.Hive, Scala, Spark, AWS, HBase.Proficient in Hadoop distributions such as. Good working experience on technologies...Show more
    Last updated: 30+ days ago
    Data Engineer

    Data Engineer

    TerraGiG • Hyderabad, IN
    Lead the design, development, and implementation of data solutions using AWS and Snowflake.Collaborate with cross-functional teams to understand business requirements and translate them into techni...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    EXL • Hyderabad, IN
    The person will be part of the EDP Data Platform (EDP) team for a major Insurance client.He / She will work with different stakeholders for architecting & Building EDP application platform to suppor...Show more
    Last updated: 30+ days ago • Promoted
    Data Engineer

    Data Engineer

    IntraEdge • Hyderabad, IN
    We are seeking a highly skilled Data Engineer with strong experience in Python, PySpark, Snowflake, and AWS Glue to join our growing data team. You will be responsible for building scalable and reli...Show more
    Last updated: 30+ days ago • Promoted
    Big Data Engineer

    Big Data Engineer

    K&K Talents - India • Hyderabad, IN
    This position is with one of our.Title : Big Data Engineer (Scala & Spark).Required Experience : 8 Years and above.We are seeking a highly skilled. Azure cloud, Big Data technologies, and distributed ...Show more
    Last updated: 3 days ago • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Confidential • Hyderabad, IN
    Expertise in big data technologies such as Apache Spark and real-time streaming technologies like Apache Kafka.Strong programming skills in Python, Java, C++, SQL etc. Advanced knowledge of a major ...Show more
    Last updated: 14 hours ago • Promoted • New!
    Data Engineer

    Data Engineer

    Staffingine LLC • Hyderabad, IN
    The Data Engineer will be responsible for designing, developing, and optimizing scalable data pipelines and cloud-based data solutions. This role requires strong Python programming skills, expertise...Show more
    Last updated: 7 days ago • Promoted
    Data Engineer

    Data Engineer

    BayOne Solutions • Hyderabad, IN
    We are seeking a highly experienced Data Engineer to join our MarTech team and play a pivotal role in driving innovation within our microservices architecture, with a strong emphasis on data engine...Show more
    Last updated: 30+ days ago • Promoted
    Senior Data Engineer - Data Acquisition

    Senior Data Engineer - Data Acquisition

    InfoBeans • Hyderabad, IN
    We are seeking a highly skilled.Senior Data Engineer – Data Acquisition (ODS).The ideal candidate will have extensive hands-on experience in building and optimizing data ingestion and transformatio...Show more
    Last updated: 23 days ago • Promoted
    Data Engineer

    Data Engineer

    MyRemoteTeam Inc • Hyderabad, IN
    MyRemoteTeam, Inc is a fast-growing distributed workforce enabler, helping companies scale with top global talent.We empower businesses by providing world-class software engineers, operations suppo...Show more
    Last updated: 14 hours ago • Promoted • New!
    Immediate Opening for Data Engineer (Databricks & D365)

    Immediate Opening for Data Engineer (Databricks & D365)

    GyanSys Inc. • Hyderabad, IN
    Need to have databricks experience as well as experience with handling D365 data.Need strong SQL skills, strong analytical skills.Show more
    Last updated: 21 days ago • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Ironbook AI • Hyderabad, IN
    We are seeking an experienced and driven Lead Data Engineer to spearhead the.AI use cases across the organization.Minimum 7 years of experience in data engineering, with at.Strong hands-on experien...Show more
    Last updated: 14 hours ago • Promoted • New!
    Data Engineer (Airflow, Azure)

    Data Engineer (Airflow, Azure)

    Luxoft • Greater Hyderabad Area, India
    This project is part of a strategic initiative to migrate legacy on-premises data systems to a modern, scalable cloud-based data platform using Snowflake on Azure. The goal is to enhance data access...Show more
    Last updated: 9 days ago • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Ironbook AI • Hyderabad, IN
    The ideal candidate will have strong experience with cloud platforms, modern ETL / ELT tools, and deep technical skills in Python, SQL, and distributed data frameworks. Design, develop, and maintain s...Show more
    Last updated: 14 hours ago • Promoted • New!
    Data Engineer - Fully Remote (Global Data Platform & Analytics Projects)

    Data Engineer - Fully Remote (Global Data Platform & Analytics Projects)

    SkillsCapital • Hyderabad, IN
    Remote
    These fully remote, long-term freelance roles are ideal for engineers who can build scalable data pipelines, work with modern cloud-native data stacks, and support large-scale enterprise data initi...Show more
    Last updated: 5 days ago • Promoted
    Big Data Engineer

    Big Data Engineer

    Tata Consultancy Services • Hyderabad, Republic Of India, IN
    Greetings from Tata Consulting Services.TCS is Hiring for Big Data Developer.Location : Chennai / Hyderabad / Bangalore.Required Technical Skill - Pyspark, Hadoop,Hive.Show more
    Last updated: 30+ days ago • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Guidanz Inc • Hyderabad, IN
    BI Connector is the industry leading solution for integrating Oracle Fusion Cloud data into modern BI platforms like Power BI, Tableau, and Data Warehouse, without complex ETL.Our Data Architecture...Show more
    Last updated: 1 day ago • Promoted