Talent.com
This job offer is not available in your country.
Big Data Engineer (Lead)

Big Data Engineer (Lead)

Infogainbangalore, India
1 day ago
Job description

Big Data Engineer (Lead) with skills Data Engineering, Big Data, Apache Hadoop, Apache Hive, Azure-Infra, SQL, Apache Spark, Azure Datalake for location Any Infogain Base Location (Noida, Gurugram, Bangalore, Mumbai, Pune)

Posted on : July 20, Share on Linkedin Share on Twitter Share on Facebook

ROLES & RESPONSIBILITIES

Job Profile Summary :

In this role, you will support the Data Engineering team in setting up the Data Lake on Cloud and the implementation of standardized Data Model, single view of customer. You will develop data pipelines for new sources, data transformations within the Data Lake , implementing graphql , work on no sql database, CI / CD and data delivery as per the business requirements.

Job Description :

  • Build pipelines to bring in wide variety of data from multiple sources within the organization as well as from social media and public data sources.
  • Collaborate with cross functional teams to source data and make it available for downstream consumption.
  • Work with the team to provide an effective solution design to meet business needs.
  • Ensure regular communication with key stakeholders, understand any key concerns in how the initiative is being delivered or any risks / issues that have either not yet been identified or are not being progressed.
  • Ensure dependencies and challenges (risks) are escalated and managed. Escalate critical issues to the Sponsor and / or Head of Data Engineering.
  • Ensure timelines (milestones, decisions and delivery) are managed and value of initiative is achieved, without compromising quality and within budget.
  • Ensure an appropriate and coordinated communications plan is in place for initiative execution and delivery, both internal and external.
  • Ensure final handover of initiative to business-as-usual processes, carry out a post implementation review (as necessary) to ensure initiative objectives have been delivered, and any lessons learned are fed into future initiative management processes.

Who we are looking for :

Competencies & Personal Traits

  • Work as a team player
  • Excellent problem analysis skills
  • Experience with at least one Cloud Infra provider (Azure / AWS)
  • Experience in building data pipelines using batch processing with Apache Spark (Spark SQL, Dataframe API) or Hive query language (HQL)
  • Experience in building streaming data pipeline using Apache Spark Structured Streaming or Apache Flink on Kafka & Delta Lake
  • Knowledge of NOSQL databases. Good to have experience in Cosmos DB, Restful API’s and GraphQL
  • Knowledge of Big data ETL processing tools, Data modelling and Data mapping.
  • Experience with Hive and Hadoop file formats (Avro / Parquet / ORC)
  • Basic knowledge of scripting (shell / bash)
  • Experience of working with multiple data sources including relational databases (SQL Server / Oracle / DB2 / Netezza), NoSQL / document databases, flat files
  • Basic understanding of CI CD tools such as Jenkins, JIRA, Bitbucket, Artifactory, Bamboo and Azure Dev-ops.
  • Basic understanding of DevOps practices using Git version control
  • Ability to debug, fine tune and optimize large scale data processing jobs
  • Working Experience

  • 12-15 years of broad experience of working with Enterprise IT applications in cloud platform and big data environments.
  • Professional Qualifications

  • Certifications related to Data and Analytics would be an added advantage
  • Education

  • Master / bachelor’s degree in STEM (Science, Technology, Engineering, Mathematics)
  • Language

  • Fluency in written and spoken English
  • EXPERIENCE

  • 11-12 Years
  • SKILLS

  • Primary Skill : Data Engineering
  • Sub Skill(s) : Data Engineering
  • Additional Skill(s) : Big Data, Apache Hadoop, Apache Hive, Azure-Infra, SQL, Apache Spark, Azure Datalake
  • ABOUT THE COMPANY

    Infogain is a human-centered digital platform and software engineering company based out of Silicon Valley. We engineer business outcomes for Fortune companies and digital natives in the technology, healthcare, insurance, travel, telecom, and retail & CPG industries using technologies such as cloud, microservices, automation, IoT, and artificial intelligence. We accelerate experience-led transformation in the delivery of digital platforms. Infogain is also a Microsoft (NASDAQ : MSFT) Gold Partner and Azure Expert Managed Services Provider (MSP).

    Infogain, an Apax Funds portfolio company, has offices in California, Washington, Texas, the UK, the UAE, and Singapore, with delivery centers in Seattle, Houston, Austin, Kraków, Noida, Gurgaon, Mumbai, Pune, and Bengaluru.

    Create a job alert for this search

    Big Data Engineer • bangalore, India

    Related jobs
    • Promoted
    Principal Data Engineer

    Principal Data Engineer

    Xebiahosur, tamil nadu, in
    We’re Hiring : Principal Data Engineer | Any Xebia Location (Hybrid, 3 days in office per week).Any Xebia Location (Hybrid – 3 days in office per week). Data Engineering with 4+ years team leadership...Show moreLast updated: 23 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Saras Analyticshosur, tamil nadu, in
    We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value. Our suite of work spans extraction, transformatio...Show moreLast updated: 26 days ago
    • Promoted
    Scala Big Data Lead Engineer - 7 YoE - Immediate Joiner - Any UST Location

    Scala Big Data Lead Engineer - 7 YoE - Immediate Joiner - Any UST Location

    USThosur, tamil nadu, in
    CTC, notice period, and current location details to.Apache Hadoop, Airflow, Kubernetes, and Containers.Troubleshoot Hadoop log files and work with. Required Skills & Qualifications : .Scala, Spark, Py...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Otomeyt AIhosur, tamil nadu, in
    We are seeking a highly skilled 7+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 20 days ago
    • Promoted
    Engineer

    Engineer

    Alp Consulting Ltd.hosur, tamil nadu, in
    As a GCP Data Engineer, you will be responsible for designing, building, and maintaining scalable data processing systems on. You will collaborate with cross-functional teams to ensure data is acces...Show moreLast updated: 21 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Bahwan CyberTekhosur, tamil nadu, in
    Job Title : Data Engineer – Google Cloud Platform (GCP).We are seeking a skilled and motivated Data Engineer with hands-on experience in building scalable data pipelines and cloud-native data soluti...Show moreLast updated: 19 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Dexianhosur, tamil nadu, in
    We seek a Senior Data Engineer colleague to join our world-class client, one of the.Employment / Contract Type : Contract. Duration : 1 year (with potential to be extended).Hours per Week : 40 (Full-time...Show moreLast updated: 28 days ago
    • Promoted
    Senior Generative AI Engineer (Databricks Data Lake)

    Senior Generative AI Engineer (Databricks Data Lake)

    Ampstekhosur, tamil nadu, in
    Title : Senior Generative AI Engineer (Databricks Data Lake).We are seeking an experienced Senior Generative AI Engineer with a strong background in Databricks and data lake architectures.This indiv...Show moreLast updated: 2 days ago
    • Promoted
    GCP Big Data Engineer

    GCP Big Data Engineer

    TalentmaticsBengaluru, Karnataka, India
    The ideal candidate will bring strong leadership capabilities, technical depth, and a proven track record of delivering end-to-end big data solutions in cloud environments.Lead and mentor teams in ...Show moreLast updated: 6 days ago
    • Promoted
    Lead Azure Data Engineer

    Lead Azure Data Engineer

    RandomTreeshosur, tamil nadu, in
    We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies. If you're passionate about building r...Show moreLast updated: 28 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Prudent Technologies and Consulting, Inc.hosur, tamil nadu, in
    Snowflake Instance Management : Set up, configure, and manage between three to four new Snowflake data warehouse instances per quarter, tailored to support emerging products and analytics requiremen...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Big Data Engineer

    Senior Big Data Engineer

    Veltrisbangalore, karnataka, in
    Veltris is a Digital Product Engineering Services partner committed to driving technology-enabled transformation across enterprises, businesses, and industries. We specialize in delivering next-gene...Show moreLast updated: 16 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    ThinkWise Consulting LLPBengaluru, Karnataka, India
    Lead Data Engineer Experience - 10+ Years Location - Bengaluru Hybrid Model We are seeking a Lead Data Engineer to serve as the technical lead for the team responsible for managing and engineering...Show moreLast updated: 2 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    CoffeeBeansBengaluru, Karnataka, India
    Founded in the year 2017, CoffeeBeans specialises in offering high end consulting services in technology, product, and processes. We help our clients attain significant improvement in quality of del...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Net Zero Insightshosur, tamil nadu, in
    Net Zero Insights is a leading provider of market intelligence for the Net-Zero Economy, offering comprehensive insights, data, and research to climate innovation professionals worldwide.Our cuttin...Show moreLast updated: 2 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Canopus Infosystems - A CMMI Level 3 Companyhosur, tamil nadu, in
    Python expertise and hands-on experience in handling large datasets, data cleaning, analysis, and visualization.The ideal candidate should be capable of building data pipelines, performing web scra...Show moreLast updated: 19 days ago
    • Promoted
    Bigdata + Cloudera Engineer

    Bigdata + Cloudera Engineer

    FractalBengaluru, India
    About the Company : Fractal is a leading AI and analytics company that helps businesses leverage data to drive growth and innovation. Our mission is to empower organizations with advanced analytics ...Show moreLast updated: 7 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Eucloid Data SolutionsBangalore, IN
    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 9 days ago
    • Promoted
    Reporting Leader (Big Data / MES)

    Reporting Leader (Big Data / MES)

    Tata ElectronicsKolar, Karnataka, India
    Power BI, Tableau, and Grafana.Deliver multiple output format reports meeting OSAT customer needs.Develop portals for internal and external users to access reports with right level of access contro...Show moreLast updated: 9 days ago
    • Promoted
    Senior Data Engineer [T500-18489]

    Senior Data Engineer [T500-18489]

    Albertsons Companies Indiahosur, tamil nadu, in
    About Albertsons Companies India : .At Albertsons Companies India, we're not just pushing the boundaries of technology and retail innovation, we're cultivating a space where ideas flourish and career...Show moreLast updated: 9 days ago