Talent.com
This job offer is not available in your country.
Associate Staff Engineer - Big Data

Associate Staff Engineer - Big Data

Congregate TechnologiesBangalore
30+ days ago
Job description

About the Role :

We are seeking an experienced Associate Staff Engineer Big Data with strong expertise in LookML, Looker Studio Pro, and Google Cloud Platform (GCP) to lead our data engineering, modeling, and visualization initiatives.

This role demands a hands-on technologist with the ability to design, develop, and optimize large-scale data solutions, ensuring high availability, performance, and scalability.

The candidate will work closely with cross-functional teams including analytics, product, and business stakeholders to build data pipelines, manage cloud infrastructure, and develop advanced data visualization and reporting Responsibilities :

Data Pipeline Development & Optimization :

  • Design, develop, and maintain scalable and high-performance ETL / ELT pipelines using Apache Spark, Python, and related big data frameworks.
  • Ensure robustness, fault tolerance, and optimal resource utilization in distributed data processing.

Data Modeling and Visualization :

  • Build, maintain, and optimize LookML models, Explores, and dashboards within Looker Studio Pro to meet business intelligence and analytics needs.
  • Collaborate with analysts and business teams to translate requirements into actionable data models and visualizations.
  • Cloud Data Infrastructure Management :

  • Deploy, manage, and optimize GCP-based data workflows leveraging services such as BigQuery, Dataflow, Pub / Sub, and Cloud Storage.
  • Monitor, troubleshoot, and enhance data infrastructure for performance, scalability, and cost efficiency.
  • Database and Streaming Systems Management :

  • Work with multiple data storage technologies including PostgreSQL, MongoDB, Kafka, Hadoop, Hive, Pig, and Trino.
  • Ensure data quality, integrity, and security across diverse data systems.
  • Version Control, CI / CD, and Automation :

  • Implement CI / CD practices for data engineering workflows, leveraging Git and industry-standard deployment pipelines.
  • Automate data validation, testing, and deployment processes.
  • Agile Collaboration and Leadership :

  • Actively participate in Agile ceremonies including sprint planning, daily stand-ups, and retrospectives.
  • Mentor junior engineers and contribute to improving engineering best Skills and Technical Competencies : & Big Data Processing :
  • Apache Spark (Strong), Python (Expert)
  • Data Visualization & Modeling :

  • LookML, Looker Studio Pro (Advanced)
  • Cloud Platforms :

  • Google Cloud Platform (GCP) BigQuery, Dataflow, Pub / Sub, IAM
  • Databases & Query Languages :

  • PostgreSQL, MongoDB, Kafka, Hadoop, Hive, Pig, Trino
  • Proficiency in SQL-like languages (SQL, MQL, HQL)
  • DevOps & CI / CD :

  • Experience with Git version control and deployment pipelines
  • Agile Methodology :

  • Prior experience working in Agile / Scrum and Experience :
  • Education : Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.

    Experience : 5 - 8 years of progressive experience in Big Data Engineering, with at least 2 years working on LookML / Looker and GCP projects.

    Certifications (Preferred) : Google Cloud Certified Professional Data Engineer, Looker certifications

    (ref : hirist.tech)

    Create a job alert for this search

    Big Data Engineer • Bangalore

    Related jobs
    • Promoted
    • New!
    Big Data with Quantexa-Pan India

    Big Data with Quantexa-Pan India

    Tata Consultancy Serviceshosur, tamil nadu, in
    We are looking for Big data developer with Quantexa as mandatory skill.Experience Range- 5 to 12 years.Data Engineer - Big Data Hadoop, Spark and Scala. Spark, Scala, Hadoop, HIVE, Oozie, Shell Scri...Show moreLast updated: 6 hours ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Saras Analyticshosur, tamil nadu, in
    We are an ecommerce focused end to end data analytics firm assisting enterprises & brands in data driven decision making to maximize business value. Our suite of work spans extraction, transformatio...Show moreLast updated: 24 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Otomeyt AIhosur, tamil nadu, in
    We are seeking a highly skilled 7+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 18 days ago
    • Promoted
    Azure Data Engineer

    Azure Data Engineer

    Tata Consultancy Serviceshosur, tamil nadu, in
    Job Title : Azure Data Engineer.Required Skillset : Azure Databricks(Strong),scala(strong), Pyspark(Good to have).Azure Databricks using Scala, PySpark. Software Development Life Cycle experience.Work...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Manuh Technologieshosur, tamil nadu, in
    Strong proficiency in Python programming (Pandas, NumPy, PySpark, or similar).Hands-on experience with Dask for large-scale distributed data processing. Proven expertise as a Data Modeler (conceptua...Show moreLast updated: 20 days ago
    • Promoted
    Staff Data and ML Platform Engineer

    Staff Data and ML Platform Engineer

    Palo Alto NetworksBengaluru, Karnataka, India
    At Palo Alto Networks® everything starts and ends with our mission : .Being the cybersecurity partner of choice, protecting our digital way of life. Our vision is a world where each day is safer and m...Show moreLast updated: 30+ days ago
    • Promoted
    • New!
    Data Engineer

    Data Engineer

    TechVeritohosur, tamil nadu, in
    You will play a critical role in designing, building, and optimizing data workflows that enable scalable analytics and real-time insights. The ideal candidate is hands-on, detail-oriented, and passi...Show moreLast updated: 6 hours ago
    • Promoted
    Staff / Principal Data Engineer

    Staff / Principal Data Engineer

    Credit Saison IndiaBengaluru, Karnataka, India
    Established in 2019, CS India is one of the country’s fastest growing Non-Bank Financial Company (NBFC) lenders, with verticals in wholesale, direct lending and tech-enabled partnerships with Non-B...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    VAANTECHhosur, tamil nadu, in
    Immediate Joiner (or within 15 days).Includes Night Shifts (US Shift).Preferred Candidates : From Chennai.Data Pipeline Optimization & Tuning. Hadoop Infra / Cloud Platforms.Lead and mentor a team of...Show moreLast updated: 28 days ago
    • Promoted
    Staff Data Engineer

    Staff Data Engineer

    Commonwealth BankBengaluru, Karnataka, India
    At CommBank, we never lose sight of the role we play in other people’s financial wellbeing.Our focus is to help people and businesses move forward to progress. To make the right financial decisions ...Show moreLast updated: 17 days ago
    • Promoted
    Associate Architect – Data (Azure Databricks)

    Associate Architect – Data (Azure Databricks)

    QuantiphiBengaluru, Karnataka, India
    Associate Architect - Data Engineering.Bangalore / Mumbai (any) (Hybrid WFO).We are looking for a skilled Data Engineers with hands-on experience in the Microsoft Azure. The ideal candidate will pla...Show moreLast updated: 15 days ago
    • Promoted
    Staff AI Engineer

    Staff AI Engineer

    Foodsmarthosur, tamil nadu, in
    Foodsmart is the leading telenutrition and foodcare solution, backed by a robust network of Registered Dietitians.Our platform is designed to foster healthier food choices, drive lasting behavior c...Show moreLast updated: 25 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    CEShosur, tamil nadu, in
    As part of our data engineering team, you’ll build and maintain.This is a hands-on role where you’ll design and optimize modern data infrastructure, ensuring reliability, scalability, and performan...Show moreLast updated: 5 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kanerika Inchosur, tamil nadu, in
    Following are high level responsibilities that you will play but not limited to : .Analyze the Data Model and do GAP analysis with Business Requirements and Power BI. Design and Model Power BI schema....Show moreLast updated: 19 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    ImpetusBengaluru, Karnataka, India
    Indore, Noida, Gurgaon, Bangalore, Hyderabad, Pune.Immediate Joiners are preferred.Big Data technologies – pySpark (Data frame and SparkSQL), Hadoop, and Hive. Good hands-on experience of python and...Show moreLast updated: 30+ days ago
    • Promoted
    Associate Architect - Data Engineering

    Associate Architect - Data Engineering

    Response Informaticshosur, tamil nadu, in
    We are seeking an experienced Data Architect to lead the transformation of enterprise data.Alteryx workflows into Azure Databricks. Microsoft Azure ecosystem, including Azure.Data Factory, Databrick...Show moreLast updated: 26 days ago
    • Promoted
    Staff Engineer

    Staff Engineer

    OnArrivalhosur, tamil nadu, in
    OnArrival is redefining the travel tech industry by building the world’s most advanced full-stack travel platform.We provide seamless, intelligent travel infrastructure, powering everything from fl...Show moreLast updated: 26 days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgehosur, tamil nadu, in
    Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanc...Show moreLast updated: 26 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kumaran Systemshosur, tamil nadu, in
    Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 7 days ago
    • Promoted
    Sr. Full Stack AI Engineer

    Sr. Full Stack AI Engineer

    ContexQhosur, tamil nadu, in
    Senior Full Stack Engineer – Enterprise Analytics Platform.Remote (Preference for India-based candidates).Competitive salary + significant equity package. At ContexQ, a Singapore based B2B SaaS AI S...Show moreLast updated: 30+ days ago