Talent.com
This job offer is not available in your country.
Lead Data Engineer

Lead Data Engineer

Razorpaybangalore, India
17 hours ago
Job description

The Role :

We are looking for a savvy Lead Data Engineer to join our growing team, responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The right candidate will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.

Roles and Responsibilities :

  • Create and maintain optimal data pipeline architecture
  • Create and maintain events / streaming based architecture / design
  • Identify, design, and implement internal process improvements : automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
  • Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
  • Work with data scientists to strive for greater functionality in our data systems.

Mandatory Qualifications :

  • We are looking for a candidate with 5+ years of experience in a Data Engineer role.
  • Experience with big data tools : HDFS / S3, Spark / Flink,Hive,Hbase, Kafka / Kinesis, etc.
  • Experience with relational SQL and NoSQL databases, including Elasticsearch and Cassandra / Mongodb.
  • Experience with data pipeline and workflow management tools : Azkaban, Luigi, Airflow, etc.
  • Experience with AWS / GCP cloud services
  • Experience with stream-processing systems : Spark-Streaming / Flink etc.
  • Experience with object-oriented / object function scripting languages : Java, Scala, etc.
  • Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
  • Strong analytic skills related to working with structured / unstructured datasets.
  • Build processes supporting data transformation, data structures, dimensional modeling, metadata, dependency, schema registration / evolution and workload management.
  • Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Have a few weekend side projects up on GitHub
  • Have contributed to an open-source project
  • Have worked at a product company
  • Have a working knowledge of a backend programming language
  • Create a job alert for this search

    Lead Data Engineer • bangalore, India

    Related jobs
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    CoffeeBeansBengaluru, Karnataka, India
    Founded in the year 2017, CoffeeBeans specialises in offering high end consulting services in technology, product, and processes. We help our clients attain significant improvement in quality of del...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Data Engineer [T500-20510]

    Lead Data Engineer [T500-20510]

    ANSRBengaluru, Karnataka, India
    ANSR is hiring for one of its clients.Martens is an iconic British brand founded in 1960 in Northamptonshire.Produced originally for workers looking for tough, durable boots, the brand was quickly ...Show moreLast updated: 9 days ago
    • Promoted
    Azure Data Engineer Lead

    Azure Data Engineer Lead

    Creative Synergies GroupBengaluru, Karnataka, India
    Creative Synergies Group Overview.Creative is a leading Global Technology Services firm delivering innovative solutions that build upon synergies between our Deep Engineering Domain Expertise and D...Show moreLast updated: 1 day ago
    • Promoted
    Lead Azure Data Engineer

    Lead Azure Data Engineer

    RandomTreesBengaluru, IN
    We’re a leading software company specializing in Artificial Intelligence, Machine Learning, Data Analytics, Innovative data solutions, Cloud-based technologies. If you're passionate about building r...Show moreLast updated: 27 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    MakeMyTripBangalore Urban, Karnataka, India
    We at MakeMyTrip understand that every traveller is unique and being the leading OTA in India we have the leverage to redefine the travel booking experience to meet their need.If you love to travel...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Generation Team Lead

    Lead Generation Team Lead

    MphasisBangalore Rural, Karnataka, India
    Job Title : Lead Generation Team Lead.We are seeking a dynamic and experienced.With 10 years of experience in the ITES sector, the ideal candidate will lead a team of analysts and specialists to dri...Show moreLast updated: 21 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Deltacubeshosur, tamil nadu, in
    Build and maintain scalable ETL / ELT pipelines.Work with Snowflake and BigQuery for data storage.Implement orchestration with Airflow or Prefect. Integrate data workflows with Python.Optimize data pi...Show moreLast updated: 16 days ago
    • Promoted
    Senior Data Engineer

    Senior Data Engineer

    Otomeyt AIhosur, tamil nadu, in
    We are seeking a highly skilled 7+.The ideal candidate will have strong technical expertise in Azure, Data Engineering tools, and advanced ETL design along with excellent communication and problem-...Show moreLast updated: 19 days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Genisys GroupBengaluru, Karnataka, India
    Take ownership of pipeline stability and performance across our GCP-based stack (BigQuery, GCS, Dataprep / Dataflow).Lead the enhancement of our existing ETL workflows to support better modularity, r...Show moreLast updated: 30+ days ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    ThinkWise Consulting LLPBengaluru, Karnataka, India
    We are seeking a Lead Data Engineer to serve as the technical lead for the team responsible for managing and engineering customer data. The ideal candidate will have a strong background in data engi...Show moreLast updated: 1 day ago
    • Promoted
    Lead Data Engineer

    Lead Data Engineer

    Eucloid Data SolutionsBengaluru, IN
    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications.The ideal candidate will support development of data infrastructure on Databricks...Show moreLast updated: 9 days ago
    • Promoted
    Technical Lead - Data Engineer

    Technical Lead - Data Engineer

    Visara PartnersBengaluru, Karnataka, India
    Location : Bengaluru / Hyderabad.Databricks workflow & Unity catalog.Python programming & package builds.Hands-on experience in designing and building scalable data pipelines using Databricks with ...Show moreLast updated: 6 days ago
    • Promoted
    Data Engineer

    Data Engineer

    INFEC Serviceshosur, tamil nadu, in
    Design, develop, and optimize data pipelines and ETL processes on GCP or Azure.Work with structured and unstructured data, integrating sources such as databases, APIs, and streaming platforms.Imple...Show moreLast updated: 5 days ago
    • Promoted
    Data Engineer

    Data Engineer

    VAANTECHhosur, tamil nadu, in
    Immediate Joiner (or within 15 days).Includes Night Shifts (US Shift).Preferred Candidates : From Chennai.Data Pipeline Optimization & Tuning. Hadoop Infra / Cloud Platforms.Lead and mentor a team of...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kanerika Inchosur, tamil nadu, in
    Following are high level responsibilities that you will play but not limited to : .Analyze the Data Model and do GAP analysis with Business Requirements and Power BI. Design and Model Power BI schema....Show moreLast updated: 20 days ago
    • Promoted
    Data Engineer

    Data Engineer

    Everisehosur, tamil nadu, in
    Join us on our mission to elevate customer experiences for people around the world.As a member of the Everise family, you will be part of a global experience company that believes in being people-f...Show moreLast updated: 7 days ago
    • Promoted
    Data Engineer Team Lead

    Data Engineer Team Lead

    SGIhosur, tamil nadu, in
    To be discussed based on your skills and experience.Strong hands-on data engineering experience with a proven ability to design, build, and optimize scalable data pipelines in .Deep technical exper...Show moreLast updated: 7 days ago
    • Promoted
    Data Engineer

    Data Engineer

    IntraEdgehosur, tamil nadu, in
    Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanc...Show moreLast updated: 27 days ago
    • Promoted
    Data Engineer

    Data Engineer

    ACL Digitalhosur, tamil nadu, in
    Design, develop, and optimize Spark-based data pipelines on Databricks for large-scale data processing.Design, develop, and optimize AWS pipeline as applicable. Implement and manage GitHub asset bun...Show moreLast updated: 30+ days ago
    • Promoted
    Data Engineer

    Data Engineer

    Kumaran Systemshosur, tamil nadu, in
    Strong hands-on experience in Databricks.Proven expertise in building and managing data ingestion pipelines.Exposure to Databricks data ingestion jobs along with incident management frameworks.Expe...Show moreLast updated: 8 days ago