Talent.com
Data Pipeline Architect

Data Pipeline Architect

Randstad EnterpriseRepublic Of India, IN
30+ days ago
Job description

Role : Data Engineer

Location : Remote

Shift Timing : 2 : 00 Pm - 11 : 00 Pm

Experience : 2- 4 years relevant Experience only ( this is a Junior position with us )

Must have skillset :

GCP - 2 years minimum working Experience

Python and Pyspark - 2 years

SQL - 2 years

Excellent communication

Worked with global stakeholders

Who we are :

Randstad Sourceright’s global talent solutions provides instant access to experienced recruitment

and contingent workforce management support by combining technology, analytics and deep

global and local expertise. Our operations consist of the client aligned service delivery teams

operating across RPO, MSP and Blended Workforce Solutions. We are certified as a “great place to

work” for the last 3 consecutive years and are recognized as the best place to work by Glassdoor

Group Objective

The mission of the business intelligence team is to create a data-driven culture that empowers leaders to integrate data into daily decisions and strategic planning. We aim to provide visibility, transparency, and guidance regarding the quantity and quality of results, activities, financial KPIs, and leading indicators to identify trends aimed at data-based decision-making easily.

Position Objective

As a Senior Data Engineer, you will be responsible for designing, architecting, and implementing robust data solutions in a cloud-based environment (GCP). You will partner with other data engineers and technical teams to ensure the availability, reliability, and performance of our data systems.

Position Summary

Programming & Code Writing

  • Architect and build complex data pipelines using advanced cloud data technologies
  • Lead efforts to optimize data pipelines for performance, scalability, and cost-efficiency
  • Define industry best practices for building data pipelines
  • Ensure data security, compliance, and governance standards are met.
  • Partner with leadership team to define and implement agile and DevOps methodologies

Consulting & Partnership

  • Serve as subject matter expert and define data architecture and infrastructure requirements
  • Partner with business analysts to plan project execution including appropriate product and technical specifications, direction, resources, and establishing realistic completion times
  • Understand data technology trends and identify opportunities to implement new technologies and provide forward-thinking recommendations
  • Proactively partner with internal stakeholders to bridge gaps, provide historical references, and design the appropriate processes

    • Troubleshooting & Continuous Improvement

  • Design and implement a robust data observability process
  • Resolve escalated reporting requests and communicate proactively and timely
  • Troubleshoot, and provide technical guidance to resolve issues related to misaligned or inaccurate data or data fields or new customer requirements
  • Maintain new release, migration, and sprint schedules for software upgrades, enhancements, and fixes to aid with product evolution
  • Write QA / QC Scripts to conduct first round of testing and partner with BA team for test validation for new developments prior to moving to production
  • Use industry knowledge & feedback to aid in the development of technology roadmap and future product(s) vision
  • Document standard ways of working via QRGs, intranet pages, and video series

    • Senior activities

  • Drive day-to-day development activities of development team in close collaboration with on-site and off-shore resources, scrum masters and product owners
  • Bootstrapping a data engineering team at an early stage in the team’s evolution
  • Provide leadership on technical front in difficult situations facilitate contentious discussions, and report up when necessary
  • Guide, mentor and coach offshore resources
  • Provide input in forming a long-term data strategy

    • Education

  • Master’s degree in Computer Science / Information Technology or related field, highly preferred

    • Experience

  • Extensive knowledge of BI concepts and related technologies that help drive sustainable technical solutions
  • Extensive Experience with data lakes, ETL and data warehouses
  • Advanced experience of building data pipelines
  • Passion for building quality BI software
  • Project Management and / or process improvement experience highly preferred

    • Knowledge, Skills, and Abilities

  • Polyglot coder and expert level in multiple languages including, Python, R, Java, SQL, relational databases, ERP, DOMO or other data visualization tools i.E. Tableau
  • Advanced and proven experience with Google cloud platform (GCP) is preferred. But experience with Microsoft Azure / Amazon will be considered.
  • Any exposure to Kafka, Spark, and Scala will be an added advantage.
  • Should demonstrate a strong understanding of OOPS concepts and methodologies
  • Expert level understanding of data engineering
  • Intrinsic motivation and problem-solving
  • Proactive leadership, project management, time management, and problem-solving skills
  • Demonstrated continuous improvement, process documentation, and workflow skills
  • Extensive experience with data analysis, modeling, and data pipelining including data cleaning, standardizing, scaling, tuning, scheduling and deployment
  • Experience composing detailed technical documentation and procedures for data models
  • Ability to prioritize and manage multiple projects, tasks, and meeting deadlines while maintaining quality
  • Strong drive and commitment for delivering outstanding results
  • Strong follow up and service orientation

    • Supervisory Responsibility

    ☒ Provides guidance, leadership, or training to junior employees

    ☐ Directly responsible for supervising non-exempt, clerical, or office administrative personnel

    ☐ Directly responsible for supervising exempt, professional, or technical employees

    ☒ Directly responsible for supervising supervisory / managerial employees

    Organizational Structure :

    Job Title this position reports : Manager, Data Engineering

    Create a job alert for this search

    Data Pipeline Architect • Republic Of India, IN

    Related jobs
    • Promoted
    Senior Data Pipeline Architect

    Senior Data Pipeline Architect

    BrightEdgeRepublic Of India, IN
    Senior Software Engineer, Remote.The Software Engineer, Data Ingestion will be a critical individual contributor responsible for designing collection strategies, developing, and maintaining robust ...Show moreLast updated: 29 days ago
    • Promoted
    Principal / Senior Data Architect

    Principal / Senior Data Architect

    AaysNagpur, IN
    Position : Principal / Senior Data Architect.You will act as a key member of the consulting team helping Clients to re-invent their corporate finance function by leveraging advanced analytics.You wil...Show moreLast updated: 30+ days ago
    • Promoted
    GCP Data Architect

    GCP Data Architect

    Tata Consultancy Servicesnagpur, maharashtra, in
    Job Title : - GCP Data Architect.Design and Implement Data Architectures : Architect and build scalable, end-to-end data solutions on GCP, encompassing data ingestion, transformation, storage, and co...Show moreLast updated: 21 days ago
    • Promoted
    Enterprise Data Architect

    Enterprise Data Architect

    DBiz.aiNagpur, IN
    You will ensure that data is modeled, structured and integrated across the organization to support analytics and AI / ML initiatives at scale. You will work closely with business stakeholders and tech...Show moreLast updated: 11 days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    KaarTechChennai, Republic Of India, IN
    Data Pipeline Development : Design, develop, and maintain scalable data pipelines to support various data analytics and reporting needs. Data Integration : Integrate data from multiple sources and ens...Show moreLast updated: 29 days ago
    • Promoted
    Lead Data Pipeline Architect

    Lead Data Pipeline Architect

    MSBC GroupPune, Republic Of India, IN
    As a Data Engineer, your primary role is to create and maintain high-performance data pipelines, warehouses, and cloud-based systems. This includes creating efficient batch and streaming pipelines, ...Show moreLast updated: 21 days ago
    • Promoted
    Backend and Data Pipeline Engineer

    Backend and Data Pipeline Engineer

    JRD SystemsNagpur, IN
    Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    Solytics PartnersPune, Republic Of India, IN
    Solytics Partners is a Global Analytics firm, recognized with multiple industry awards for innovation and excellence.Our team comprises experts with deep knowledge in risk, analytics, AI / ML, AML / FC...Show moreLast updated: 12 days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    BrightEdgeRepublic Of India, IN
    BrightEdge is a global leader in enterprise SEO and content performance solutions, driving AI-powered digital marketing success for the world’s top brands. Our culture is product-first, innovation-d...Show moreLast updated: 30+ days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    EXLChennai, Republic Of India, IN
    Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation.Lead the data validation...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Data Pipeline Architect

    Senior Data Pipeline Architect

    CoffeeBeansPune, Republic Of India, IN
    Hybrid (3 days WFO, 2 days WFH).Tech-lead in one of the feature teams, candidate need to be work along with team lead in handling the team without much guidance. Good communication and leadership sk...Show moreLast updated: 29 days ago
    • Promoted
    AI Data Architect

    AI Data Architect

    Tata Consultancy Servicesnagpur, maharashtra, in
    We are seeking an inventive Data Architect for AI with 8–15 years of experience to lead the strategic design and implementation of enterprise-scale AI solutions. ETL / ELT, Big Data cloud data solutio...Show moreLast updated: 16 days ago
    • Promoted
    Data Architect - Snowflake (Job Location : Kerala)

    Data Architect - Snowflake (Job Location : Kerala)

    Reflections Info Systemsnagpur, maharashtra, in
    We are looking for 10 + year experienced Data Architect with strong background in Snowflake, demonstrating leadership in technical design, architecture, and implementation of complex data solutions...Show moreLast updated: 13 days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    TagChennai, Republic Of India, IN
    We are seeking a highly skilled.This role is a key part of our.Business Intelligence team, responsible for enabling robust data flows that power enterprise dashboards, analytics, and machine learni...Show moreLast updated: 30+ days ago
    • Promoted
    Data Architect

    Data Architect

    Tata Consultancy Servicesnagpur, maharashtra, in
    Greetings from TCS Human Resources Team!!!.Years of experience : 8 to 10 Years.Airflow • : Data pipeline orchestration.Data Form & DBT • : Data architecture and modeling. Architecture Design • : Scalable d...Show moreLast updated: 30+ days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    JRD SystemsRepublic Of India, IN
    Job Role : Backend and Data Pipeline Engineer - Python.We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda.These include new da...Show moreLast updated: 30+ days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    Exult GlobalCochin, Republic Of India, IN
    Proficient in Python (Including popular python packages e.Strong background in distributed data processing and storage (e. Large scale (TBs of data) data engineering skills - Model data, create prod...Show moreLast updated: 30+ days ago
    • Promoted
    Data Pipeline Architect

    Data Pipeline Architect

    VAANTECHChennai, Republic Of India, IN
    Immediate Joiner (or within 15 days).Includes Night Shifts (US Shift).Preferred Candidates : From Chennai.Data Pipeline Optimization & Tuning. Hadoop Infra / Cloud Platforms.Lead and mentor a team of...Show moreLast updated: 30+ days ago