Talent.com
This job offer is not available in your country.
Pyspark Architect

Pyspark Architect

Talent WorxKolkata, West Bengal, India
1 day ago
Job description

Job Description : PySpark Architect

Location : Bangalore,Pune,Kolkata

Experience Required : 7 to 13 Years

Role Overview :

We are seeking a highly skilled and motivated PySpark Manager to lead our data

engineering initiatives. The ideal candidate will have deep expertise in PySpark,

Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be responsible for

designing, building, and managing large-scale data processing solutions while leading a

team of data engineers to deliver high-quality outcomes.

Key Responsibilities :

1. Leadership & Team Management :

  • Lead and mentor a team of data engineers, ensuring high performance and

professional growth.

  • Oversee project timelines, quality, and deliverables for multiple data initiatives.
  • 2. Data Engineering & Development :

  • Design and implement robust data pipelines and workflows using PySpark and
  • Hadoop.

  • Optimize performance and scalability of big data solutions for complex data sets.
  • Develop and integrate solutions with AWS services for cloud-based data processing.
  • 3. Technical Expertise :

  • Leverage expertise in PySpark, Hadoop, and Scala to solve complex technical
  • challenges.

  • Work with Teradata for large-scale data warehousing solutions.
  • Utilize data visualization tools (e.g., Tableau, Power BI) to provide insights and
  • reporting solutions.

    4. Stakeholder Collaboration :

  • Collaborate with cross-functional teams, including business stakeholders and product
  • managers, to understand requirements and deliver solutions.

  • Provide thought leadership on best practices in big data technologies and architecture.
  • Required Skills & Qualifications :

  • Technical Expertise :
  • Proficiency in PySpark, Hadoop, AWS (S3, Glue, EMR, Redshift), and Teradata.
  • Hands-on experience with Scala for data engineering tasks.
  • Familiarity with data visualization tools like Tableau, Power BI, or similar platforms.
  • Experience :
  • 7–13 years of experience in data engineering, with a focus on big data technologies.
  • Proven track record of managing end-to-end data engineering projects and leading
  • teams.

  • Educational Background :
  • Bachelor’s or Master’s degree in Computer Science, Information Technology, or a
  • related field.

  • Soft Skills :
  • Strong analytical and problem-solving skills.
  • Excellent communication and stakeholder management abilities.
  • Ability to work in a fast-paced, dynamic environment.
  • Preferred Skills :

  • Certification in AWS or related big data tools.
  • Experience in real-time data streaming platforms like Kafka.
  • Knowledge of DevOps practices, including CI / CD pipelines and infrastructure as code.
  • Requirements

    Talent Worx is a growing services & recruitment consulting firm, we are hiring for our client which is a Global and leading provider of financial intelligence, data analytics, and AI-driven solutions, empowering businesses worldwide with insights for confident decision-making. Join to work on cutting-edge technologies, drive digital transformation, and shape the future of global markets.

    Create a job alert for this search

    Architect Pyspark • Kolkata, West Bengal, India

    Related jobs
    • Promoted
    Lead / Architect Data and AI Platform

    Lead / Architect Data and AI Platform

    AkkodisKolkata, IN
    Deep experience building on-premises or hybrid data systems (e.Spark, Kafka, Trino, Iceberg, Airflow).Strong understanding of LLM architecture, embeddings, vector search, and agentic workflows.Prov...Show moreLast updated: 23 days ago
    • Promoted
    Cloud Architect

    Cloud Architect

    IntraEdgeKolkata, IN
    Senior Cloud Architecture Engineer.The Senior Cloud Architecture Engineer is responsible for designing, building and maintaining the underlying PaaS systems that products and services run on, with ...Show moreLast updated: 4 days ago
    • Promoted
    Sap Solutions Architect

    Sap Solutions Architect

    Numeric TechnologiesKolkata, IN
    Type : Permanent with Numeric Technologies.Numeric, incorporated in 1996 is a worldwide Business & Information Technology, Consulting and Services company. Headquartered in Chicago, IL with additiona...Show moreLast updated: 4 days ago
    • Promoted
    Salesforce Architect

    Salesforce Architect

    AscendionKolkata, IN
    Job Title : : Salesforce Architect.Domain experience : : Electrification, Robotics.Experience : : Minimum 10 years relevant experience. We are seeking a Salesforce Solution Architect to lead the design...Show moreLast updated: 30+ days ago
    • Promoted
    Technical Architect

    Technical Architect

    LanceSoft, Inc.Kolkata, IN
    We have an immediate opportunity for "Technical Architect" with our client.Interested candidates send me your CV to Kirthika. Job Title : Technical Architect.IT experience, with at least 5 years as a...Show moreLast updated: 4 days ago
    • Promoted
    Principal / Senior Data Architect

    Principal / Senior Data Architect

    AaysKolkata, IN
    Position : Principal / Senior Data Architect.You will act as a key member of the consulting team helping Clients to re-invent their corporate finance function by leveraging advanced analytics.You wil...Show moreLast updated: 19 days ago
    • Promoted
    System Architect

    System Architect

    Crescent Techserviceskolkata, west bengal, in
    Freelance / Contractor / Parttime).Python, FastAPI, Azure, PostgreSQL, ReactJS.SaaS or data platforms, with 5+ years in architectural leadership. Proven production use of LLM-powered features (e.RAG, mu...Show moreLast updated: 4 days ago
    • Promoted
    Salesforce Data Cloud Architect

    Salesforce Data Cloud Architect

    RialtesKolkata, IN
    Job Description : Salesforce Data Cloud Lead.Freelance / Contract - 6 months.We are looking for an experienced.This role requires deep knowledge of. Salesforce Data Cloud (formerly Customer Data Plat...Show moreLast updated: 30+ days ago
    • Promoted
    Power Platform Solution Architect

    Power Platform Solution Architect

    VeriParkKolkata, IN
    We enable financial institutions to become digital leaders.As a professional team of global scale, we work with best clients for great and exciting projects, in an environment where we learn amazin...Show moreLast updated: 30+ days ago
    • Promoted
    Data Solutions Architect

    Data Solutions Architect

    BristleconeKolkata, IN
    We are seeking an accomplished Data Solutions Architect with deep experience in designing, implementing, and optimizing modern cloud data platforms. The ideal candidate will expertly leverage Snowfl...Show moreLast updated: 4 days ago
    • Promoted
    Cloud HCM Functional Architect

    Cloud HCM Functional Architect

    OracleKolkata, West Bengal, India
    Oracle GSC is hiring for Cloud HCM Functional Architect with minimum 8 years of relevant experience.Work Location - PAN India (Pune, Mumbai, Bangalore, Hyderabad, Chennai, Kolkata, Trivandrum, Gand...Show moreLast updated: 22 days ago
    • Promoted
    SAP Cloud Advisory Architect

    SAP Cloud Advisory Architect

    HCLTechkolkata, west bengal, in
    This is a customer-facing role where you will support the sales teams during the entire cycle of a RISE with SAP S / 4HANA Cloud Private Edition. Since it is a customer facing role, there will not be ...Show moreLast updated: 23 days ago
    • Promoted
    SAP Basis Cloud architect Advisory

    SAP Basis Cloud architect Advisory

    HCLTechKolkata, IN
    CAA (Cloud Architect Advisory).This is SAP Cloud Architect position which is elevated role as per career progression / path for SAP Basis consultant. Architect’s primary role / responsibility is to pl...Show moreLast updated: 30+ days ago
    • Promoted
    Data Architect

    Data Architect

    G10XKolkata, IN
    Senior Data Architect (15+ Years Experience).We are looking for a seasoned Data Architect to lead and shape enterprise-wide data initiatives. This role requires a strategic leader with deep technica...Show moreLast updated: 4 days ago
    • Promoted
    Data Architect

    Data Architect

    Infogainkolkata, west bengal, in
    Design, develop, and optimize scalable data pipelines and workflows using Databricks and Apache Spark on AWS.Collaborate with data scientists, analysts, and other stakeholders to transform data int...Show moreLast updated: 8 days ago
    • Promoted
    SAP FICO Solution Architect

    SAP FICO Solution Architect

    Numeric TechnologiesKolkata, IN
    Role : SAP Solution Architect – Finance.Location : India / UAE - Full time / permanent.We are seeking an experienced SAP Solution Architect - Finance with strong expertise in Central Finance (CFIN), ...Show moreLast updated: 10 days ago
    • Promoted
    Oracle Architect

    Oracle Architect

    CognizantKolkata, IN
    Oracle Architect - Hybrid Working.Looking for candidates who are willing to relocate to Sweden.What makes Cognizant a unique place to work? The combination of rapid growth and an international and ...Show moreLast updated: 4 days ago
    • Promoted
    Data Architect

    Data Architect

    InfoBeansKolkata, IN
    Data Architect – Snowflake & Matillion.Snowflake, Matillion, Data Modeling, SQL, Python / Bash, Cloud Platforms (AWS / Azure). Immediate to 15 days preferred.The ideal candidate is proactive, self-direc...Show moreLast updated: 20 days ago
    • Promoted
    Cloud Architect

    Cloud Architect

    iVedha Inc.Kolkata, IN
    Seeking a highly experienced Cloud Architect to design and oversee robust, scalable, and secure.Architect end-to-end cloud solutions (public, private, hybrid) with a focus on reliability, security,...Show moreLast updated: 30+ days ago
    • Promoted
    SAP Data Architect

    SAP Data Architect

    RED GlobalKolkata, IN
    Title : SAP Data Architect Consultant.Type : 12 month initial contract with extensions.We have a current opportunity for a SAP Data Architect on a contract basis. The position will be based in India.Y...Show moreLast updated: 12 days ago