Talent.com
Advanced PySpark

Advanced PySpark

TEKNIKOZvapi, gujarat, in
1 day ago
Job description

Greetings from Teknikoz

Experience : 9+Years

Roles and Responsibilities

Advanced PySpark (Expert Level - Required)

  • Mastery of DataFrame APIs and Spark SQL
  • Performance tuning : broadcast joins, caching strategies, shuffle reduction
  • Handling skewed data and optimizing wide transformations
  • Writing scalable, testable PySpark code with modular design

Databricks Expertise (Expert Level - Required)

  • Cluster sizing, autoscaling, and spot instance optimization
  • Job orchestration using Workflows with retries, alerts, and dependencies
  • Delta Lake :
  • OPTIMIZE with Z-Ordering
  • MERGE operations and schema evolution
  • Time travel and versioning
  • Use of Unity Catalog for data governance and access control
  • Monitoring via Databricks metrics and audit logs

    • PostgreSQL (Required)

  • Query profiling and execution plan analysis
  • Indexing strategies for analytical workloads
  • Table partitioning and parallel query execution
    • Create a job alert for this search

    Pyspark • vapi, gujarat, in

    Related jobs
    • Promoted
    • New!
    Senior AppDynamics Observability SME

    Senior AppDynamics Observability SME

    Dexian Indiavapi, gujarat, in
    Position Title : Senior AppDynamics Observability SME.IT operations, system administration, or engineering.Ansible, Jenkins, Terraform, Python to develop configuration, deployment, and orchestration...Show moreLast updated: 7 hours ago
    • Promoted
    Senior Engineer

    Senior Engineer

    Brilliovapi, gujarat, in
    Senior Python Developer( Contract to Hire).Logstash alternatives, custom ETL pipelines.AWS, GCP, Azure) for log retrieval and monitoring automation. Experience with streaming data frameworks (Kafka,...Show moreLast updated: 11 days ago
    • Promoted
    Lead Software Engineer - Python (Storage)

    Lead Software Engineer - Python (Storage)

    DDNvapi, gujarat, in
    This is an incredible opportunity to be part of a company that has been at the forefront of AI and high-performance data storage innovation for over two decades. DataDirect Networks (DDN) is a globa...Show moreLast updated: 11 days ago
    • Promoted
    • New!
    Platform Engineer – Python / Databricks / Notebooks / Kubernetes

    Platform Engineer – Python / Databricks / Notebooks / Kubernetes

    Syntasavapi, gujarat, in
    Platform Engineer – Python / Databricks / Notebooks / Kubernetes.Syntasa is seeking a high-caliber and dedicated.Syntasa Technologies India Private Limited. This position offers an exciting opportuni...Show moreLast updated: 7 hours ago
    • Promoted
    Senior Software Engineer

    Senior Software Engineer

    Sympera AIvapi, gujarat, in
    Join a well-funded fintech startup, led by a successful serial fintech entrepreneur.We aim to revolutionize f inancial services with our advanced LLM-based platform. This is a unique opportunity to ...Show moreLast updated: 5 days ago
    • Promoted
    Sap Advanced Business Application Programming Consultant

    Sap Advanced Business Application Programming Consultant

    ARA Resources Pvt. Ltd.vapi, India
    ARA’s client TechGlobal (pseudonym) is a global leader in strategy, technology consulting and digital services, enabling clients in more than 100+ countries to implement cutting edge solutions in t...Show moreLast updated: 5 days ago
    • Promoted
    Quality Engineer

    Quality Engineer

    Magnify Industries Pvt LtdValsad, Gujarat, India
    This comapny is engagged in manufacturing of Wire Harness and Control Boxes.This is a full-time on-site role for a Quality Engineer located in Valsad. The Quality Engineer will be responsible for de...Show moreLast updated: 30+ days ago
    • Promoted
    Contract Role - 6 months to 1 year - Senior Data Engineer - Python, AWS, Building Data Pipelines

    Contract Role - 6 months to 1 year - Senior Data Engineer - Python, AWS, Building Data Pipelines

    CareerXperts Consultingvapi, gujarat, in
    Our client is looking for a talented experienced software development consultants, specializing in Agile Delivery.A place where you are a polyglot, working alongside the client,.A world where you a...Show moreLast updated: 1 day ago
    • Promoted
    AWS Data Engineer

    AWS Data Engineer

    TerraGiGvapi, gujarat, in
    Design, development, and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR. Writing reusable, testable, and efficient code.Integration of data storage ...Show moreLast updated: 1 day ago
    • Promoted
    Senior Python Developer

    Senior Python Developer

    Aarav Solutionsvapi, gujarat, in
    Aarav Solutions is a global Product Engineering and IT Consulting services provider.Our goal is to enable organizations in their digital transformation journey, leveraging our domain knowledge and ...Show moreLast updated: 7 days ago
    • Promoted
    Data Engineer(Azure / AWS, Python / Pyspark, SQL)

    Data Engineer(Azure / AWS, Python / Pyspark, SQL)

    Sail Analyticsvapi, gujarat, in
    Architect, develop, test and maintain scalable data warehouses and data pipelines.Expertise in SQL, PySpark / Python and Azure(ADB, ADF) or AWS(Glue, Lambda, Redshift). Bachelor's degree or equivalent...Show moreLast updated: 20 days ago
    • Promoted
    AEM Engineer

    AEM Engineer

    Insight Globalvapi, gujarat, in
    Adobe Experience Manager (AEM).The team is now expanding AEM capabilities beyond web content into.This is a hands-on development role within a fast-moving team focused on building and optimizing cu...Show moreLast updated: 3 days ago
    • Promoted
    Sap Advanced Business Application Programming Developer

    Sap Advanced Business Application Programming Developer

    Droisysvapi, gujarat, in
    Droisys is an innovation technology company focused on helping companies accelerate their digital initiatives from strategy and planning through execution. We leverage deep technical expertise, Agil...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Python Engineer

    Senior Python Engineer

    Acorn Globusvapi, gujarat, in
    This is a full-time remote role for a Python Developer at Acorn Globus.The Python Developer will be responsible for back-end web development, software development, object-oriented programming (OOP)...Show moreLast updated: 20 days ago
    • Promoted
    Python AWS Data Engineer

    Python AWS Data Engineer

    Digitrix Software LLPvapi, gujarat, in
    Mandatory skills - python , pyspark, who can write codes , any cloud exp - aws / gcp / azure.Python, AWS Python (core language skill) Backend, Pandas, PySpark (DataFrame API), interacting with AW...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Python Developer

    Senior Python Developer

    Skedara Technologyvapi, gujarat, in
    ETL and data integration pipelines.This ideal candidate will be working with data from diverse sources, transforming it efficiently, and ensuring seamless downstream delivery to analytics and repor...Show moreLast updated: 7 days ago
    • Promoted
    Solutions Engineer – Onboarding & Implementation

    Solutions Engineer – Onboarding & Implementation

    OnArrivalvapi, gujarat, in
    OnArrival is the AWS of travel—powering flights, hotels, insurance, and more via modular APIs and SDKs.We enable fintechs, banks, and large ecosystems to launch embedded travel experiences in under...Show moreLast updated: 20 days ago
    • Promoted
    Production Planning Control Manager

    Production Planning Control Manager

    Lathia RubberPardi, Gujarat, India
    The Production Planner is responsible to ensure the smooth operations throughout the production & distribution life of the product. The role is also responsible for organizing paperwork, such as pur...Show moreLast updated: 22 days ago