Talent.com
No longer accepting applications
Advanced PySpark

Advanced PySpark

TEKNIKOZDindigul, IN
14 hours ago
Job description

Greetings from Teknikoz

Experience : 9+Years

Roles and Responsibilities

Advanced PySpark (Expert Level - Required)

  • Mastery of DataFrame APIs and Spark SQL
  • Performance tuning : broadcast joins, caching strategies, shuffle reduction
  • Handling skewed data and optimizing wide transformations
  • Writing scalable, testable PySpark code with modular design

Databricks Expertise (Expert Level - Required)

  • Cluster sizing, autoscaling, and spot instance optimization
  • Job orchestration using Workflows with retries, alerts, and dependencies
  • Delta Lake :
  • OPTIMIZE with Z-Ordering
  • MERGE operations and schema evolution
  • Time travel and versioning
  • Use of Unity Catalog for data governance and access control
  • Monitoring via Databricks metrics and audit logs

    • PostgreSQL (Required)

  • Query profiling and execution plan analysis
  • Indexing strategies for analytical workloads
  • Table partitioning and parallel query execution
    • Create a job alert for this search

    Pyspark • Dindigul, IN