PySpark Developer
ConfidentialPune6 days ago
Job descriptionExperience in building or integrating APIs for data provisioning. Exposure to ETL or reporting tools such as Informatica, Tableau, Jasper, or QlikView. Familiarity with AI / ML model development using PySpark in cloud environments Skills : ci / cd,zeppelin,pycharm,pyspark,etl tools,control-m,unit test cases,tableau,performance tuning,jenkins,qlikview,informatica,jupyter notebook,api integration,unix / linux,git,aws s3,hive,cloudera,jasper,airflow,cdc,pyspark, apache spark, python, aws s3, airflow / control-m, sql, unix / linux, hive, hadoop, data modeling, and performance tuning,agile methodologies,aws,s3,data modeling,data validation,ai / ml model development,batch integration,apache spark,python,etl pipelines,shell scripting,hortonworks,real-time integration,hadoop. Mandatory Key Skills - Apache Spark,Python,ETL,Unix,Linux,data engineering,Agile methodologies,CI / CD,data modeling,data validation,PySpark.
Key Skills & Responsibilities
- Strong expertise in PySpark and Apache Spark for batch and real-time data processing.
- Experience in designing and implementing ETL pipelines, including data ingestion, transformation, and validation.
- Proficiency in Python for scripting, automation, and building reusable components.
- Hands-on experience with scheduling tools like Airflow or Control-M to orchestrate workflows.
- Familiarity with AWS ecosystem, especially S3 and related file system operations.
- Strong understanding of Unix / Linux environments and Shell scripting.
- Experience with Hadoop, Hive, and platforms like Cloudera or Hortonworks.
- Ability to handle CDC (Change Data Capture) operations on large datasets.
- Experience in performance tuning, optimizing Spark jobs, and troubleshooting.
- Strong knowledge of data modeling, data validation, and writing unit test cases.
- Exposure to real-time and batch integration with downstream / upstream systems.
- Working knowledge of Jupyter Notebook, Zeppelin, or PyCharm for development and debugging.
- Understanding of Agile methodologies, with experience in CI / CD tools (e.g., Jenkins, Git).
Preferred Skills
Skills Required
Pyspark, Hadoop, Spark SQL, Aws, Scala
Create a job alert for this search
Pyspark Developer • Pune
Related jobs
- Promoted
Python Developer
IntraEdgePune, IN We are building a scalable AWS-based backend application that integrates with CrowdStrike and Palo Alto APIs to process and enforce Host / IP block requests.
The application will log and audit each ac...Show moreLast updated: 30+ days ago
- Promoted
Senior Python Developer
Aarav Solutionspune, maharashtra, in Aarav Solutions is a global Product Engineering and IT Consulting services provider.Our goal is to enable organizations in their digital transformation journey, leveraging our domain knowledge and ...Show moreLast updated: 19 days ago
- Promoted
Scala Developer
Tata Consultancy ServicesPune, Maharashtra, India Minimum 5+ years of experience in development of Spark Scala.Experience in designing and development of solutions for Big Data using Hadoop ecosystem technologies such as with Hadoop Bigdata compon...Show moreLast updated: 30+ days ago
- Promoted
Lead Python Developer_ Exp : 6+ Years
Atyeti IncPune, Maharashtra, India Job Description : Strong hands-on experience in Python.Strong experience on Django / Flask framework development.Solid understanding of OOP, REST APIs, and microservices architecture.Exposure t...Show moreLast updated: 2 days ago
- Promoted
- New!
Python Developer
SystemBenderPune, IN We’re Hiring : Python Developers (API Development – AWS Lambda).Are you a Python expert who loves building scalable, cloud-native APIs?.
Join our remote engineering team to design and deploy intellig...Show moreLast updated: 21 hours ago
- Promoted
Senior Python Developer
Huptech HR SolutionsPune, IN Job Post : - Senior Python Developer.Software Development : Write clean, scalable, and efficient Python code to develop software applications.
Backend Development : Design, develop, and maintain server...Show moreLast updated: 2 days ago
- Promoted
- New!
Python Lead Developer
People Prime WorldwidePune, IN Our client is one of the world's fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems.
They helps customers in two ways : Working with the world’s leading ...Show moreLast updated: 11 hours ago
- Promoted
- New!
Python Developer
ProDT Consulting ServicesPune, Maharashtra, India Job Title : Tech Lead – Python Full Stack Developer (AI & SaaS).Contact Person -Anuja Deshmukh.Tech Lead with strong Python Full Stack expertise.
The ideal candidate should have hands-on experience l...Show moreLast updated: 20 hours ago
- Promoted
Sr Python Developer
ACL DigitalPune, India Position : Sr Python Developer Exp : 3-5 yrs.Mandatory Skills Set : Python Development, Rest API, Linux.Having experience on designing REST API's.
Experience working with databases, preferably with k...Show moreLast updated: 30+ days ago
- Promoted
SAP Datasphere Developer
Quantum Integratorspune, maharashtra, in Role : SAP Datasphere Developer.Location : Pune / Nagpur, Maharashtra (Work From Office, Hybrid can be done).Job Type : Full-time with Quantum Integrators.
SAP with 1-2 experience in building and impleme...Show moreLast updated: 21 days ago
- Promoted
Senior Python Developer
IntelliasPune, IN We seek a Senior Python Software Engineer to join our Recommendations team and bring new features to life, and support our products' move to modern architecture.
You will work closely with the produ...Show moreLast updated: 2 days ago
- Promoted
- New!
Senior Python Developer
Intellify SolutionsPune, IN We are looking for an experienced.The ideal candidate should have a solid understanding of modern backend systems, architecture design, and performance optimization.
The role involves working on sca...Show moreLast updated: 11 hours ago
- Promoted
- New!
Senior Python Developer
Whitefield Careerspune, maharashtra, in Bachelor’s / Master’s degree in Computer Science, Engineering, or related field.Excellent communication and leadership skills are a must for this role.
Effectively conveying ideas and creative concept...Show moreLast updated: 8 hours ago
- Promoted
Lead Backend Developer - Python
VXI Global Solutionspune, maharashtra, in Job Title : Backend Developer (Lead).Job Description : We are seeking an experienced Backend Developer to join our team in the role of Lead Developer.
The ideal candidate should have a strong backgrou...Show moreLast updated: 13 days ago
- Promoted
Python Developer
Tredence Inc.pune, maharashtra, in Job Summary : We are looking for a highly skilled Python Developer with strong experience in AWS serverless technologies, particularly AWS Lambda.
The ideal candidate will be proficient in building a...Show moreLast updated: 13 days ago
- Promoted
Pyspark Developer
Tata Consultancy Servicespune, maharashtra, in TCS is conducting Weekday Virtual Drive for the role of Pyspark Developer.Location : Bangalore, Pune, Hyderabad, Gurugram, Chennai.
Years of Experience : 5 to 10 yrs.Notice Period : 0 to 30 / 60 days.Da...Show moreLast updated: 30+ days ago
- Promoted
- New!
Senior Python Developer
Nature Technologies Pvt Ltdpune, maharashtra, in Job Title : Python Developer (3–7 Years Experience).Our teams work on cutting-edge technologies to deliver intelligent, scalable, and high-performing products.
Python-based applications and APIs.Coll...Show moreLast updated: 8 hours ago
- Promoted
Python Developer
UndisclosedPune, Haveli We are looking for a Senior Pro who has worked deeply in Python development and hands-on with Data Engineer activities with strong experience in Python, Dataiku, and AWS to support data processing,...Show moreLast updated: 1 day ago