Must Have skills :Extensive experience with Azure Databricks (ADB), Delta Lake, Azure Data Lake Storage (ADLS), Azure Data factory (ADF), Azure SQL Database (SQL DB), SQL, ELT / ETL Pipeline Development in Spark based environment.Extensive Experience with SparkCore, PySpark, Python, SparkSQL, Scala, Azure Blob Storage.Experience in Real-Time Data Processing using Apache Kafka / EventHub / IoT, Structured Streaming and Stream analytics.Experience with Apache Airflow for ELT Orchestration.Experience with infrastructure management, Infrastructure as code (e.g. Terraform)Experience with CI / CD, Version control tools like GitHub, Azure DevOps.Experience with Version control tools, building CI / CD pipelines.Experience with Azure cloud platform.Good to have :Experience / Knowledge on Containerizations - Docker, KubernetesExperience working in Agile MethodologyQualificationsQualifications - BE, MS, M.Tech or MCAAdditional InformationCertifications : Azure Big Data, Databricks Certified AssociateSkills Required
Pyspark, Python, Sparksql, Scala