Job Location- Bangalore, Chennai, Gurgoan, Pune, Kolkata
Work Mode : Hybrid
Exp Level : 5+ Years
Mandatory Skills -Azure Databricks +SQL +Pyspark
Primary Roles and Responsibilities :
- Developing Modern Data Warehouse solutions using Databricks and Azure Stack
- Ability to provide solutions that are forward-thinking in data engineering and analytics space
- Collaborate with DW / BI leads to understand new ETL pipeline development requirements.
- Triage issues to find gaps in existing pipelines and fix the issues
- Work with business to understand the need in reporting layer and develop data model to fulfill reporting needs
- Help joiner team members to resolve issues and technical challenges.
- Drive technical discussion with client architect and team members
- Orchestrate the data pipelines in scheduler via Airflow
Skills and Qualifications :
Bachelor's and / or master’s degree in computer science or equivalent experience.Must have experience in Data warehouse / ETL projects.Deep understanding of Star and Snowflake dimensional modelling.Strong knowledge of Data Management principlesGood understanding of Databricks Data & AI platform and Databricks Delta Lake ArchitectureShould have hands-on experience in SQL , Python and Spark (PySpark)Candidate must have experience in Azure stackDesirable to have ETL with batch and streaming (Kinesis).Experience in building ETL / data warehouse transformation processesExperience with Apache Kafka for use with streaming data / event-based dataExperience with other Open-Source big data products Hadoop (incl. Hive, Pig, Impala)Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)Experience working with structured and unstructured data including imaging & geospatial data.Experience working in a Dev / Ops environment with tools such as Terraform, CircleCI, GIT.Proficiency in RDBMS, complex SQL, PL / SQL, Unix Shell Scripting, performance tuning and troubleshootDatabricks Certified Data Engineer Associate / Professional Certification (Desirable).Comfortable working in a dynamic, fast-paced, innovative environment with several ongoing concurrent projectsShould have experience working in Agile methodologyStrong verbal and written communication skills.Strong analytical and problem-solving skills with a high attention to detail.