Minimum 5 - 8Years
Skill Set
Databricks Engineer, Azure Databricks, SQL, Pyspark
Skill to Evaluate
Databricks Engineer, Azure Databricks, SQL
Experience
5 to 8 Years
Location
Pune, Maharashtra, India
Job Description
Key Responsibilities :
- Design, build, and maintain scalable data pipelines and ETL processes.
- Collaborate with data scientists and analysts to understand data needs and provide support for their analytics requirements.
- Develop and implement data models, database designs, and data management solutions.
- Ensure data quality, integrity, and consistency across various data sources.
- Optimize data storage, processing, and retrieval for performance and efficiency.
- Monitor and troubleshoot data pipelines to ensure they are running smoothly.
- Stay up to date with industry trends and best practices in data engineering.
- Mentor junior data engineers and promote a culture of knowledge sharing.
Skills and Tools Required :
Strong proficiency in programming languages such as Python, Java, or Scala.Extensive experience with SQL and NoSQL databases, including but not limited to PostgreSQL, MySQL, MongoDB, or Cassandra.Expertise in data warehousing solutions, such as Amazon Redshift, Google BigQuery, or Snowflake.Familiarity with data pipeline orchestration tools like Apache Airflow, Apache NiFi, or Prefect.Knowledge of big data technologies such as Hadoop, Spark, or Kafka.Understanding of cloud platforms, including AWS, Azure, or Google Cloud.Experience with data modeling, data governance, and data security best practices.Strong analytical and problem-solving skills, with attention to detail.Excellent communication skills and ability to work collaboratively in a team environment.