Job Description : Eucloid is seeking a Lead Data Engineer to support various business applications. The ideal candidate will participate in activities including technology selection, designing and building data infrastructure on Databricks for clients. This involves integrating data from various sources, managing big data pipelines with optimized performance.
Key Responsibilities :
- Design, implement, and improve processes & automation of Data infrastructure
- Tune Data pipelines for reliability & performance
- Build tools and scripts to develop, monitor, troubleshoot ETL's
- Perform scalability, latency, and availability tests on a regular basis
- Investigate, analyze, correct and document reported data defects
- Create and maintain technical specification documentation
Qualifications :
B.Tech / BS degree in Computer Science, Computer Engineering, Statistics, or other Engineering disciplinesMin. 5 years of Professional work experience as a data engineerHighly proficient in SQL & Data model (conceptual and logical) conceptsHighly proficient with Python & Spark (3+ year)Knowledge of Distributed computing and cloud databases like Redshift, Big query etc.2+ years of Hands-on experience with AWSExperience with Modern Data stack tools like Airflow, Terraform, dbt, Glue, Dataproc etc.Exposure to Hadoop & Shell scripting a plusBenefits : Eucloid offers a high growth path along with great compensation.