Senior Data Engineer
Position Overview
We are seeking an experienced Senior Data Engineer with 8-12 years of hands-on experience to join our team on-site. This role requires a self-starter who can architect and implement robust data foundation layers while providing technical leadership and guidance to the engineering team. The ideal candidate will have deep expertise in modern data platforms, particularly Databricks / Delta Lake, and a strong understanding of how to build scalable, reliable data infrastructure that powers enterprise analytics and AI initiatives.
Key Responsibilities
Data Foundation & Architecture
- Design, build, and maintain scalable data foundation layers that serve as the backbone for analytics, reporting, and AI / ML initiatives
- Architect end-to-end data solutions including storage, integration, pipelines, and modeling components
- Establish best practices and standards for data architecture across the organization
- Evaluate and recommend new technologies and tools to enhance the data platform
Data Storage & Management
Design and implement data storage solutions using modern cloud-based platformsBuild and optimize data warehouses, data lakes, and lakehouse architecturesImplement data partitioning, indexing, and optimization strategies for performance at scaleEnsure data availability, reliability, and disaster recovery capabilitiesData Pipelines & Integration
Develop robust, scalable ETL / ELT pipelines for batch and real-time data processingBuild data integration frameworks connecting diverse data sources (APIs, databases, files, streaming)Implement data quality checks and monitoring throughout the pipelineOptimize pipeline performance and troubleshoot production issuesAutomate data workflows and orchestration processesData Modeling
Design conceptual, logical, and physical data models aligned with business requirementsImplement dimensional modeling (star / snowflake schemas) for analytics use casesCreate and maintain data dictionaries and documentationEnsure data models support both operational and analytical workloadsTeam Leadership & Collaboration
Mentor and guide junior and mid-level data engineers on technical best practicesConduct code reviews and ensure high-quality, maintainable code standardsLead technical design discussions and architectural decisionsCollaborate with data scientists, analysts, and business stakeholders to understand requirementsDrive continuous improvement initiatives within the data engineering teamRequired Qualifications
Experience & Education
8-12 years of experience in data engineering or related rolesBachelor's degree in Computer Science, Engineering, Information Systems, or related field (or equivalent experience)Proven track record of building enterprise-scale data platformsExperience in building intelligent AI agents and RAG-based applications along with good exposure to Prompt engineeringExpertise in implementing and optimizations with Databases such as SQL Server, Postgres SQL, Clickhouse etc.,Core Technical Skills
Must Have :
Databricks / Delta Lake : Extensive hands-on experience building data solutions on Databricks platform using Delta Lake for lakehouse architectureData Foundation Layer Expertise : Deep understanding and practical implementation of :o Data storage architectures (data lakes, warehouses, lakehouse)
o Data pipeline development and orchestration
o Data integration patterns and frameworks
o Data modeling techniques and best practices
Programming : Strong proficiency in Python and SQLCloud Platforms : Experience with Azure (or comparable)Big Data Technologies : Spark, Kafka, or similar distributed processing frameworksAI & Modern Tools
Working knowledge of AI tools and their integration with data platformsExperience supporting ML / AI workflows and feature engineering pipelinesSoft Skills
Self-Starter : Ability to work independently, take initiative, and drive projects to completionLeadership : Proven ability to guide and mentor team membersCommunication : Excellent verbal and written communication skills to collaborate with technical and non-technical stakeholdersProblem-Solving : Strong analytical and troubleshooting abilitiesAdaptability : Comfortable working in fast-paced environments with evolving requirements