Principal Data Engineer

Roles & Responsibilities :

• Architect and maintain robust, scalable data pipelines using Databricks, Spark, and Delta Lake for both batch and real-time data processing.
• Lead technology evaluation and adoption initiatives to improve productivity, scalability, and data delivery.
• Optimize data processing performance through Spark tuning, job scheduling, and efficient resource utilization.
• Develop innovative solutions to enhance data ingestion, transformation, lineage tracking, and observability.
• Build metadata-driven frameworks to promote pipeline consistency and reuse.
• Promote a culture of engineering excellence, continuous improvement, and experimentation.
• Collaborate with architecture, platform, governance, and analytics teams to support the enterprise data strategy.
• Define and monitor SLOs, KPIs, and data quality metrics for production systems.
• Translate business requirements into scalable, governed data products in partnership with stakeholders.
• Mentor and guide engineers to adopt modern engineering tools and practices.
• Work closely with DevOps, architects, and analysts to ensure alignment of engineering strategies with business objectives.
• Stay current on data technology trends and best practices to continually enhance the data platform architecture.

Must-Have Skills :

Good-to-Have Skills :

Education & Certifications :

Soft Skills :

Skills Required

Databricks, Pyspark, Apache Spark, Sparksql, Aws, Python