Senior Data Engineer

Job Summary :

We are seeking an experienced Data Engineer with strong expertise in AWS cloud services and Databricks to design and implement scalable, high-performance data pipelines. The role involves building and optimizing ETL / ELT workflows, managing big data environments, and ensuring seamless data availability for analytics, AI / ML, and business intelligence use cases.

Key Responsibilities :

Design, develop, and maintain data pipelines and workflows using AWS (S3, Redshift, Glue, EMR, Lambda, Kinesis, Athena) and Databricks .

Build and optimize ETL / ELT processes for large-scale structured and unstructured datasets.

Leverage Databricks for data engineering, advanced analytics, and machine learning workloads.

Collaborate with data scientists, analysts, and stakeholders to deliver reliable and scalable data solutions.

Implement data governance, quality checks, and monitoring frameworks .

Ensure high availability, scalability, and cost-effectiveness of data solutions.

Required Skills :

7+ years of experience in data engineering .

Strong hands-on experience with AWS cloud services (S3, Redshift, Glue, EMR, Athena, Lambda, Kinesis).

Expertise in Databricks (PySpark, Delta Lake, ML pipelines).

Proficiency in Python and SQL for data transformation and querying.

Strong understanding of data warehousing, data modeling, and big data frameworks .

Experience with workflow orchestration tools (Airflow, Step Functions).

Preferred Qualifications :

Familiarity with MLOps / ML integration within Databricks.

Experience with real-time / streaming data (Kafka, Spark Streaming).

Knowledge of infrastructure as code (Terraform / CloudFormation).

Exposure to data security, compliance, and governance practices.