Data Engineer

About the Role

As a Data Engineer, you will design, build, and optimize data pipelines and real-time systems that power AI-driven decisioning and analytics.

Responsibilities

Develop and maintain scalable ETL / ELT pipelines using Python, Airflow, and dbt

Build and optimize real-time streaming pipelines using Kafka, RabbitMQ, Spark and event-driven microservices

Leverage AI coding agents (Claude Code, Cursor AI, GitHub Copilot) to accelerate data pipeline development, data transformation, and optimization

Design and manage data storage solutions with PostgreSQL, ClickHouse and Elasticsearch

Define and implement data modeling best practices for efficient and scalable data storage.

Ensure high availability, performance, and security of data infrastructure on AWS cloud

Design and implement BI-ready datasets in ClickHouse optimized for OLAP workloads and reporting use cases.

Develop and manage denormalized schemas and data models to support analytics and amCharts-based reporting.

Optimize query performance, partitioning, and indexing strategies in ClickHouse to handle large-scale analytical workloads.

Implement best practices for data quality, validation, and observability

Work on data ingestion using Airbyte or other connector frameworks

Implement data transformation pipelines supporting analytical and ML workloads

Collaborate with cloud and DevOps teams for containerized deployments (Docker, Kubernetes, AWS)

Support integration of ML workflows and model APIs into production pipelines

Qualifications

6+ years of hands-on experience in data engineering and distributed systems.

Required Skills

Experience with AI-assisted coding tools (Claude Code, GitHub Copilot, Cursor AI)

Strong expertise in ClickHouse for analytics, including data modeling, partitioning, indexing, and query optimization.

Proven ability to design and manage denormalized OLAP datasets to support BI and reporting workloads.

Solid understanding of ETL / ELT pipelines and experience with tools such as Airbyte, DBT, Airflow or Python-based transformations.

Proficiency in SQL (advanced queries, performance tuning, window functions, aggregations).

Hands-on experience working with reporting / visualization frameworks (e.g., amCharts or similar libraries).

Strong foundation in data quality, governance, and validation to ensure reliable analytics.

Strong knowledge of real-time data streaming (Kafka, Spark)

Experience with relational and analytical databases (Postgres, ClickHouse, Elasticsearch)

Exposure to ML model integration, MLOps is a strong plus

Strong problem-solving, debugging, and cross-team collaboration skills

Preferred Skills

Experience with relational and analytical databases (Postgres, ClickHouse, Elasticsearch)

Exposure to ML model integration, MLOps is a strong plus