No longer accepting applications

Senior Software Engineer (Immediate Start)

CogniteIndia

9 hours ago

Job description

Cognite is revolutionising industrial data management through our flagship product, Cognite Data Fusion - a state-of-the-art SaaS platform that transforms how industrial companies leverage their data. We're seeking a Senior Data Platform Engineer who excels at building high-performance distributed systems and thrives in a fast-paced startup environment. You'll be working on cutting-edge data infrastructure challenges that directly impact how Fortune 500 industrial companies manage their most critical operational data.

Responsibilities :

High-Performance Data Systems : Design and implement robust data processing pipelines using Apache Spark, Flink, and Kafka for terabyte-scale industrial datasets.
Build efficient APIs and services that serve thousands of concurrent users with sub-second response times.
Optimise data storage and retrieval patterns for time-series, sensor, and operational data.
Implement advanced caching strategies using Redis and in-memory data structures.

Distributed Processing Excellence :

Engineer Spark applications with a deep understanding of Catalyst optimiser, partitioning strategies, and performance tuning

Develop real-time streaming solutions processing millions of events per second with Kafka and Flink.

Design efficient data lake architectures using S3 / GCS with optimised partitioning and file formats (Parquet, ORC).

Implement query optimisation techniques for OLAP datastores like ClickHouse, Pinot, or Druid.

Scalability and Performance :

Scale systems to 10K+ QPS while maintaining high availability and data consistency.

Optimise JVM performance through garbage collection tuning and memory management.

Implement comprehensive monitoring using Prometheus, Grafana, and distributed tracing.

Design fault-tolerant architectures with proper circuit breakers and retry mechanisms.

Technical Innovation :

Contribute to open-source projects in the big data ecosystem (Spark, Kafka, Airflow).

Research and prototype new technologies for industrial data challenges.

Collaborate with product teams to translate complex requirements into scalable technical solutions.

Participate in architectural reviews and technical design discussions.

Requirements :

Distributed Systems Experience (4-6 years) - Production Spark experience - built and optimised large-scale Spark applications with understanding of internals - Streaming systems proficiency - implemented real-time data processing using Kafka, Flink, or Spark Streaming - JVM Language expertise - strong programming skills in Java, Scala, or Kotlin with performance optimisation experience.

Data Platform Foundations (3+ years) - Big data storage systems - hands-on experience with data lakes, columnar formats, and table formats (Iceberg, Delta Lake) - OLAP query engines - worked with Presto / Trino, ClickHouse, Pinot, or similar high-performance analytical databases - ETL / ELT pipeline development - built robust data transformation pipelines using tools like DBT, Airflow, or custom frameworks

Infrastructure and Operations - Kubernetes production experience -deployed and operated containerised applications in production environments. Cloud platform proficiency - hands-on experience with AWS, Azure, or GCP data services.

Monitoring and observability - implemented comprehensive logging, metrics, and alerting for data systems.

Technical Depth Indicators :

Performance Engineering - System optimisation experience - delivered measurable performance improvements (2x+ throughput gains).

Resource efficiency - optimised systems for cost while maintaining performance requirements.

Concurrency expertise - designed thread-safe, high-concurrency data processing systems.

Data Engineering Best Practices - Data quality frameworks -implemented validation, testing, and monitoring for data pipelines.

Schema evolution - managed backwards-compatible schema changes in production systems.

Data modelling expertise - designed efficient schemas for analytical workloads.

Collaboration and Growth :

Technical Collaboration - Cross-functional partnership - worked effectively with product managers, ML engineers, and data scientists.

Codereview excellence - provided thoughtful technical feedback and maintained high code quality standards.

Documentation and knowledge sharing - created technical documentation and participated in knowledge transfer.

Continuous Learning - Technology adoption - quickly learned and applied new technologies to solve business problems.

Industry awareness - stayed current with big data ecosystem developments and best practices.

Problem-solving approach - demonstrated a systematic approach to debugging complex distributed system issues.

Startup Mindset :

Execution Excellence - Rapid delivery - consistently shipped high-quality features within aggressive timelines.

Technical pragmatism - made smart trade-offs between technical debt, velocity, and system reliability.

End-to-end ownership - took responsibility for features from design through production deployment and monitoring.

Ambiguity comfort - thrived in environments with evolving requirements and unclear specifications.

Technology flexibility - adapted to new tools and frameworks based on project needs.

Customer focus - understood how technical decisions impact user experience and business metrics.

Bonus Points :

Open-source contributions to major Apache projects in the data space (e. g. Apache Spark or Kafka) are a big plus.

Conference speaking or technical blog writing experience, Industrial domain knowledge - previous experience with IoT, manufacturing, or operational technology systems.

Technical Stack :

Primary Technologies :

Languages : Kotlin, Scala, Python, Java.

Big Data : Apache Spark, Apache Flink, Apache Kafka.

Storage : PostgreSQL, ClickHouse, Elasticsearch, S3-compatible systems.

Infrastructure : Kubernetes, Docker, Terraform.

Technologies You May Work With :

Table Formats : Apache Iceberg, Delta Lake, Apache Hudi.

Query Engines : Trino / Presto, Apache Pinot, DuckDB.

Orchestration : Apache Airflow, Dagster.

Monitoring : Prometheus, Grafana, Jaeger, ELK Stack.

Create a job alert for this search

Senior Software Engineer • India

Related jobs

Promoted

Senior Software Engineer

ConcentrixNagpur, IN

An understanding of software patterns and practices, affinity for developing unit tests with complete code coverage.More than 8 years exposure to Modern web application development.Minimum 4+ years...Show moreLast updated: 9 days ago

Promoted

Senior Software Engineer

SearginNagpur, IN

The successful candidate will : Be technically opinionated, with demonstrated adherence to best practices in software engineering including appropriate use of abstraction, testing, usability, and ve...Show moreLast updated: 1 day ago

Promoted

Senior Software Engineer

Sympera AINagpur, IN

Join a well-funded fintech startup, led by a successful serial fintech entrepreneur.We aim to revolutionize f inancial services with our advanced LLM-based platform. This is a unique opportunity to ...Show moreLast updated: 4 days ago

Promoted

Senior Software Engineer (Future CTO Pathway)

FileSpinNagpur, IN

Remote-first (UK / EU / India preferred).AI-powered Digital Asset Management SaaS.AI-powered Digital Asset Management (DAM) and Media Processing. We help enterprises scale their content operations with ...Show moreLast updated: 11 days ago

Promoted

Senior Software Engineer

The GloveNagpur, IN

Senior Enterprise Software Engineer.Responsibilities may include the following and other duties may be assigned : .You will read, analyze, understand product requirements. You will actively participat...Show moreLast updated: 11 days ago

Promoted

Senior Software Engineer

StimScience IndiaNagpur, IN

We are seeking a highly skilled.Senior Google Cloud Functions (GCF) App Developer.The ideal candidate will have deep expertise in building scalable, secure, and high-performance serverless architec...Show moreLast updated: 25 days ago

Promoted
New!

Senior Software Engineer I

OptumIndia

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives.The work you do with our team will directly improve health outcomes by connect...Show moreLast updated: 4 hours ago

Promoted

Senior Software Engineer

Cimpress IndiaIndia, India

Cimpress Technology develops cutting-edge, best-in-world software that our mass customization businesses use to create personalized products for millions of global customers.Our Mass Customization ...Show moreLast updated: 30+ days ago

Promoted

Senior Software Engineer

ArrowIndia, India

Arrow is focused on revolutionizing healthcare payments by bringing speed, accuracy, and transparency to the process.Our mission is to simplify healthcare payments for patients, providers, and heal...Show moreLast updated: 25 days ago

Promoted

Senior Software Engineer

BrightEdgeNagpur, IN

Senior Software Engineer, Remote.The Software Engineer, Data Ingestion will be a critical individual contributor responsible for designing collection strategies, developing, and maintaining robust ...Show moreLast updated: 25 days ago

Promoted

Senior Software Engineer

EvlvNagpur, IN

Sector : IT & Services Consulting.Location : Permanent Work From Home (Anywhere in India).Duration : 1-Year - Renewable Contract. Working Hours : Monday to Friday, Eastern Standard Time Zone.Our client ...Show moreLast updated: 20 days ago

Promoted

Senior Software Engineer

MSHIndia, India

As a consultant… You are responsible for delivering an end-to-end solution from initial concept through finished solution. You’ll need the ability to quickly understand needs, from a customer’s pers...Show moreLast updated: 1 day ago

Promoted

Senior Software Engineer (C++ & C#)

KLDiscoveryNagpur, IN

KLDiscovery, a leading global provider of electronic discovery, information governance and data recovery services, is currently seeking a Senior Software Engineer (C++ & C#) for an exciting new opp...Show moreLast updated: 30+ days ago

Promoted

Senior Software Engineer

Programmers.ioNagpur, IN

We are seeking a highly skilled and experienced Senior Azure Data Engineer to join our team.The ideal candidate will have deep expertise in Microsoft Azure data services, cloud-based data engineeri...Show moreLast updated: 11 days ago

Promoted

Senior Software Engineer

ArcserveNagpur, IN

Established in 1983, Arcserve is the world’s most experienced provider of business continuity solutions that safeguard every application and system, on every premises and every cloud.Organizations ...Show moreLast updated: 30+ days ago

Promoted

Software Engineer

capeironNagpur, IN

We’re Hiring : Software Engineer [Remote].Month Contract Role at Capeiron Technology.Fixed Compensation for contract : INR 5,00,000. As a Software Engineer, you’ll be responsible for developing and ma...Show moreLast updated: 25 days ago

Promoted
New!

Senior Software Engineer

TalentnautsIndia

We are a new-age AI-First Digital and Cloud Engineering services company that drives Agility and Relevance for our client’s success. Powered by cutting-edge technology solutions that enable new busi...Show moreLast updated: 4 hours ago

Promoted

Senior Software Engineer - 35501

TuringNagpur, IN

Turing is seeking experienced software engineers to partner with a leading AI research lab in creating datasets and evaluation pipelines that advance next-generation large language models.In this r...Show moreLast updated: 11 days ago

Promoted

Senior Software Engineer

TalentojNagpur, IN

Design and develop backend services using Java, Spring Boot, and related frameworks.Write clean, modular, and testable code following best practices. Participate in technical design discussions and ...Show moreLast updated: 11 days ago

Promoted

Senior Software Engineer

Ubique SystemsNagpur, IN

Strong expertise in React & Golang development.Hands-on experience with Stripe, PayPal, Braintree, or similar payment processing tools. Proven ability to design & scale secure payment systems (PCI-D...Show moreLast updated: 20 days ago