No longer accepting applications
Senior Software Engineer (Immediate Start)
CogniteBengaluru, Karnataka, India7 hours ago
Job descriptionEngineer Spark applications with a deep understanding of Catalyst optimiser, partitioning strategies, and performance tuning Develop real-time streaming solutions processing millions of events per second with Kafka and Flink. Design efficient data lake architectures using S3 / GCS with optimised partitioning and file formats (Parquet, ORC). Implement query optimisation techniques for OLAP datastores like ClickHouse, Pinot, or Druid. Scale systems to 10K+ QPS while maintaining high availability and data consistency. Optimise JVM performance through garbage collection tuning and memory management. Implement comprehensive monitoring using Prometheus, Grafana, and distributed tracing. Design fault-tolerant architectures with proper circuit breakers and retry mechanisms. Contribute to open-source projects in the big data ecosystem (Spark, Kafka, Airflow). Research and prototype new technologies for industrial data challenges. Collaborate with product teams to translate complex requirements into scalable technical solutions. Participate in architectural reviews and technical design discussions. Distributed Systems Experience (4-6 years) - Production Spark experience - built and optimised large-scale Spark applications with understanding of internals - Streaming systems proficiency - implemented real-time data processing using Kafka, Flink, or Spark Streaming - JVM Language expertise - strong programming skills in Java, Scala, or Kotlin with performance optimisation experience. Data Platform Foundations (3+ years) - Big data storage systems - hands-on experience with data lakes, columnar formats, and table formats (Iceberg, Delta Lake) - OLAP query engines - worked with Presto / Trino, ClickHouse, Pinot, or similar high-performance analytical databases - ETL / ELT pipeline development - built robust data transformation pipelines using tools like DBT, Airflow, or custom frameworks Infrastructure and Operations - Kubernetes production experience -deployed and operated containerised applications in production environments. Cloud platform proficiency - hands-on experience with AWS, Azure, or GCP data services. Monitoring and observability - implemented comprehensive logging, metrics, and alerting for data systems. Performance Engineering - System optimisation experience - delivered measurable performance improvements (2x+ throughput gains). Resource efficiency - optimised systems for cost while maintaining performance requirements. Concurrency expertise - designed thread-safe, high-concurrency data processing systems. Data Engineering Best Practices - Data quality frameworks -implemented validation, testing, and monitoring for data pipelines. Schema evolution - managed backwards-compatible schema changes in production systems. Data modelling expertise - designed efficient schemas for analytical workloads. Technical Collaboration - Cross-functional partnership - worked effectively with product managers, ML engineers, and data scientists. Codereview excellence - provided thoughtful technical feedback and maintained high code quality standards. Documentation and knowledge sharing - created technical documentation and participated in knowledge transfer. Continuous Learning - Technology adoption - quickly learned and applied new technologies to solve business problems. Industry awareness - stayed current with big data ecosystem developments and best practices. Problem-solving approach - demonstrated a systematic approach to debugging complex distributed system issues. Execution Excellence - Rapid delivery - consistently shipped high-quality features within aggressive timelines. Technical pragmatism - made smart trade-offs between technical debt, velocity, and system reliability. End-to-end ownership - took responsibility for features from design through production deployment and monitoring. Ambiguity comfort - thrived in environments with evolving requirements and unclear specifications. Technology flexibility - adapted to new tools and frameworks based on project needs. Customer focus - understood how technical decisions impact user experience and business metrics. Open-source contributions to major Apache projects in the data space (e. g. Apache Spark or Kafka) are a big plus. Conference speaking or technical blog writing experience, Industrial domain knowledge - previous experience with IoT, manufacturing, or operational technology systems. Languages : Kotlin, Scala, Python, Java. Big Data : Apache Spark, Apache Flink, Apache Kafka. Storage : PostgreSQL, ClickHouse, Elasticsearch, S3-compatible systems. Infrastructure : Kubernetes, Docker, Terraform. Table Formats : Apache Iceberg, Delta Lake, Apache Hudi. Query Engines : Trino / Presto, Apache Pinot, DuckDB. Orchestration : Apache Airflow, Dagster. Monitoring : Prometheus, Grafana, Jaeger, ELK Stack.
Cognite is revolutionising industrial data management through our flagship product, Cognite Data Fusion - a state-of-the-art SaaS platform that transforms how industrial companies leverage their data. We're seeking a Senior Data Platform Engineer who excels at building high-performance distributed systems and thrives in a fast-paced startup environment. You'll be working on cutting-edge data infrastructure challenges that directly impact how Fortune 500 industrial companies manage their most critical operational data.
Responsibilities :
- High-Performance Data Systems : Design and implement robust data processing pipelines using Apache Spark, Flink, and Kafka for terabyte-scale industrial datasets.
- Build efficient APIs and services that serve thousands of concurrent users with sub-second response times.
- Optimise data storage and retrieval patterns for time-series, sensor, and operational data.
- Implement advanced caching strategies using Redis and in-memory data structures.
Distributed Processing Excellence :
Scalability and Performance :
Technical Innovation :
Requirements :
Technical Depth Indicators :
Collaboration and Growth :
Startup Mindset :
Bonus Points :
Technical Stack :
Primary Technologies :
Technologies You May Work With :
Create a job alert for this search
Senior Software Engineer • Bengaluru, Karnataka, India
Related jobs
![Senior Engineer Software [T500-20481]](https://cdn-logos.talent.com/v1/logo/image?feedcode=attb-in&company_name=Albertsons%20Companies%20India)
![Senior Engineer Software [T500-20791]](https://cdn-logos.talent.com/v1/logo/image?feedcode=jobtraffic-in&company_name=Albertsons%20Companies%20India)
![Senior Engineer Software [T500-20405]](https://cdn-logos.talent.com/v1/logo/image?feedcode=whatjobs-in&company_name=Albertsons%20Companies%20India)
- Promoted
Senior Software Engineer
RipplingBengaluru, Karnataka, India Rippling gives businesses one place to run HR, IT, and Finance.It brings together all of the workforce systems that are normally scattered across a company, like payroll, expenses, benefits, and co...Show moreLast updated: 30+ days ago
- Promoted
Senior Software Engineer
Cimpress IndiaBengaluru, IN Cimpress Technology develops cutting-edge, best-in-world software that our mass customization businesses use to create personalized products for millions of global customers.Our Mass Customization ...Show moreLast updated: 30+ days ago
- Promoted
Senior Software Engineer
Enjen AI - An Entity of Qrios VenturesGreater Bengaluru Area, India Qrios Ventures is a Bengaluru-based ERP and digital consulting firm, offering integrated enterprise solutions and brand transformation services.
We help manufacturing and job-work clients enhance ef...Show moreLast updated: 11 days ago
- Promoted
- New!
Senior Software Engineer I
OptumBengaluru, Karnataka, India Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives.The work you do with our team will directly improve health outcomes by connect...Show moreLast updated: less than 1 hour ago
- Promoted
Software Engineer
AlphaNimbleGreater Bengaluru Area, India The Kantata Cloud for Professional Services™ gives businesses the clarity, control, and confidence they need to optimize resource planning and elevate operational performance.Our purpose-built soft...Show moreLast updated: 25 days ago
- Promoted
Senior Engineer Software [T500-20481]
Albertsons Companies IndiaBengaluru, Karnataka, India About Albertsons Companies Inc.As a leading food and drug retailer in the United States, Albertsons Companies, Inc.Our well-known banners across the United States, including Albertsons, Safeway, Vo...Show moreLast updated: 10 days ago
- Promoted
Senior Software Engineer
ChargebeeGreater Bengaluru Area, India Chargebee is the leading Revenue Growth Management (RGM) platform for subscription businesses.Thousands of companies at every stage of development — from startups to enterprises — use Chargebee to ...Show moreLast updated: 2 days ago
- Promoted
Senior Engineer Software [T500-20791]
Albertsons Companies IndiaBengaluru, Karnataka, India About Albertsons Companies Inc.As a leading food and drug retailer in the United States, Albertsons Companies, Inc.Our well-known banners across the United States, including Albertsons, Safeway, Vo...Show moreLast updated: 11 days ago
- Promoted
Senior Software Engineer
Moody'sBangalore, India At Moody's, we unite the brightest minds to turn today's risks into tomorrow's opportunities.We do this by striving to create an inclusive environment where everyone feels welcome to be who they ar...Show moreLast updated: 5 days ago
- Promoted
Senior Engineer Software [T500-20405]
Albertsons Companies IndiaBengaluru, Karnataka, India About Albertsons Companies Inc.As a leading food and drug retailer in the United States, Albertsons Companies, Inc.Our well-known banners across the United States, including Albertsons, Safeway, Vo...Show moreLast updated: 10 days ago
- Promoted
Senior Software Engineer
ArcserveBangalore, IN Established in 1983, Arcserve is the world’s most experienced provider of business continuity solutions that safeguard every application and system, on every premises and every cloud.Organizations ...Show moreLast updated: 30+ days ago
- Promoted
Senior Java Software Engineer
MphasisGreater Bengaluru Area, India Exciting Career Opportunities at Mphasis!.We’re looking for passionate and skilled professionals to join.IT solutions provider that’s transforming businesses through technology and innovation.If yo...Show moreLast updated: 8 days ago
- Promoted
Senior Software Engineer
EpsilonBengaluru, Karnataka, India The Product team forms the crux of our powerful platforms and helps connect millions of customers worldwide with the brands that matter most to them.
This team of innovative thinkers develops and bu...Show moreLast updated: 30+ days ago
- Promoted
Senior Engineer Software [T500-20794]
Albertsons Companies IndiaBengaluru, Karnataka, India About Albertsons Companies Inc.As a leading food and drug retailer in the United States, Albertsons Companies, Inc.Our well-known banners across the United States, including Albertsons, Safeway, Vo...Show moreLast updated: 11 days ago
- Promoted
Senior Software Engineer
RubrikGreater Bengaluru Area, India Senior Software Engineer - Blobstore.Blobstore is the core storage management and storage abstraction layer.For a multi node cluster this layer is responsible for some extremely important activitie...Show moreLast updated: 25 days ago
- Promoted
Senior Software Engineer
BrillioBengaluru, Karnataka, India Design, develop, and maintain robust and scalable software applications using modern development best practices.Actively participate in an agile development process, contributing to sprint planning...Show moreLast updated: 11 days ago
- Promoted
Senior Software Engineer
TrellixGreater Bengaluru Area, India We are looking for an experienced and passionate Senior Engineer to join our cutting-edge product development team.In this role, you will help design, build, and enhance our next-generation email s...Show moreLast updated: 25 days ago
- Promoted
- New!
Senior Software Engineer
MSHBengaluru, IN As a consultant… You are responsible for delivering an end-to-end solution from initial concept through finished solution.
You’ll need the ability to quickly understand needs, from a customer’s pers...Show moreLast updated: 22 hours ago
- Promoted
Senior Software Engineer - Full Stack
RipplingBengaluru, Karnataka, India Rippling gives businesses one place to run HR, IT, and Finance.It brings together all of the workforce systems that are normally scattered across a company, like payroll, expenses, benefits, and co...Show moreLast updated: 25 days ago
- Promoted
Senior Engineer Software [T500-20792]
Albertsons Companies IndiaBengaluru, Karnataka, India About Albertsons Companies Inc.As a leading food and drug retailer in the United States, Albertsons Companies, Inc.Our well-known banners across the United States, including Albertsons, Safeway, Vo...Show moreLast updated: 11 days ago