Sr. Lead, Infra Cloud Kafka

About Northern Trust:

Northern Trust a Fortune 500 company is a globally recognized award-winning financial institution that has been in continuous operation since 1889.

Northern Trust is proud to provide innovative financial services and guidance to the worlds most successful individuals families and institutions by remaining true to our enduring principles of service expertise and integrity. With more than 130 years of financial experience and over 22000 partners we serve the worlds most sophisticated clients using leading technology and exceptional service.

Main Duties:

Northern Trust Enterprise Middleware Services is seeking a highly experienced Sr. Lead Confluent Kafka Platform Engineer to join the team on a full-time basis. This role will lead the strategy and execution of a modern Kafka platform ensuring scalability security and high availability across hybrid environments. The individual will drive automation enable self-service capabilities and accelerate cloud adoption to simplify operations and improve onboarding efficiency. Working closely with engineering teams this position will also play a key role in advancing real-time data streaming capabilities while strengthening platform reliability governance and overall engineering maturity.

Responsibilities

• Lead the design implementation and optimization of enterprise Kafka platforms across on-prem and cloud environments.
• Architect and manage hybrid Kafka solutions integrating Confluent Platform and Confluent Cloud.
• Ensure high availability scalability and resiliency through multi-region and multi-zone deployments.
• Drive platform modernization including migration from on-prem Kafka to cloud-native streaming platforms.
• Implement cluster linking and data replication strategies for hybrid and DR use cases.
• Develop and maintain automation frameworks for Kafka provisioning configuration and lifecycle management.
• Implement no-touch / low-touch Kafka operations using Infrastructure as Code and CI/CD pipelines.
• Design and deliver self-service and web-based platforms for Kafka onboarding and management.
• Collaborate with application teams to design event-driven architectures and real-time streaming solutions.
• Plan and execute Kafka platform upgrades ensuring minimal downtime and operational risk.
• Implement monitoring alerting and observability frameworks for platform health and reliability.
• Perform quarterly capacity planning performance tuning and scaling of Kafka clusters.
• Enforce enterprise security standards and governance for Kafka platforms.
• Provide L3 support and perform root cause analysis for production incidents.
• Lead and mentor Kafka engineering teams and act as a technical SME.

SKILLS & EXPERIENCE

Qualifications:

• Bachelors degree in Computer Science Information Systems or related field
• 812 years of experience in Middleware / Messaging systems.
• Strong focus on Apache Kafka / Confluent Kafka administration and engineering.
• Experience working in enterprise-scale production-critical environments.
• Proven leadership experience in managing technical teams or platforms.

Strong Experience In:

• Building self-service platforms or developer portals for infrastructure provisioning
• CI/CD pipelines and GitOps-based infrastructure management
• Observability tools like Prometheus Grafana or enterprise monitoring platforms
• Kafka upgrades migration and platform lifecycle management
• Working with multi-region DR architectures and failover strategies
• Supporting mission-critical production systems with strict SLAs

Solid experience In:

• Kafka ecosystem components such as Kafka Connect Kafka Streams Schema Registry
• Monitoring and observability tools (Prometheus Grafana Dynatrace).
• Kafka security implementation (SSL/mTLS SASL ACLs RBAC).
• Managing production incidents and performing root cause analysis.
• Working with application teams for Kafka onboarding and integration.
• Implementing self-service and automation platforms.

Strong Knowledge Of:

• Distributed systems design and event-driven architecture patterns
• Streaming architectures data pipelines and integration patterns
• High availability fault tolerance and replication strategies
• Kafka internals (partitioning replication leader election ISR retention)
• Security frameworks including mTLS RBAC ACLs authentication/authorization
• Cloud networking concepts (private endpoints VNet DNS secure connectivity)

Experience in:

• Migrating applications from on-prem Kafka to cloud-based Kafka platforms
• Building automation-first infrastructure platforms with minimal manual operations
• Designing self-service onboarding workflows for developers
• Implementing Kafka governance including topic lifecycle retention and access controls
• Working in audit-compliant and regulated environments
• Collaborating across cloud networking security and application teams

Ability To:

• Design scalable resilient and future-ready Kafka platforms.
• Drive platform standardization and engineering maturity across teams.
• Automate repetitive operational tasks and reduce manual effort.
• Quickly diagnose and resolve complex system issues.
• Influence architecture decisions and guide engineering teams.
• Communicate effectively with both technical and non-technical stakeholders.

Demonstrated ability to:

• Lead Kafka platform initiatives end-to-end with strong ownership and accountability.
• Mentor engineers and build high-performing technical teams.
• Drive large-scale transformation initiatives such as cloud migration and automation adoption.
• Deliver measurable improvements in performance reliability and operational efficiency.
• Establish Kafka as a core enterprise data streaming platform.

Strong mindset for:

• Automation-first and platform engineering thinking
• Continuous improvement and modernization of legacy systems
• Building scalable reusable and developer-friendly platforms
• Proactive problem-solving and operational excellence
• Driving innovation in hybrid cloud and streaming technologies

Nice to Have

• Experience with Confluent Cloud or managed Kafka services
• Exposure to hybrid cloud Kafka architectures (on-prem cloud integration)
• Experience with Kafka Streams KSQLDB or real-time stream processing
• Experience building self-service or web-based Kafka platforms
• Knowledge of DevOps practices GitHub Actions or cloud-native automation
• Experience with other messaging systems (IBM MQ Solace etc.)
• Exposure to multi-region DR architecture and cluster linking
• Experience working with cloud networking (Private Link DNS VNet design)
• Familiarity with data governance data lineage and schema validation tools

Northern Trust Corporation (Nasdaq: NTRS) is a leading provider of wealth management asset servicing asset management and banking to corporations institutions affluent families and individuals. Founded in Chicago in 1889 Northern Trust has offices in the United States in 19 states and Washington D.C. and 20 international locations in Canada Europe the Middle East and the Asia-Pacific region.

Working with Us:

As a Northern Trust partner greater achievements await. You will be part of a flexible and collaborative work culture in an organization where financial strength and stability is an asset that emboldens us to explore new ideas.

Movement within the organization is encouraged senior leaders are accessible and you can take pride in working for a company committed to assisting the communities we serve! Join a workplace with a greater purpose.

Wed love to learn more about how your interests and experience could be a fit with one of the worlds most admired and sustainable companies! Build your career with us and apply today. #MadeForGreater

Reasonable accommodation

Northern Trust is committed to working with and providing reasonable accommodations to individuals with disabilities. If you need a reasonable accommodation for any part of the employment process please email our HR Service Center at .

We hope youre excited about the role and the opportunity to work with us. We value an inclusive workplace and understand flexibility means different things to different people.

Apply today and talk to us about your flexible working requirements and together we can achieve greater.

About Our Pune Office

The Northern Trust Pune office established in 2016 is now home to over 3000 employees. The office handles various functions including Operations for Asset Servicing and Wealth Management as well as delivering critical technology solutions that support business operations across the globe.

Our Pune team takes our commitment to service to 2024 they volunteered more than 10000 hours into the communities where they live and work. Learn more.

Required Experience:

Senior IC

Employment Type : Full-Time
Experience: years
Vacancy: 1