Designing Kafka Systems: Event-Driven Architecture, Logs, Microservices, Real-Time Event Processing

Get a free eBook (PDF or ePub) from Manning as well as access to the online liveBook format (and its AI assistant that will answer your questions in any language) when you purchase the print book.

This book teaches you how to incorporate Kafka into enterprise applications. This book stays above the code-level details, focusing instead on how to use Kafka to achieve your technical and business goals.

Architects across industries are turning to Kafka for its unparalleled speed, reliability, and scalability. In this book, author Katya Gorshkova lays out how Kafka fits into complex system designs, expertly illustrating how you can use Kafka for effective logging, telemetry, microservices communication, and more in event driven enterprise applications.

In Kafka for Architects you will find:

- Kafka’s role in enterprise software
- The event-driven architecture pattern
- Data streaming solutions
- Event driven architecture in messaging systems
- Explaining Kafka clusters
- Data streaming solutions
- Designing data contracts
- Kafka in real world use cases

About the Technology

Kafka is a powerful distributed event streaming platform perfect for the real-time data pipelines you find in all modern software systems. As a software architect, it’s mission critical to understand how Kafka impacts every aspect of system design, deployment, management, and maintenance. This unique book provides the architectural principles you need to integrate Kafka effectively into enterprise applications at any scale.

About the Book

Kafka for Architects focuses on architectural principles rather than code, showing exactly how to align Kafka with your organization’s technical and business goals. You’ll explore proven patterns and anti-patterns, integration strategies, and the broader Kafka ecosystem--from event pipelines to microservices communication.

What’s Inside

- Event-driven architecture patterns: CQRS, event sourcing, and microservices integration
- Efficient messaging systems: message production, consumption, batching, partitioning, and throughput
- Kafka clusters: cluster architecture, topics, partitions, replication, and configuration
- Designing data contracts: event schema design, schema evolution, governance, and use of Schema Registry

About the Readers

For working and aspiring enterprise and solutions architects.

About the Author

Katya Gorshkova is a consulting engineer and architect in the FinTech sector with extensive experience designing applications for real-time data processing.

Table of Contents

Part 1
1 Getting to know Kafka as an architect
2 Kafka cluster data architecture
3 Kafka clients and message production
4 Creating consumer applications
Part 2
5 Kafka in real-world use cases
6 Defining data contracts
7 Kafka interaction patterns
8 Designing streaming applications
Part 3
9 Managing Kafka within the enterprise
10 Organizing a Kafka project
11 Operating Kafka
12 What’s next for Kafka

Katya Gorshkova has extensive experience in the FinTech sector, serving as both a Software Engineer and a Solution Architect. As a Kafka engineer, she provides consultancy services for messaging systems architecture and developing applications for real-time data processing.