Apache Kafka Masterclass
Apache Kafka for Java backend developers, from fundamentals to production: topics, partitions and offsets, KRaft and replication, producers and consumers, offsets and rebalancing, exactly-once and the outbox pattern, retention and compaction, Spring Kafka, retries and dead-letter topics, Schema Registry, Kafka Streams and Connect, performance, monitoring, security and interview scenarios.
Introduction to Apache Kafka
What event streaming is, why systems use Kafka instead of direct API calls, publish–subscribe vs queues, real-world use cases, and how Kafka compares with RabbitMQ and cloud services.
Kafka Core Concepts
Records, topics, partitions and offsets, how keys decide partitions and ordering, brokers and clusters, producers, consumers and consumer groups, and how they all fit together.
Kafka Architecture, KRaft & Replication
Brokers, controllers and the KRaft metadata quorum that replaced ZooKeeper, leaders, followers and in-sync replicas, high watermark, acks and min.insync.replicas, leader election, and how Kafka survives broker failures.
Running Kafka Locally & the CLI
Starting a single-node KRaft cluster with Docker, creating and describing topics, producing and consuming from the console with keys, inspecting consumer groups and lag, resetting offsets, and the most useful admin commands.
Kafka Producers
The producer send pipeline, a Java producer with sync and async sends and callbacks, serializers, partitioning, batching with linger.ms and batch.size, compression, acks, retries, idempotence and ordering, and the key producer settings.
Kafka Consumers & Consumer Groups
The poll loop, a Java consumer, deserializers, consumer groups and the group coordinator, partition assignment strategies, scaling consumers, auto.offset.reset, the key consumer settings, and processing records safely with multiple threads.
Offset Management & Rebalancing
How committed offsets work, auto-commit and its risks, commitSync vs commitAsync, committing per record or per batch, seeking and replaying, what triggers a rebalance, eager vs cooperative rebalancing, static membership, and Kafka 4's new consumer group protocol.
Delivery Semantics & Exactly-Once
At-most-once, at-least-once and exactly-once, where duplicates and losses come from, idempotent producers, Kafka transactions and read_committed, exactly-once stream processing, idempotent consumers, and the transactional outbox pattern.
Retention, Log Compaction & Storage
How partitions are stored as segment files, time- and size-based retention, the delete vs compact cleanup policies, how compaction keeps the latest value per key, tombstones, compacted topics as state, tiered storage, and important topic settings.
Kafka with Spring Boot (Spring Kafka)
Spring Boot auto-configuration, application.yml settings, creating topics with NewTopic beans, sending with KafkaTemplate and handling the result, @KafkaListener, JSON events, concurrency, manual acknowledgement, batch listeners, headers, and testing with embedded Kafka and Testcontainers.
Error Handling, Retries & Dead-Letter Topics
Transient vs permanent failures, poison pills, Spring Kafka's DefaultErrorHandler and back-off, non-retryable exceptions, dead-letter topics with DeadLetterPublishingRecoverer, non-blocking retries with @RetryableTopic, deserialization errors, and producer-side failures.
Schemas, Serialization & Schema Registry
Why event contracts matter, JSON vs Avro vs Protobuf, how a Schema Registry works, backward, forward and full compatibility, safe schema evolution rules, event design and versioning, and configuring Avro in Spring Boot.
Kafka Streams & Kafka Connect
Stream processing with Kafka Streams — KStream vs KTable, stateless and stateful operations, aggregations, windowing, joins and state stores — plus Kafka Connect source and sink connectors, change data capture with Debezium, and when to use each tool.
Kafka in Production — Performance, Monitoring & Security
Choosing partition counts, tuning producers and consumers for throughput or latency, broker sizing, the metrics that matter (consumer lag, under-replicated partitions), alerting, encryption, authentication and ACLs, and a production checklist.
Kafka Interview Masterclass
The most-asked Kafka interview questions with crisp answers, real-world design scenarios (ordering, duplicates, poison pills, lag, outbox, sagas), event-driven patterns, and a one-page quick revision sheet.