Kafka Interview Masterclass

A cluster of brokers stores topics split into partitions. Each partition is an ordered, append-only log replicated across brokers, with one leader handling reads and writes and followers copying it. A KRaft controller quorum manages metadata and leader election (ZooKeeper was removed in Kafka 4.0). Producers write records — routed to partitions by key — and consumers in consumer groups poll partitions and commit offsets to track progress. Records are retained by time or size, not deleted on consumption.

Sequential appends to log segments, the OS page cache, batching and compression on the producer, zero-copy transfer from page cache to network, partitioning across brokers, and pull-based consumers.

Only within a partition. Use the entity ID as the key so related events go to the same partition, and keep the idempotent producer enabled so retries can't reorder batches.

Within one consumer group, each partition is assigned to exactly one consumer; a consumer can have several partitions. Parallelism per group is capped at the partition count; extra consumers are idle.

It stops heartbeating; after session.timeout.ms the coordinator removes it and rebalances, giving its partitions to other members, which resume from the last committed offsets — so records processed but not committed are reprocessed.

At-most-once (commit before processing), at-least-once (commit after processing — the usual choice, with idempotent consumers) and exactly-once (idempotent producer + transactions + read_committed, for Kafka-to-Kafka processing).

A write is acknowledged only once at least two in-sync replicas have it; if fewer are in sync, the write fails rather than becoming under-replicated. With RF = 3, the topic stays writable with one broker down.

A producer whose batches carry a producer ID and per-partition sequence numbers so brokers discard duplicates from retries. On by default since Kafka 3.0.

Redistribution of partitions among a group's consumers when members join, leave, fail or exceed max.poll.interval.ms. Minimise impact with the cooperative sticky assignor or the new Kafka 4 group protocol, static membership, fast processing per poll and graceful shutdowns.

A cleanup policy that keeps the latest record per key and removes older ones; null values (tombstones) delete keys. Used for state such as __consumer_offsets and Kafka Streams changelogs.

The difference between a partition's log-end offset and the group's committed offset — records waiting to be processed. The key health metric for consumers.

Kafka is a replicated, partitioned, replayable log for high-throughput event streaming with many independent consumers; RabbitMQ is a broker with flexible routing where messages are removed when acknowledged — better for task queues, priorities and request/reply.

KRaft — Kafka's built-in Raft-based metadata quorum. Production-ready since 3.3; the only option from 4.0.

Bounded retries with exponential back-off for transient errors, immediate routing of permanent errors to a dead-letter topic with error headers, alerting on the DLT and a replay tool.

Writing the event to an outbox table in the same database transaction as the business change, then publishing it to Kafka from the table (polling or Debezium CDC) — avoiding unreliable dual writes.

Likely causes: no key or a non-entity key (events spread across partitions); partitions were added, changing key placement; non-blocking retries (@RetryableTopic) let later events overtake a failing one; or the consumer processes records in parallel threads. Fix: key by orderId; keep idempotence on; use blocking retries where order matters; process each partition (or each key) sequentially; and make consumers tolerate out-of-order events with version numbers where needed.

At-least-once delivery means a crash or rebalance between processing and committing redelivers records. Make the consumer idempotent: include an eventId in every event and record processed IDs in a table with a unique constraint in the same transaction as the charge, or use the payment provider's idempotency key. Also ensure the producer is idempotent and events are published via an outbox, not re-sent on retry.

A poison pill: the listener keeps retrying the same record, blocking its partition. Configure bounded retries (DefaultErrorHandler with a back-off), mark validation and deserialization errors as non-retryable, wrap deserializers in ErrorHandlingDeserializer, and publish failures to a dead-letter topic — then fix and replay.

Check whether consumers are healthy (rebalancing loops, errors, stuck threads). If they're simply too slow: scale consumers up to the partition count (and add partitions if already at the limit), batch database writes, remove slow synchronous calls from the hot path, tune max.poll.records and fetch sizes, and consider async or parallel processing per key. Monitor lag per partition — a single hot partition points to a skewed key.

Processing a poll batch takes longer than max.poll.interval.ms, so the consumer is kicked out, its commit fails and another consumer reprocesses the batch. Reduce max.poll.records, speed up processing, raise max.poll.interval.ms if justified, and use static membership and the cooperative assignor to reduce deployment rebalances.

Use the transactional outbox: save the order and an OrderPlaced outbox row in one database transaction; a relay (Debezium or a poller) publishes outbox rows to Kafka and marks them sent. Consumers deduplicate by eventId because the relay is at-least-once.

Each service performs a local transaction and publishes events through an outbox; failures publish events that trigger compensating actions (refund payment, release stock, cancel order). Key by orderId for ordering, make every handler idempotent, and use a correlation ID header for tracing. For long or complex flows, prefer an orchestrator (see the Saga pattern chapter).

If retention allows, start a new consumer group with auto.offset.reset=earliest (or reset its offsets to the beginning) and replay. If the data must be available indefinitely, keep a compacted "latest state" topic or set long retention / tiered storage, or bootstrap from a database snapshot and then consume from a known offset.

Use a Schema Registry with BACKWARD (or FULL) compatibility; add new fields as optional with defaults; never rename or retype fields in place; upgrade consumers first for backward-compatible changes; and introduce a new topic or event version for breaking changes.

For partitions it led, the controller elects new leaders from the ISR within seconds; clients get errors such as NotLeaderOrFollower, refresh metadata and retry against the new leaders. With RF = 3 and min.insync.replicas=2, no acknowledged data is lost and writes continue. The broker rejoins as a follower when it returns.