Delivery Semantics & Exactly-Once

#Introduction

Failures happen at every step — the producer, the network, the broker, the consumer. Delivery semantics describe what happens to a message when they do.

#The Three Guarantees

At-least-once is Kafka's practical default for well-configured applications, and the most common production choice.

#Where Losses and Duplicates Come From

#At-Most-Once

Commit first, then process:

ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
consumer.commitSync();             // commit before processing
records.forEach(this::process);    // a crash here loses these records

Acceptable only where occasional loss is fine — metrics sampling, non-critical telemetry.

#At-Least-Once

Producer: acks=all, idempotence on, retries on, and handle send failures. Consumer: process, then commit:

ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
records.forEach(this::process);
consumer.commitSync();             // a crash before this line → records are reprocessed

Nothing is lost, but a crash or rebalance between processing and committing means some records are processed twice. So at-least-once must be paired with idempotent processing.

#Idempotent Consumers

Make processing a record twice have the same effect as processing it once:

@Transactional
public void handle(OrderPaidEvent event) {
    if (processedEvents.existsById(event.eventId())) {
        return;                                    // duplicate — already handled
    }
    orders.markPaid(event.orderId());
    processedEvents.save(new ProcessedEvent(event.eventId()));   // unique constraint on eventId
}

#Exactly-Once Inside Kafka

Kafka provides exactly-once semantics (EOS) for read-process-write pipelines where both input and output are Kafka topics.

Part 1 — Idempotent Producer

Prevents duplicates from producer retries within a producer session (producer ID + sequence numbers). Enabled by default since Kafka 3.0. See Kafka Producers.

Part 2 — Transactions

A transactional producer writes to several partitions and commits consumer offsets atomically — all or nothing:

props.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "payment-processor-1");   // stable per instance
KafkaProducer<String, String> producer = new KafkaProducer<>(props);
producer.initTransactions();

while (running) {
    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(500));
    if (records.isEmpty()) continue;

    producer.beginTransaction();
    try {
        for (ConsumerRecord<String, String> record : records) {
            producer.send(new ProducerRecord<>("payments-processed", record.key(), transform(record.value())));
        }
        // commit the consumer's offsets as part of the same transaction
        producer.sendOffsetsToTransaction(nextOffsets(records), consumer.groupMetadata());
        producer.commitTransaction();
    } catch (ProducerFencedException e) {
        producer.close();          // another instance with the same transactional.id took over
        throw e;
    } catch (KafkaException e) {
        producer.abortTransaction();   // nothing from this batch becomes visible; it will be reprocessed
    }
}

Part 3 — read_committed Consumers

Downstream consumers must ignore records from aborted or still-open transactions:

props.put(ConsumerConfig.ISOLATION_LEVEL_CONFIG, "read_committed");   // default is read_uncommitted

Kafka Streams

Kafka Streams wraps all of this behind one setting:

props.put(StreamsConfig.PROCESSING_GUARANTEE_CONFIG, StreamsConfig.EXACTLY_ONCE_V2);

#The Limit of Exactly-Once

Storing Offsets with the Result

If the output goes to a relational database, commit the offsets in that database in the same transaction, and on startup seek() to them (using a ConsumerRebalanceListener). The result and the progress marker then succeed or fail together.

#The Transactional Outbox Pattern

The opposite problem: a service must update its database and publish an event. Doing both directly isn't atomic:

// ❌ Dual write — if the send fails (or the app crashes) after the commit, the event is lost;
// if you send first and the DB commit fails, you've announced something that didn't happen
orderRepository.save(order);
kafkaTemplate.send("orders", order.getId(), new OrderPlaced(order));

Outbox: write the event to an outbox table in the same database transaction as the business change, and let a separate process publish it:

The database transaction guarantees the event is recorded if — and only if — the order is. Debezium's outbox event router is a common implementation. This is the standard way to publish events reliably from microservices and sagas.

#Choosing a Guarantee

#Key Takeaways

• At-most-once, at-least-once and exactly-once differ in what happens on failure.
• At-least-once + idempotent consumers is the standard production approach.
• Kafka EOS = idempotent producer + transactions + read_committed, for Kafka-to-Kafka pipelines.
• sendOffsetsToTransaction commits output and consumer progress atomically.
• Use the transactional outbox to publish events reliably alongside database changes.