MQTT Delivery and Session Design

MQTT delivery design combines several independent choices: QoS for a protocol delivery, retained messages for subscription-time state, sessions for continuity across connections, and application rules for processing effects. Specify each boundary. A protocol acknowledgment does not prove that a business operation has completed exactly once.

This article uses MQTT 5.0 terminology. Check the broker and client versions and supported features before transferring the design to an actual deployment. In particular, do not copy MQTT 5 session-expiry assumptions into a different protocol version without reviewing its semantics.

TL;DR

Quick Example

This Python example illustrates application-level event identity. It is not an MQTT client, broker, packet exchange, or protocol emulator. It shows why a device-scoped identity can distinguish a repeated event from another device's event with the same local identifier.

The repeated sensor-a event produces no second toy effect, while sensor-b can use the same event identifier independently. The in-memory set and list do not survive a crash or provide an atomic persistent transaction. They also do not reject conflicting payloads with the same identity. Those are explicit application design questions, not behaviors supplied by this example.

Core Concepts

QoS belongs to the protocol exchange

MQTT 5.0 defines QoS 0, 1 and 2 delivery behavior and the state needed for their exchanges. Choose among supported levels according to the cost of loss, possible duplicates, protocol overhead and the actual link. Do not extend the protocol's delivery language into a guarantee about database writes, notifications or physical actions performed by an application. MQTT 5.0 standard

Map the path from publisher to broker to subscriber and then to the effect. Each step has its own acknowledgment and failure conditions. A subscriber can receive a message, apply an effect, and fail before recording application completion. A retry decision needs durable application evidence, not just the remembered QoS value.

Retained state and session state differ

The standard explicitly states that retained messages are not part of a server's session state and are not deleted merely because a session ends. A retained value helps a qualifying subscriber receive stored topic state under the subscription rules; it is not a replay of every historical event.

Session state supports continuity across network connections, subject to MQTT 5's clean-start and session-expiry behavior and implementation limits. Inspect the session-present response and the actual reconnect outcome rather than assuming every reconnect resumes everything. Broker capacity, administrative actions and storage failures also need an operating plan.

This matrix is a proposed design review. No row represents an executed broker or device test in this article.

Reconnection is a state transition

Define the client identity's ownership and what happens when the connection returns without the expected session. Decide whether to resubscribe, request a state refresh, reconcile pending operations, or start a new application epoch. A reconnect event by itself does not tell the application which historical effects occurred.

Avoid using a protocol packet identifier as an unlimited business-event identity. Its role and lifetime belong to the MQTT session and protocol exchanges. An application identity should be scoped to the actual business operation and retained for the period in which duplicates or replays matter.

Define freshness independently. A perfectly delivered command can be obsolete by the time the consumer acts. Use an application validity rule appropriate to the action and verify the MQTT 5 expiry behavior you rely on. A retained sensor reading and a control command generally need different interpretation.

Best Practices

Write down the intended message class: current state, measurement event, command, or acknowledgment. Choose topic and payload semantics that make that distinction visible. Do not retain every command simply because retaining the latest state was useful elsewhere; assess what a later subscriber would do with that stored command.

Build a reconnect test matrix with the actual client and broker. Include brief disconnects, expiry, broker restart, unexpected session absence, repeated delivery and application failure after effect but before completion recording. Record which layer acknowledged what. Keep tests on disposable topics and devices with no production control effect.

Couple durable effect identity with the operation when correctness requires it. Define what happens if the same identity arrives with a different payload and how long deduplication evidence remains available. The toy accepts the first seen identity without a conflict check, so it is an illustration of scope rather than a production receipt implementation.

Observe client and broker behavior together. Track disconnect reasons, reconnect attempts, queue age, expired messages, application success/failure and reconciliation outcomes using non-sensitive identifiers. A connected client can still be consuming stale work or failing every downstream operation.

Reverse a high-QoS recommendation if the real measurement stream tolerates loss and the protocol/storage cost is unjustified. Choose stronger application recovery when a command's effect cannot safely repeat. The correct design follows the semantics and consequences of the message, not a universal rule to select the largest QoS number.

Common Mistakes

Using retained messages as an event log

Confusing session expiry with retained-message deletion

Treating QoS 2 as exactly-once business effects

Testing reconnection only with a live session

FAQ

Does QoS 2 make my database update exactly once?

Not by itself. The protocol exchange and the application effect have separate boundaries. Design and test durable operation identity, commit behavior, retry and reconciliation for the actual consumer.

Do retained messages disappear with a session?

MQTT 5.0 specifies that retained messages are not part of server session state and are not deleted merely because a session ends. Their other validity and deletion rules still matter.

What should the first integration test cover?

An authorized disposable publisher/broker/subscriber path with a disconnect and explicit verification of session and application outcomes. The Python illustration exercised no MQTT transport or broker behavior.

Related Topics

References