AWS SQS & SNS: Managed Queues and Pub/Sub Messaging

Q: SQS or SNS — which do I use?

SQS when you're distributing work — a task that one consumer should process (resize an image, send an email). SNS when you're broadcasting an event that multiple systems care about (an order was placed → notify billing, inventory, analytics). If you need both fan-out and per-consumer reliability, combine them: SNS→SQS fan-out.

Q: Standard or FIFO queue?

Standard for almost everything — near-unlimited throughput, at-least-once delivery (make consumers idempotent). FIFO only when you genuinely need strict ordering or exactly-once processing (sequential financial operations, ordered workflows), accepting its lower throughput. Don't reach for FIFO by default; idempotent Standard consumers handle most cases.

Q: When should I use EventBridge instead?

EventBridge when you need richer event routing — rules that filter and route events to different targets, integration with SaaS event sources, or scheduled/cron events. SNS is simpler pub/sub; EventBridge is a full event bus with routing logic. For basic fan-out, SNS; for complex routing across many sources, EventBridge.

AWS SQS & SNS

SQS (Simple Queue Service) and SNS (Simple Notification Service) are AWS's core managed messaging services — the building blocks for decoupling components so they don't call each other directly. SQS is a message queue: producers send messages, consumers pull and process them one at a time, reliably. SNS is a pub/sub topic: publishers send a message once and it fans out to many subscribers simultaneously. They're the AWS-native, fully-managed answer to what RabbitMQ and Kafka do — with no brokers to operate.

Together they enable the standard AWS event-driven patterns: offload slow work to background workers (SQS), broadcast an event to multiple systems (SNS), or combine them (SNS→SQS fan-out) so one event reliably reaches several independent queues. Everything on the message queues concept page applies — this is the serverless AWS implementation.

TL;DR

SQS = queue (one-to-one-ish): producers send, consumers pull and process; messages persist until acknowledged (deleted). For task/work distribution.
SNS = pub/sub topic (one-to-many): a message published once is pushed to all subscribers (SQS queues, Lambda, HTTP endpoints, email/SMS). For fan-out/broadcast.
SQS types: Standard (high throughput, at-least-once, best-effort order) and FIFO (exactly-once processing, strict order, lower throughput).
The fan-out pattern (SNS → multiple SQS queues) is the classic combo: publish an event once, and each interested service gets its own reliable queue copy.
Both are fully managed and serverless — no brokers to run, auto-scaling, pay per request.
Design consumers to be idempotent (Standard SQS delivers at-least-once) and use dead-letter queues for messages that repeatedly fail.

SQS vs SNS: The Core Difference

SQS: a message sits in the queue until a consumer pulls it, processes it, and deletes it. Work is done once by one consumer. Great for distributing tasks to a pool of workers.
SNS: a message is pushed immediately to every subscriber. Same message, delivered to many. Great for broadcasting an event to multiple systems that each care.

The mental model: SQS answers "do this work" (a task); SNS answers "this happened" (an event) — tell everyone.

Core Concepts

SQS: Reliable Task Queues

Key mechanics:

Visibility timeout: when a consumer receives a message, it becomes invisible to others for a window; if the consumer crashes before deleting it, it reappears for another worker (reliability). Set it longer than your processing time.
At-least-once (Standard): a message can be delivered more than once — consumers must be idempotent.
Long polling (WaitTimeSeconds) reduces empty receives and cost.
Dead-letter queue (DLQ): after N failed processing attempts, route the message to a DLQ instead of looping forever — the standard poison-message handling.

Standard vs FIFO Queues

Default to Standard with idempotent consumers; use FIFO only when strict ordering or deduplication is genuinely required — it's the same trade Kafka makes with partition ordering.

SNS: Pub/Sub and Fan-Out

SNS publishes a message to a topic, which pushes it to all subscribers: SQS queues, Lambda functions, HTTP/S endpoints, email, or SMS. Message filtering lets each subscriber receive only messages matching attributes (so not every subscriber gets everything).

The Fan-Out Pattern (SNS → SQS)

The most important combined pattern: publish an event to an SNS topic, with multiple SQS queues subscribed. Each interested service gets its own durable queue copy of the event:

This gives you the best of both: SNS's one-publish fan-out plus SQS's per-consumer reliability (each service processes at its own pace, retries independently, has its own DLQ). It's the canonical AWS event-driven architecture, and structurally similar to what Kafka consumer groups provide.

SQS/SNS vs EventBridge vs Kafka

AWS has several messaging options; picking right matters:

Quick guide: SQS for tasks, SNS for simple fan-out, EventBridge when you need rich routing rules and many event sources, Kafka/Kinesis for high-throughput streaming with replay. SQS+SNS is the simplest and cheapest for the common decoupling and fan-out cases.

Common Mistakes

Non-Idempotent Consumers on Standard Queues

Standard SQS is at-least-once — the same message will occasionally be delivered twice (redrives, visibility-timeout expiries). A consumer that blindly re-executes (charging, inserting) duplicates work. Make processing idempotent (dedupe by message ID, upsert), or use FIFO if you truly need exactly-once.

Visibility Timeout Too Short

If processing takes longer than the visibility timeout, the message reappears and a second worker starts processing it in parallel — duplicate work and confusion. Set the visibility timeout comfortably above your max processing time (or extend it dynamically for long tasks).

No Dead-Letter Queue

Without a DLQ, a message that always fails is retried forever (wasting resources, blocking) or eventually dropped (data loss). Configure a DLQ with a max-receive count so poison messages are captured for inspection instead of looping or vanishing.

Using SNS When You Need Durability Per Consumer

SNS direct-to-Lambda/HTTP delivery has limited retry and no per-consumer buffering — if a subscriber is down, messages can be lost. For reliable delivery to each consumer, use the SNS→SQS fan-out pattern so each service has a durable queue that survives its downtime.

Reaching for Kafka When SQS/SNS Suffices

Standing up Kafka/MSK for simple task queues or fan-out is heavy operational overhead for problems SQS+SNS solve with zero servers. Use Kafka when you genuinely need streaming, replay, or extreme throughput — not for basic decoupling.

FAQ

SQS or SNS — which do I use?

SQS when you're distributing work — a task that one consumer should process (resize an image, send an email). SNS when you're broadcasting an event that multiple systems care about (an order was placed → notify billing, inventory, analytics). If you need both fan-out and per-consumer reliability, combine them: SNS→SQS fan-out.

What's the fan-out pattern?

Publish an event to an SNS topic that has multiple SQS queues subscribed. Each subscribed service gets its own durable queue copy of the event, processing independently with its own retries and DLQ. It combines SNS's one-to-many broadcast with SQS's reliability — the canonical AWS event-driven pattern.

Standard or FIFO queue?

Standard for almost everything — near-unlimited throughput, at-least-once delivery (make consumers idempotent). FIFO only when you genuinely need strict ordering or exactly-once processing (sequential financial operations, ordered workflows), accepting its lower throughput. Don't reach for FIFO by default; idempotent Standard consumers handle most cases.

How do these compare to RabbitMQ or Kafka?

SQS/SNS are the fully-managed, serverless AWS equivalents — no brokers to run. SQS ≈ RabbitMQ-style task queues (though with less routing flexibility); SNS ≈ pub/sub fan-out. For Kafka-style event streaming with replay and high throughput, AWS offers Kinesis or MSK instead. SQS+SNS win on simplicity and zero ops for the common cases.

When should I use EventBridge instead?

EventBridge when you need richer event routing — rules that filter and route events to different targets, integration with SaaS event sources, or scheduled/cron events. SNS is simpler pub/sub; EventBridge is a full event bus with routing logic. For basic fan-out, SNS; for complex routing across many sources, EventBridge.