You hit publish on a new microblog post. The page refreshes, and your words are live. But what you don't see is the real work happening behind the scenes—and that's exactly how I wanted to design it.

Your post isn't just stored in a database. Jottings generates a complete static website from your content: HTML pages, RSS feeds, sitemaps, tag clouds. All of it. And that generation happens asynchronously through a queue system, not the moment you click publish.

This was a deliberate architectural choice, and I want to walk you through why queues beat synchronous processing every single time.

The Synchronous Problem

Imagine if I'd chosen the simpler path: you click publish, your browser waits for the entire site to regenerate, and only then does the page respond. Sounds reasonable, right?

It's not.

Here's what breaks immediately:

Latency becomes unpredictable. Publishing a quick one-liner shouldn't take three seconds while regenerating 500 pages of history. But with synchronous processing, that's exactly what happens. Your site gets bigger, the response time gets longer. Users notice. They close the tab and move on.

Traffic spikes kill everything. One blog post gets shared on social media, ten people publish simultaneously, and suddenly your Lambda is drowning. Each request blocks waiting for the build to complete. You hit concurrency limits. Some requests timeout. Your publish feature breaks at the worst possible moment—when you're actually getting traffic.

You can't retry failed builds gracefully. If the build fails for any reason—a network blip, a DynamoDB throttle, an R2 connection issue—the user gets an error and has to manually republish. No resilience. No automatic recovery.

Debugging is a nightmare. When something breaks, the errors are synchronous and unpredictable. You have no history of failed builds, no way to understand patterns, no dead letter queue to investigate later.

I built Jottings to be resilient and scalable from day one, so synchronous processing was never really an option.

Enter the Queue

With SQS, the flow changes fundamentally:

1. You click publish → message goes into a queue
2. Your browser gets an immediate response: "publishing"
3. A Lambda function quietly picks up the message when it's ready
4. The build happens, your site updates
5. If something fails, the message goes to a dead letter queue for investigation

This decoupling changes everything.

Response times are instant. Your browser doesn't wait for the build. You get feedback immediately. The publish completes in your UI while the actual work happens in parallel. The site might not be updated for a few hundred milliseconds, but you've already moved on to writing your next post.

Traffic spikes become non-events. Get shared on Hacker News? Great. If you get 100 publishes at once, they queue up peacefully. The Lambda processes them one-by-one (or a few in parallel, depending on configuration). There's no explosion, no timeouts, no errors. Just a queue that fills up and drains at a steady pace.

Failed builds have a recovery path. Network glitches happen. Cloudflare occasionally throttles requests. With SQS, a failed build automatically retries. If it still fails after a few attempts, it moves to a dead letter queue where I can investigate and replay it later. The user can republish with confidence knowing failure doesn't mean manual intervention.

You get observability for free. Every message in the queue is visible. Failed messages don't disappear—they sit in the DLQ waiting to be examined. I can write queries to find patterns: "Did builds fail more during this time window? Did performance degrade as the queue got deeper?" This data is invaluable.

Priority Queues for Fairness

Here's where it gets interesting: not all builds are equally urgent.

Jottings uses three priority tiers:

High priority: You just published something. You want to see it live soon. These go to the front of the line.

Normal priority: Your site is being accessed by readers. A build triggered by some background process. Not as urgent as a fresh publish, but still important.

Low priority: Maintenance work. Regenerating old pages. Cleanup tasks. These can wait.

With a single synchronous process, there's no room for priority. Every request gets the same treatment. But with SQS, I can maintain separate queues and have Lambda prefer high-priority work. You publish something, it regenerates your home page quickly while maintenance tasks patiently wait their turn.

The user experience is better. The infrastructure is more efficient. Everyone wins.

The Dead Letter Queue: Your Safety Net

Here's the part that really matters: what happens when something goes wrong?

Every message has a retry count. If a build fails—maybe Cloudflare R2 is temporarily unavailable—the message automatically goes back into the queue for a retry. This happens a few times. But if it keeps failing, SQS has the decency to send it somewhere it won't cause further problems: the dead letter queue.

The DLQ is like a hospital. Failed messages go there so they don't infect the rest of the system, but they're preserved for examination. I can look at them, understand why they failed, potentially fix the underlying issue, and replay them.

This is how Jottings handles the chaos of distributed systems. Not by preventing failures—that's impossible—but by having a clear path forward when they happen.

The Real Cost: Monitoring

This architecture isn't entirely free. You do need to monitor it.

I've set up CloudWatch dashboards to track queue depth, message age, and DLQ count. If the queue gets backed up or messages languish in the DLQ, I get alerted. This is the tradeoff: I gain resilience and scalability, but I need to actively watch the system.

It's worth it. Reliability requires visibility.

Why This Matters for Jottings

I built Jottings to be the anti-platform. No account limits. No artificial throttling. No surprise downtime.

Queue-based processing is a fundamental part of making that promise real. When you publish, the system doesn't break. When traffic spikes, the system scales. When something fails, the system recovers gracefully.

You don't have to think about any of this. You just write.

That's the goal.

Want to see this in action? Start a microblog on Jottings. Publish something, and watch your site update in real-time—powered by a system designed for reliability, not fragility.

The queue is working for you, even when you can't see it.