Tarmac

Workflows

Field-tested recipes — debugging consumers, draining DLQs, replaying poison messages, and developing against LocalStack.

Recipes that combine Tarmac's features into complete jobs. Each assumes you've read Getting Started; the why behind the steps lives in SQS Essentials.

Diagnose a Failing Consumer

Symptoms: DLQ depth climbing, or messages cycling without being processed.

  1. Open Topology on the source queue — confirm which DLQ it feeds and note the Max receives threshold.
  2. Fetch a sample from the source queue (default 10s visibility — you're only looking).
  3. Sort the list and check System ▸ Receive Count on a few messages. High receive counts on old messages mean consumers are receiving but failing — the poison is in the payload or the handler, not connectivity.
  4. Switch to the DLQ, fetch, and open Facet Filters. Facet on error/status fields in the body to bucket failures by cause instead of reading messages one by one.
  5. Inspect a representative from the biggest bucket. The Envelope tab tells you whether an SNS layer is mangling the payload; the S3 tab tells you whether the real payload even still exists.

Deliverable: a failure taxonomy with counts — which is exactly what the fix's PR description needs.

Drain a DLQ After a Fix

  1. Deploy and verify the consumer fix first. Redriving into a broken consumer just increments receive counts.
  2. Select the DLQ and fetch with a long visibility timeout (600–900s) sized to your session — redrive needs live receipt handles at publish time.
  3. Filter to the messages the fix addresses (facets + date range), multi-select, and stage Redrive. For failure classes the fix doesn't cover: stage Delete for the hopeless, Edit & Resend for the salvageable.
  4. Review the Staged tab and Publish before the visibility window closes. Skipped actions (stale handles) → re-fetch, re-stage, publish again.
  5. Live Tail the source queue and watch for boomerangs — redriven messages reappearing in the DLQ mean the fix missed a case.

For an everything-goes-back bulk redrive with no triage, use the AWS console's native DLQ redrive instead — see DLQ & Redrive.

Repair and Replay a Poison Message

A message with malformed data that your consumer can never process:

  1. Fetch with a generous visibility timeout (you'll want to delete the original afterward).
  2. Open the message, copy the body for your bug report, then stage Edit & Resend — fix the payload in the JSON editor (linting catches syntax slips; Format JSON keeps diffs readable) and target the source queue.
  3. Stage Delete for the original? Not possible on the same message — each message holds one staged action, and edit-resend doesn't remove the original. Publish the edit first, confirm the repaired copy processes, then stage and publish the delete.
  4. On FIFO queues, remember the 5-minute dedup window (Staged Actions) — an edited body helps only with content-based deduplication; an explicit dedup ID is preserved and may suppress the resend.

Inspect Production Safely

  • Connect with a read-only profile (see Connecting to AWS) — fetching, caching, searching, and inspecting need zero write permissions.
  • Verify the identity chip in the top bar says the account you think it does.
  • Keep the default 10-second visibility, fetch once, and do all analysis in the local cache — zero further SQS traffic, no competition with consumers.
  • Mind low maxReceiveCount queues: your receives count toward the DLQ threshold like anyone else's.
  • When you're done on a shared machine, Settings ▸ Delete all local data removes cached payloads.

Develop Against LocalStack

  1. In the credentials modal choose AWS-compatible endpoint and apply the LocalStack preset (http://localhost:4566, test/test, path-style S3).
  2. Create queues with awslocal sqs create-queue --queue-name my-queue (Tarmac lists queues; it doesn't create them).
  3. Everything works as against AWS — fetch, live tail, staged actions, even extended-client S3 payloads via LocalStack's S3.
  4. Use Live Tail while exercising your producer code for an instant "did my message actually send, and what does it look like?" loop — the message inspector's unwrapped JSON view catches double-stringified bodies immediately, the classic producer bug.

Watch a Deployment

During a rollout, Live Tail plus the cache make a lightweight canary:

  1. Live Tail the queue your new producer writes to — confirm shape and attributes of fresh messages in real time (tail samples ~1 message per 3 seconds; it's a spot check, not a firehose).
  2. Keep the DLQ selected in a second glance — its rail badge updates as summaries refresh; a moving DLQ depth during rollout is your rollback signal.
  3. After rollout, fetch from both queues and use the date-range filter pinned to the deploy window to compare before/after message shapes.

On this page