Treating "failed" as terminal: idempotency when you cannot ask the recipient

typescript dev.to

Our app sends messages to landlords on a user's behalf. Somewhere in the state machine there is a set of statuses that mean "never touch this listing again", and one entry in it looks wrong every time someone reads it:

/**
 * Statuses that mean "this listing has been dealt with, never touch it again".
 * `failed` is deliberately included: a submit that errored may still have gone
 * through, and messaging a landlord twice is worse than missing one.
 */
const TERMINAL_STATUSES: ReadonlySet<ReplyStatus> = new Set<ReplyStatus>([
  'queued',
  'in_progress',
  'awaiting_approval',
  'sent',
  'probably_sent',
  'failed',
  'needs_review',
  'duplicate',
]);
Enter fullscreen mode Exit fullscreen mode

failed is terminal. The thing that did not work is the thing we will never retry.

Why that is not a bug

The standard reflex is retry-on-failure, and it is correct whenever you can determine what actually happened. Here we cannot.

Our failure is: the replayer clicked submit, and then something went wrong — a navigation timeout, a torn connection, an unexpected page. What we know is that our side errored. What we do not know is whether the HTTP request reached the server, whether the server processed it, and whether an email is already sitting in a landlord's inbox.

This is the classic uncertainty of a non-idempotent request over an unreliable channel, and it has exactly two safe resolutions: make the operation idempotent, or pick which error you would rather make.

We cannot make it idempotent. The operation is "a stranger's contact form receives a message". There is no idempotency key to send, no API that will deduplicate for us, no way to query whether the previous attempt landed. The recipient is a human with an inbox, and the protocol is a <form>.

So we choose the error. Sending twice means a landlord gets two identical messages from the same applicant minutes apart, which reads as either a bot or someone who cannot operate a website — from a product whose entire purpose is making the user look like a strong, prompt applicant. Missing one means the user does not get that flat, out of the dozens the app will find them that week.

One failure is silent and recoverable. The other is visible to the exact person the user is trying to impress.

probably_sent is a real status

Notice it in the list. Most state machines have sent and failed, because most state machines are written as if the world answers yes or no.

An honest one has a third state for "we submitted, and we could not confirm". Collapsing that into sent overstates what you know. Collapsing it into failed is worse, because failed invites a retry in every reader's head. Giving the uncertainty its own name means the UI can say "we think this went through, check with them" instead of asserting something false in either direction.

Anywhere you have an external side effect you cannot confirm, the outcome type wants three cases, not two. If you only have two, one of them is lying.

Terminal from the moment you start

queued and in_progress are in the set too, which does a second job: it makes the check a lock.

The poll loop runs every 30 seconds and a replay can take up to two minutes. Without those entries, three consecutive cycles would each look at the same listing, see no completed reply, and start a third and fourth attempt — concurrent replays of the same recipe against the same landlord.

Writing the queued row before attempting, and treating it as terminal, means the reservation and the record are the same append. No separate lock table, no lease expiry, no second thing to keep consistent with the first.

The trade is that a process killed mid-reply leaves a queued row that never advances, and that listing is never retried. Which is, again, the error we chose on purpose: the row is visible to the user, and the failure is the safe one.

The question worth stealing

Before writing a retry, ask: if this operation half-happened, can I find out?

If yes, go and find out, then retry precisely.

If no, you are not choosing between "retry" and "give up". You are choosing which of two wrong outcomes you would rather explain to the user — and that decision belongs in a comment next to the code, in the words of the person who made it, because six months later it will look like a bug to everyone including you.

See the state machine in the wild

The reply engine and its status list ship in the desktop app — notifio.app — and every reply attempt is written to a local newline-delimited ledger you can tail -f while it runs. One line per transition, so the whole machine is legible.

If you want to see the shape of the messages it is protecting before installing anything, our guide on what a first message to a landlord should contain is at notifio.app/guides/first-message-to-a-landlord — which is also the clearest explanation of why sending that message twice is a self-inflicted wound.

Source: dev.to

arrow_back Back to Tutorials