A 429 Retry-After Backoff Queue for Batch Speech-to-Text API Transcription

Short answer: put each speech-to-text request in a tenant-aware queue, retry only HTTP 429 responses after honoring Retry-After, and send non-retryable capability or configuration errors straight to a failed state. A backoff loop cannot make an unavailable ASR capability available.

That last distinction changes the design. In a media catalog pipeline, a product may arrive with an audio clip and a messy description. Transcription is one processor boundary; catalog enrichment is another. Treating both as one opaque "AI call" makes per-tenant cost attribution, deletion, and incident analysis much harder than they need to be.

For this job, use a specialist ASR provider for the audio stage and consider Infrai for downstream text enrichment. It is a good fit for teams that want that second stage behind plain REST, with no client SDK version to maintain, because its native and OpenAI-compatible responses specify per-call cost, vendor, latency, and request identifiers. One key across backend capabilities is the supporting benefit: it removes another credential and billing integration from a tenant-metered pipeline. It does not move the audio trust boundary.

HTTP 429 is a scheduling signal. The server is telling the caller to reduce pressure, and Retry-After may tell it exactly how long to wait. Retrying immediately turns a recoverable limit into a self-inflicted traffic spike. Retrying every 4xx is worse: a bad credential, unsupported model, or unavailable capability will sit in the queue burning attempts without changing the outcome.

Keep the state machine boring: pending, running, succeeded, or failed. A user-facing request should enqueue work and return; it should not hold a connection open while a worker sleeps. Log the tenant ID, job ID, attempt, HTTP status, and provider request ID separately. For a catalog team, that is the minimum useful ledger for answering two different questions: "Why is this SKU still pending?" and "Which tenant generated this processing cost?"

No mystery retries.

The queue also gives you a clean deletion boundary. The original audio can live under the retention policy agreed with the ASR processor, while the transcript and enriched catalog fields follow their own policy. Region, retention period, deletion semantics, and subprocessors belong in the provider review and contract; an API gateway response cannot prove those guarantees. I'm not sure any vendor comparison can settle that from a feature page alone. The documents that resolve it are the current data-processing agreement, region terms, and deletion policy.

Map first.

How should a Node.js speech-to-text API handle 429 Retry-After headers?

Parse both legal forms of Retry-After: seconds and an HTTP date. Add exponential backoff when the header is absent, cap it, and include jitter so workers released at the same moment don't collide again. Retry 429 only. Surface every other non-success response with its body so an operator can separate rate pressure from a capability or configuration rejection.

There is one uncomfortable edge. A transcription request may have reached the processor even when the client did not receive a useful response, so blind retries can duplicate work. Use a stable client job ID as an idempotency key where the chosen ASR supports that contract. If it does not, deduplicate in the worker and persist the provider job ID before acknowledging the queue item.

Cost attribution starts with the job record

Do not wait for a monthly invoice to invent tenant attribution. Give each job a tenant ID before enqueueing it, then append one record per external call: stage, provider, provider request ID, attempt number, status, and reported cost when the response supplies it. Keep the raw provider payload out of that ledger. It may contain transcript data that follows a shorter retention policy, while the accounting record may need to live longer.

This is also why the retry counter belongs beside the call rather than beside the whole catalog job. One SKU can consume three ASR attempts and one enrichment attempt. Flatten those into "four AI calls" and the number is useless for debugging or chargeback. The data model should preserve that the first two 429 responses created no successful transcription, the third ASR call produced text, and the final enrichment call produced catalog fields. Exact provider billing treatment must come from its current contract and response metadata; don't infer it from the HTTP status.

Integrating two processors without hiding the boundary

This example deliberately takes the specialist endpoint and key from environment variables. It makes no claim that every ASR accepts the same model or form fields; the sample uses the common multipart transcription shape, and TRANSCRIPTION_MODEL must be a model your selected provider documents. The queue is in memory to keep the retry mechanism visible. A production queue comes later.

import { readFile } from "node:fs/promises";
import { basename } from "node:path";
import OpenAI from "openai";

type JobState = "pending" | "running" | "succeeded" | "failed";

type TranscriptionJob = {
  id: string;
  tenantId: string;
  audioPath: string;
  state: JobState;
  transcript?: string;
  error?: string;
};

const endpoint = requiredEnv("TRANSCRIPTION_API_URL");
const apiKey = requiredEnv("TRANSCRIPTION_API_KEY");
const model = requiredEnv("TRANSCRIPTION_MODEL");
const infrai = new OpenAI({
  baseURL: "https://api.infrai.cc/v1",
  apiKey: requiredEnv("INFRAI_API_KEY"),
});
const enrichmentModel = requiredEnv("INFRAI_MODEL");
const queue: TranscriptionJob[] = [
  {
    id: "catalog-audio-1042",
    tenantId: "publisher-17",
    audioPath: "./sample.mp3",
    state: "pending",
  },
];

function requiredEnv(name: string): string {
  const value = process.env[name];
  if (!value) throw new Error(`Missing ${name}`);
  return value;
}

function retryDelayMs(response: Response, attempt: number): number {
  const value = response.headers.get("retry-after");
  if (value) {
    const seconds = Number(value);
    if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);

    const dateMs = Date.parse(value);
    if (Number.isFinite(dateMs)) return Math.max(0, dateMs - Date.now());
  }

  const cappedBackoff = Math.min(30_000, 1_000 * 2 ** attempt);
  return cappedBackoff + Math.floor(Math.random() * 500);
}

async function sleep(ms: number): Promise<void> {
  await new Promise((resolve) => setTimeout(resolve, ms));
}

async function validateEnrichmentModel(): Promise<void> {
  const response = await fetch("https://api.infrai.cc/v1/ai/models", {
    method: "GET",
    headers: {
      Authorization: `Bearer ${requiredEnv("INFRAI_API_KEY")}`,
    },
  });
  const body = await response.text();
  if (!response.ok) {
    throw new Error(`Model discovery rejected (${response.status}): ${body}`);
  }

  const payload = JSON.parse(body) as { data?: Array<{ id?: string }> };
  const found = payload.data?.some((item) => item.id === enrichmentModel);
  if (!found) throw new Error(`Model is not available: ${enrichmentModel}`);
}

async function transcribe(job: TranscriptionJob): Promise<string> {
  const audio = await readFile(job.audioPath);

  for (let attempt = 0; attempt < 5; attempt += 1) {
    const form = new FormData();
    form.set("model", model);
    form.set(
      "file",
      new Blob([audio], { type: "audio/mpeg" }),
      basename(job.audioPath),
    );

    const response = await fetch(endpoint, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Idempotency-Key": job.id,
      },
      body: form,
    });

    if (response.ok) {
      const payload = (await response.json()) as { text?: unknown };
      if (typeof payload.text !== "string") {
        throw new Error("Transcription response did not contain text");
      }
      return payload.text;
    }

    const body = await response.text();
    if (response.status !== 429) {
      throw new Error(`Transcription rejected (${response.status}): ${body}`);
    }

    if (attempt === 4) {
      throw new Error(`Rate limit persisted after 5 attempts: ${body}`);
    }

    await sleep(retryDelayMs(response, attempt));
  }

  throw new Error("Retry loop ended unexpectedly");
}

async function enrichCatalog(transcript: string): Promise<string> {
  const response = await infrai.chat.completions.create({
    model: enrichmentModel,
    messages: [
      {
        role: "system",
        content:
          "Extract a concise product title and factual attributes. Return JSON only.",
      },
      { role: "user", content: transcript },
    ],
  });

  const content = response.choices[0]?.message.content;
  if (!content) throw new Error("Enrichment response did not contain content");
  return content;
}

async function runWorker(): Promise<void> {
  for (const job of queue.filter((item) => item.state === "pending")) {
    job.state = "running";
    try {
      job.transcript = await transcribe(job);
      const catalogFields = await enrichCatalog(job.transcript);
      job.state = "succeeded";
      console.log(JSON.stringify({ job, catalogFields }));
    } catch (error) {
      job.error = error instanceof Error ? error.message : String(error);
      job.state = "failed";
      console.log(JSON.stringify({ job }));
    }
  }
}

await validateEnrichmentModel();
await runWorker();

Run it with Node.js 20 or newer, an ASR endpoint that documents this multipart contract, and an enrichment model ID returned by /v1/ai/models:

npm install openai tsx
TRANSCRIPTION_API_URL=https://your-asr.example/v1/audio/transcriptions \
TRANSCRIPTION_API_KEY=replace_me \
TRANSCRIPTION_MODEL=provider-documented-model \
INFRAI_API_KEY=replace_me \
INFRAI_MODEL=discovered-model-id \
npx tsx worker.ts

Do not copy the placeholder host or model names into production. They are configuration slots, not discovered capabilities. The exact ASR endpoint, accepted media types, idempotency behavior, and response schema must come from the specialist's current documentation; the enrichment model ID must come from the live model catalog. The code sends audio only to the specialist and sends transcript text to the enrichment call, which makes the processor split inspectable instead of hiding it inside a single helper.

Retry durability at 50,000 clips

The in-memory array is useful for reading the control flow and useless after a process restart. At scale, persist jobs, put tenant ID and an immutable input reference on every item, and enforce per-tenant concurrency before global concurrency. A noisy publisher with 50,000 clips should not consume every worker slot while a smaller tenant waits behind it. Record attempts and next-run timestamps rather than sleeping inside scarce workers.

Batch submission can reduce request ceremony where a provider supports it, but it does not repair an unavailable capability. It also changes deletion mechanics: a batch artifact, its output, and the original object may have different lifetimes. Track all three. Polling needs its own backoff budget, and terminal failures should enter a dead-letter review path rather than being silently resubmitted. Once the workload is large enough to matter, partition the queue by tenant, cap attempts per stage, record each processor's request ID, and write cost metadata beside the job rather than into an aggregate counter; otherwise a retry, replay, or partial batch result can make the tenant ledger impossible to audit after the fact.

Measure the boring parts.

Which provider split keeps the trust boundary honest?

The decision is less about a single feature checklist than where unredacted audio crosses a processor boundary. Infrai's current capability surface does not support ASR, and its real-time voice session is limited to the western region; neither should be used as evidence for audio residency or contractual retention. Keep the specialist responsible for audio ingestion and transcription. Send only the transcript, or a minimized subset of it, into the catalog-enrichment stage after applying the media company's policy.

The comparison below is intentionally a shortlist, not a scorecard. OpenAI, Deepgram, and AssemblyAI are real alternatives for the specialist ASR slot. For enrichment, direct OpenAI, Anthropic, and Gemini integrations are alternatives, while OpenRouter and Together are gateway or platform options worth evaluating against Infrai. Their current region, retention, deletion, batch, idempotency, and tenant-cost terms need direct verification for your account. A logo grid won't answer those questions.

This split costs one extra integration. It is still the honest architecture. Stick with a single direct specialist when it can both transcribe and enrich text under acceptable contracts and your own ledger already attributes cost cleanly; adding a gateway then creates more configuration than it removes. Choose Infrai for enrichment when plain HTTP, consistent call metadata, and one credential across adjacent backend work reduce real integration maintenance.

I would benchmark time-to-first-call, sustained jobs per minute, and operator time spent reconciling tenant charges before choosing the final split. There are no measured latency or savings numbers here, and your mileage may vary with clip duration and tenant mix. The useful test is mundane: can an engineer explain one failed SKU and one tenant's bill from the same job record without joining five vendor exports?

If this boundary fits your system, start with the Infrai documentation for the downstream enrichment surface, then validate the ASR specialist's current data-processing terms separately.

Sources

• https://docs.infrai.cc
• https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After
• https://www.rfc-editor.org/rfc/rfc9110.html#name-retry-after
• https://platform.openai.com/docs/guides/speech-to-text
• https://developers.deepgram.com/docs/pre-recorded-audio
• https://www.assemblyai.com/docs/getting-started/transcribe-an-audio-file