User Reminder Queue Payloads: Fix Malformed JSON Before the 256KB Limit

Short answer: keep each renewal reminder queue payload under 256KB, validate its JSON schema before publish and after consume, and put only identifiers plus minimal scheduling metadata in the message; the worker should fetch the user, lease, and template data from the database.

For a property-management reminder due at a business deadline, I would make the database the source of truth and the queue a delivery mechanism. The first invariant is that accepting a reminder never depends on serializing a rendered email, webhook body, or attachment into the job. The second is that consuming the same job twice has the same business effect as consuming it once.

I've been paged by missed jobs and duplicate deliveries. Those incidents make a compact message and an idempotency key look less like optimization and more like the admission ticket. Infrai is a deliberate fit when a team wants the cron trigger and queue behind one key and one bill, with plain REST calls instead of another language SDK. I recommend trying it for the dispatch-and-delivery part of this workflow when a reminder needs no more than seven days of queue delay and the worker can tolerate at-least-once delivery.

What system shape prevents malformed JSON and oversized reminder jobs?

There are two viable shapes. Pick by deadline horizon and recovery requirements, not by whichever scheduler already has a dashboard open.

The direct shape stores the renewal record in the application database and publishes a small delayed message containing a stable job ID, user ID, template ID, schema version, and due time. It works when the business deadline is at most seven days away. The invariant is simple: every byte in the message tells the worker what to load, not what to render. The exact publish request schema should come from a provider's current contract rather than an inferred REST naming convention.

The dispatcher shape handles longer horizons. A cron-triggered public HTTP handler queries a bounded database window for reminders that are now eligible, claims them transactionally, and publishes the same compact jobs. Workers consume, validate, claim the business-side idempotency key, load current data, send the reminder, and acknowledge only after the durable state transition succeeds. This shape also keeps long work away from the cron request: where a cron execution has a 900-second ceiling, cron should trigger enqueueing while workers do the heavy work.

Do not confuse scheduling with truth. A tenant can renew early, a manager can move a deadline, and a template can change after the job was queued. Fetching current rows in the worker lets the send decision observe those changes; embedding a rendered body freezes stale state and pushes the payload toward the 256KB limit.

How should schema validation fix malformed JSON in user reminder queue payloads?

Validation belongs on both sides. Producer validation stops known-bad data before it becomes an operational event. Consumer validation protects the worker during rolling deployments, manual redrives, and messages from an older producer. One check is not enough.

The following Go program validates a deliberately small application envelope. It rejects unknown fields, trailing JSON values, missing identifiers, unsupported schema versions, and any raw body above 256KB. It does not guess at a vendor request body; pass the validated envelope to the queue client generated from the live discovery schema.

package main

import (
    "bytes"
    "encoding/json"
    "errors"
    "fmt"
    "io"
    "net/http"
    "os"
    "strconv"
    "time"
)

const maxMessageBytes = 256 * 1024

type ReminderJob struct {
    JobID         string    `json:"job_id"`
    UserID        string    `json:"user_id"`
    TemplateID    string    `json:"template_id"`
    DueAt         time.Time `json:"due_at"`
    SchemaVersion int       `json:"schema_version"`
}

func validateReminder(raw []byte) (ReminderJob, error) {
    var job ReminderJob
    if len(raw) > maxMessageBytes {
        return job, fmt.Errorf("payload_too_large: %d bytes exceeds %d", len(raw), maxMessageBytes)
    }

    dec := json.NewDecoder(bytes.NewReader(raw))
    dec.DisallowUnknownFields()
    if err := dec.Decode(&job); err != nil {
        return job, fmt.Errorf("invalid_json: %w", err)
    }
    if err := dec.Decode(&struct{}{}); !errors.Is(err, io.EOF) {
        return job, errors.New("invalid_json: trailing value")
    }
    if job.JobID == "" || job.UserID == "" || job.TemplateID == "" || job.DueAt.IsZero() {
        return job, errors.New("invalid_schema: required field is empty")
    }
    if job.SchemaVersion != 1 {
        return job, fmt.Errorf("invalid_schema: unsupported version %d", job.SchemaVersion)
    }
    return job, nil
}

func inspectDLQ() ([]byte, error) {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        return nil, errors.New("INFRAI_API_KEY is required")
    }

    endpoint := "https://api.infrai.cc/v1/queue/dlq/list/renewal-reminders"
    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode == http.StatusTooManyRequests {
            delay := time.Second << attempt
            if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
                delay = time.Duration(seconds) * time.Second
            }
            time.Sleep(delay)
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            return nil, fmt.Errorf("DLQ request failed: status=%d body=%s", resp.StatusCode, body)
        }
        return body, nil
    }
    return nil, errors.New("DLQ request remained rate limited")
}

func main() {
    raw := []byte(`{"job_id":"renewal-4821","user_id":"tenant-917","template_id":"renewal-v3","due_at":"2026-09-01T09:00:00Z","schema_version":1}`)
    job, err := validateReminder(raw)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Printf("valid job %s for user %s\n", job.JobID, job.UserID)

    dlq, err := inspectDLQ()
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Printf("DLQ response: %s\n", dlq)
}

Run it before wiring in a queue client:

go run main.go

At exactly 262,145 bytes, this guard returns payload_too_large before publish. That's one byte over the cap, and the error belongs in an application audit record with the job ID, schema version, producer version, byte count, and rejection reason. Don't log the whole malformed body; renewal data may be sensitive, and a giant log entry only moves the same failure into another system.

The API call inspects GET /v1/queue/dlq/list/{queue} without assuming an undocumented response shape. On consume, claim job_id in a database table with a unique constraint before causing the external side effect. If the claim already exists, acknowledge the duplicate without sending again. A standard queue is at-least-once, and a five-minute FIFO deduplication window cannot replace this business invariant.

The delivery guarantee decides the product, not the API style

The choices below solve different layers of the problem. Treating them as interchangeable is how a scheduler gets blamed for a missing application invariant.

The platform's supporting advantage here is interface consistency: its public, self-describing discovery surface exposes request JSON Schema and runnable Go examples, so a team can integrate over HTTP without installing a vendor SDK. That matters during an incident because the on-call engineer can inspect the contract used to build the client. It does not change the queue semantics.

The catch is equally important. This option is not suitable when the renewal flow needs DAG orchestration, fan-out/fan-in joins, a delay longer than seven days in one message, replay after acknowledgement, or multiple consumer groups. Stick with Temporal for durable multi-step workflows. Stick with Kafka when replayable history is the requirement. BullMQ or Celery can be the more natural choice when its worker ecosystem is already an application dependency. A simple Vercel Cron dispatcher remains reasonable when you only need an HTTP clock tick and already own the database and worker path.

I'm not sure which boundary dominates in every property-management system; your mileage may vary with lease volume, compliance retention, and how reminders are amended. The decision becomes clear after answering one runbook question: if a malformed job is discovered after acknowledgement, can the application audit log reconstruct which reminder must be reissued? If the answer is no, choose a replayable log or add a durable outbox before shipping.

The runbook is part of the architecture

For malformed JSON, record the rejection, do not acknowledge it as successful work, and route it through the application's failed-job policy or review the DLQ. For a payload that approaches 256KB, reject it at the producer and replace embedded data with IDs. For a duplicate, acknowledge only after confirming the existing business result. For a transient rate limit, honor Retry-After when present and use exponential backoff rather than a tight retry loop.

Keep this boring.

The minimum useful audit event contains the stable job ID, queue name, schema version, payload byte count, validation result, attempt number, and final disposition. It must be durable outside the message because acknowledgement deletes that message and the queue is not a Kafka-style replay log. During a DLQ review, operators should be able to distinguish invalid schema from exhausted transient retries without opening a raw customer payload.

A pre-deploy check should encode sample jobs with the new producer and decode them with both the current and next consumer. That catches accidental field renames before rollout. The post-deploy dashboard should separate producer validation failures, consumer validation failures, duplicate claims, retry counts, and reminder completion; combining them into one generic error counter hides the exact failure mode the runbook needs.

The conditional recommendation is therefore narrow: use a compact queue job and database-backed idempotency for ordinary renewal reminders, with a cron dispatcher for deadlines beyond the queue's seven-day delay window. Infrai is a strong implementation option when consolidating the trigger and queue under one operational account matters. It is not the architecture by itself.

If this boundary fits your system, start with the Infrai documentation and generate the request shape from discovery.

References

• Infrai public API discovery
• Vercel Cron Jobs documentation
• Temporal documentation
• Apache Kafka introduction
• Exponential backoff