Short answer: the cheapest LLM API arrangement for a SaaS marketplace is the one with the lowest reconciled cost per accepted supplier-invoice extraction for each tenant, after retries and fallback attempts; compare a unified key with direct provider accounts only after the application can attribute every attempt to that business result.
That decision rule is stricter than comparing token price cards. OpenRouter, direct OpenAI access, and the Claude API represent possible account and routing boundaries, but their names do not answer the marketplace's accounting question. A route can look inexpensive in aggregate while one tenant's malformed invoice population causes repeated extraction attempts, manual review, or unallocated charges. The system needs to preserve that difference.
Start with chargeback, then choose the route.
Unknown stays unknown.
Why is the cheapest API a tenant accounting question?
A supplier-invoice extraction is useful only after its fields pass the marketplace's acceptance policy. The expected record might include supplier identity, invoice number, currency, dates, line items, tax, and totals. A response that consumed tokens but failed schema or arithmetic validation is still a cost-bearing attempt, yet it isn't an accepted result. Dividing a monthly API total by all uploaded invoices hides that distinction and can make a noisy fallback policy appear efficient.
The denominator should therefore be accepted document revisions, grouped by tenant. The numerator should contain reconciled charges for every attempt associated with those revisions, including attempts that did not become the accepted result. Shared work, such as a prompt-cache warm-up or a batch submitted for several tenants, also needs an explicit allocation rule; otherwise the largest tenant may subsidize the rest merely because the invoice arrived first.
Consider one document revision that is dispatched on route A, produces a schema-valid candidate whose line-item sum does not agree with the invoice total, and is then dispatched on route B under the approved fallback policy. Route B produces the accepted candidate. The tenant ledger must keep both attempts, allocate both settled charges to the same operation, mark only the second attempt as the source of the accepted result, and retain the validation outcome that rejected the first. If the first response arrived after the worker's local deadline, its eventual billing record still belongs to that operation even though no candidate was available at commit time. This example has no universal price result; it shows why a first-call token comparison and a last-response-only log both understate the cost that the marketplace must explain to the tenant.
I would reject any dashboard that silently treats missing usage as zero. Unknown is an accounting state, not a number. Keep the observation provisional until it can be matched with billing evidence, and preserve both the original monetary amount and its currency rather than converting away the audit trail. I'm not sure that two routes are economically comparable when their document cohorts, schemas, acceptance rules, or latency classes differ; a paired evaluation over an authorized, redacted corpus is what would resolve that uncertainty.
This framing also prevents list prices from becoming the whole argument. Token categories, output validity, batch eligibility, retries, and operational ownership can all alter the cost of an accepted extraction. The current commercial terms still need to be checked at decision time, but mutable prices do not belong hard-coded in the architectural conclusion.
Build the chargeback ledger before the router
The accounting boundary needs two durable entities: an operation and its attempts. An operation represents one tenant's extraction of one immutable document revision under one schema and policy version. An attempt represents one dispatch through one configured route. Several attempts may belong to an operation, but no more than one result may become accepted.
Exactly once is an application invariant here, not a promise made by the network. RFC 9110 explains HTTP method idempotency and the conditions under which a client can repeat a request automatically. Model inference is commonly invoked through POST, so the marketplace still needs its own stable operation key and commit rule. Derive the key from tenant ID, document digest, document revision, extraction schema version, and policy version; persist it before dispatch, then enforce uniqueness when accepting a result.
Do not use an invoice number as the key. Different tenants can buy from the same supplier, suppliers can reuse numbering schemes, and corrected documents need separate revisions. A digest alone is also insufficient because the same bytes may legitimately be processed under a new schema. The complete tuple is deliberately boring — audit systems benefit from boring identifiers.
The Go types below keep commercial providers out of domain code while retaining the evidence needed for allocation and reconciliation:
package chargeback
import (
"context"
"time"
)
type ExtractionRequest struct {
OperationID string
TenantID string
DocumentHash string
Revision int64
SchemaVersion string
PolicyVersion string
Payload []byte
}
type TokenUsage struct {
Input int64
Output int64
}
type Attempt struct {
AttemptID string
OperationID string
Route string
Model string
ProviderRequestID string
Usage TokenUsage
OutcomeCode string
StartedAt time.Time
FinishedAt time.Time
}
type Candidate struct {
AttemptID string
Fields map[string]any
}
type Extractor interface {
Extract(context.Context, ExtractionRequest) (Attempt, Candidate, error)
}
Store the model identifier and provider request identifier as observations, not assumptions assembled from local configuration. Retain original usage categories when the billing evidence supplies them, because prematurely folding every category into generic input and output counters can make later reconciliation impossible. Raw invoice content does not belong in general cost telemetry: retain a document digest and policy evidence there, while applying the marketplace's access, encryption, retention, and deletion controls to the source document and extracted fields.
Money deserves the same discipline. Use decimal or rational arithmetic rather than binary floating point, version each rate card, and never rewrite historical observations when a published price changes. A corrected estimate should be a new view over immutable attempts, not an edit that erases what supported an earlier tenant statement.
package chargeback
import "math/big"
type SettledCharge struct {
AttemptID string
Currency string
Amount *big.Rat
RateCard string
AllocationRule string
}
type TenantPeriod struct {
TenantID string
Accepted int64
Charges []SettledCharge
}
func CostPerAccepted(p TenantPeriod) (*big.Rat, bool) {
if p.Accepted == 0 {
return nil, false
}
total := new(big.Rat)
for _, charge := range p.Charges {
total.Add(total, charge.Amount)
}
return total.Quo(total, big.NewRat(p.Accepted, 1)), true
}
A period with no accepted results returns no ratio. Good. Reporting zero would claim success where the system has only expenditure.
How should a Node.js SaaS app compare token cost and fallback?
Keep the runtime boundary language-neutral even when the production app uses Node.js. The product should call an internal extraction operation; adapters can implement direct OpenAI access, direct Claude API access, or a unified-key route such as OpenRouter without leaking account decisions into invoice workflows. The Go contract above makes that separation visible, and the same contract can be exposed to a Node.js worker over an internal queue or service boundary.
Compare candidate routes on the same eligible invoice revisions and with the same schema, prompt digest, validator, deadline, and review policy. Record results by tenant and invoice class, because averages can conceal a route that behaves acceptably on simple domestic invoices and poorly on long, multi-currency documents. Execution order should be rotated when timing could affect the observation, and sensitive data should be used only with the authorization and controls appropriate to the marketplace.
Four measurements are sufficient to expose most misleading comparisons:
| Evidence | Calculation | Decision value |
|---|---|---|
| Accepted-result cost | Reconciled attempt charges / accepted revisions | Measures paid business output rather than first-call price |
| Retry amplification | All attempts / accepted revisions | Exposes cost created by transport and validation policy |
| Tenant allocation coverage | Allocated settled charges / all settled charges | Shows whether chargeback can actually close |
| Review burden | Revisions sent to review / evaluated revisions | Keeps uncertain extraction from masquerading as automation |
Fallback requires a written, versioned policy. Classify a local deadline, an ambiguous delivery, a permanent request rejection, a schema failure, and a domain-validation failure separately; then specify which classes permit another attempt, the maximum attempt count, and the operation-wide deadline. A broad catch-and-forward loop destroys the very evidence the comparison needs. Don't do it.
The acceptance gate should be deterministic wherever invoice semantics allow: required fields are present, dates and currency representations parse, the tenant is authorized to use the supplier relationship, and line arithmetic agrees with totals under an explicitly approved tolerance. A confidence value may inform review, but it should not replace those checks. Once a result commits, later workers must observe the accepted operation and stop rather than dispatching another payable attempt.
Batch work belongs in a separate cohort. The OpenAI Batch API guide describes asynchronous batch processing, so invoices that can wait can be evaluated under that execution mode, while an interactive approval path should remain in its synchronous cohort. Mixing the two and attributing the resulting difference solely to direct versus unified access confounds scheduling with account topology.
Choose the account boundary from governance evidence
A unified key can reduce the number of credentials and adapters the application operates. Direct accounts can preserve provider-specific controls, contractual boundaries, and separate billing ownership. Those are hypotheses to verify against current contracts and documentation, not reasons to declare a universal winner.
The catch is audit evidence. A unified route is not suitable when its normalized records omit a field required for reconciliation, tenant allocation, data governance, or a regulated audit trail. Stick with direct accounts when separate legal ownership, provider-native controls, or independent statements are mandatory. Conversely, direct integrations are a poor fit when the team cannot reliably maintain multiple credential lifecycles, usage adapters, rate-card mappings, and fallback tests; in that case, a unified credential boundary may reduce operational surface, provided its exported evidence passes the same reconciliation test.
No topology removes compliance obligations. Supplier invoices may contain personal or commercially sensitive information, and the applicable retention, deletion, access, residency, and contractual limits must be established for each data path. The architecture should record policy decisions and processing evidence without copying invoice bodies into logs. It should also separate operational observability from an immutable audit record: engineers need useful diagnostics, while auditors need controlled, attributable evidence with defined retention.
This is where per-tenant cost visibility becomes a governance control rather than a chart. An unmatched charge should enter a reconciliation queue with its original route, period, currency, and candidate request identifiers. It must not be distributed across tenants just to make the monthly totals align. Likewise, a disputed or provisional charge should remain distinguishable from a settled one. Accuracy beats cosmetic closure.
Roll out with a shadow ledger, then migrate
Begin by writing operation and attempt records around the existing extraction path without changing routing. Reconcile one complete billing period, measure allocation coverage, and inspect every unmatched category. Next, replay an authorized fixed cohort through candidate adapters, without allowing evaluation output to enter the live invoice workflow. Only after validation and reconciliation rules agree should a small tenant cohort use the versioned fallback policy.
Migration should be reversible at the policy boundary. Pin each operation to the route policy chosen before its first attempt, so a deployment does not send retries for the same invoice revision through two policy versions by accident. Expand by tenant cohort, monitor accepted-result cost and review burden together, and stop expansion when either reconciliation coverage or acceptance behavior leaves the approved range.
The final choice may differ by tenant or workload. That is acceptable. The defensible result is not one globally cheapest logo; it is a routing and account policy whose accepted outputs, fallback attempts, compliance constraints, and settled charges can be explained invoice by invoice.
Reconcile first.