Client-side background removal with ONNX Runtime Web — a few things that tripped me up

I built a background remover for my side project that runs the AI model fully client-side — no upload, no account, the
image never leaves the browser tab. I wanted to write a few notes on the bits that actually gave me trouble, because
most client-side ML articles I found glossed over the things that break in production.

The setup

I'm using @imgly/background-removal, which is a nice wrapper around ONNX Runtime Web. It ships two model variants:

isnet_quint8 — 8-bit quantized, around 44 MB
isnet_fp16 — half-precision, bigger but slightly cleaner edges

I default to quint8 in a "Fast" mode and fp16 in a "Best quality" mode. Most users never notice the difference.

The import is lazy so the model only starts downloading when a user drops their first image:

let _imglyPromise: Promise | null = null;
function getImgly() {
if (!_imglyPromise) _imglyPromise = import("@imgly/background-removal");
return _imglyPromise;
}

Simple enough. Now the parts that broke.

CSP wants two unsafe-eval directives, not one

My site has a strict Content-Security-Policy. First time I pushed the background remover to production, it crashed
with:

Refused to compile WebAssembly module because 'wasm-eval' is not an allowed
source of script in the following Content Security Policy directive

ONNX Runtime Web loads its WebAssembly through paths that trigger both the JS eval guard AND the WASM eval guard. You
need BOTH of these in your script-src:

'unsafe-eval' 'wasm-unsafe-eval'

I don't love it. If anyone has found a way to load ONNX models without 'unsafe-eval', I'd genuinely like to hear it.
For now, wasm-unsafe-eval alone is not enough.

COOP/COEP for threaded inference

ONNX Runtime Web can run multi-threaded via Web Workers, and threaded inference needs SharedArrayBuffer. Chrome only
exposes SharedArrayBuffer on pages that are cross-origin isolated, which needs two response headers:

{ key: "Cross-Origin-Opener-Policy", value: "same-origin" },
{ key: "Cross-Origin-Embedder-Policy", value: "credentialless" },

The credentialless value is the one you want. The stricter require-corp blocks every third-party image embed on the
page unless the remote server sends Cross-Origin-Resource-Policy back — obviously not something you can guarantee on a
tool where users drop in arbitrary images from anywhere.

Heads up: these headers can break Google Fonts if you apply them too broadly. I ended up scoping them to specific tool
routes via Next.js header config so the rest of the site isn't affected.

The hydration bug that took me a week to find

Not ML-specific but it's the one I'm happiest to have caught. I had a free-tier usage counter for AI-generated
backgrounds (3 per day) and wrote this:

const [remaining, setRemaining] = useState(() => {
if (typeof window === "undefined") return 3;
try {
const count = localStorage.getItem("ai-bg-count");
// ... compute from stored date
return Math.max(0, 3 - parseInt(count || "0", 10));
} catch { return 3; }
});

Looks fine. The typeof window check handles SSR, right? Ship to prod. Users occasionally report weird flickering on
the tool page.

Here's the bug: on the server, the initializer returns 3 (no window, fall through). On the client, during hydration,
the initializer runs AGAIN and reads localStorage, so it may return 2. React compares the server HTML to the client
render, sees the mismatch, and throws a hydration error — which tears down the tree and rebuilds it from scratch. Any
tool state the user had at that moment gets reset.

The fix is what every React doc tells you to do and I had ignored:

const [remaining, setRemaining] = useState(3); // stable on server AND first client render

useEffect(() => {
try {
const count = localStorage.getItem("ai-bg-count");
setRemaining(Math.max(0, 3 - parseInt(count || "0", 10)));
} catch {}
}, []);

Initial render is now identical on server and client. The real value lands after hydration via the effect. No
mismatch.

My rule now: if a useState initializer contains typeof window, it's a hydration bug waiting to surface. Move it to a
useEffect.

Mobile OOMs on big images

I only caught this one because Sentry started reporting "tab crashed" events from Android Chrome. A modern phone photo
is easily 48 megapixels, which decodes to roughly 192 MB of raw ImageData per copy. The pipeline allocates multiple
copies (original + mask + result) and, on a 4 GB Android, you OOM before the model even starts. Safari on iOS silently
kills the tab instead of crashing.

The cheap fix is to probe the image with a lightweight Image element before any real pipeline work:

Two things this also gave me for free:

An onerror path, so corrupt files no longer silently stall the UI in a loading state forever.
A matching revokeObjectURL on both code paths, which fixed a slow leak where rejected images were leaving blob URLs pinned in tab memory for the rest of the session.

One more small thing: when you do end up reading pixels from a canvas (for fine edge work), pass willReadFrequently:
true when you create the 2D context:

const ctx = canvas.getContext("2d", { willReadFrequently: true });

It hints to the browser to keep the bitmap CPU-side instead of flushing the GPU pipeline on every getImageData().
Meaningful speedup on mobile.

That's it

If you want to try the result, I have a live version at https://www.allplix.com/en/background-remover. Drop an image,
open DevTools → Network, verify nothing uploads.

Questions or better solutions to any of these, drop a comment.