LLM Inference Firewall: Blocking Unsafe Tokens Before Generation with Bitmasks

LLM Inference Firewall at the Logits Level

Most LLM safety approaches filter outputs after generation. resk-secure takes a different approach: it blocks forbidden tokens at the logits level before they reach the model.

The Core Idea

Compile safety rules into a GPU-compatible bitmask and apply it to the logits tensor before sampling. Unsafe tokens get zeroed out before they ever appear in the context window.

import torch
from resk_secure import BitmaskFirewall

firewall = BitmaskFirewall()
firewall.block_tokens(["jailbreak", "bypass"])

with torch.no_grad():
    logits = model(input_ids)
    masked_logits = firewall.apply_mask(logits)
    probs = torch.softmax(masked_logits, dim=-1)
    next_token = torch.multinomial(probs, 1)

Architecture Diagram

The pipeline: input tokens go through the model, logits are masked with the safety bitmask, and only safe tokens proceed to sampling.

Why Bitmasks?

• O(1) per-token overhead instead of linear scan of generated output
• Catches blocklist patterns even across token boundaries with regex
• Works with streaming inference
• CPU and CUDA backends
• Default blocklist for common jailbreak patterns included

Links

• GitHub: https://github.com/resk-security/resk-secure
• PyPI: https://pypi.org/project/resk-secure
• Docs: https://resk.fr

LLM safety should be built into the architecture, not bolted on after generation.