Token optimization

Token optimization is an optional layer that reduces the number of tokens a Sentinel sends upstream — lowering cost and latency — by compressing the prompt before it's forwarded to the model. It's off by default and enabled per Sentinel by an admin.

Headroom compression

Headroom rewrites large blocks of prompt content — tool outputs, logs, files, and retrieved context — to say the same thing in fewer tokens. It's content-aware: it detects what each block actually is (JSON, a diff, a build log, source code, …) and compresses each appropriately, and it only touches blocks above a minimum size.

It runs after your input guardrails pass, before the request reaches the model, on inbound prompts for OpenAI, Anthropic, and Gemini.

Turn it on

1. Open a Sentinel and go to the Token optimization tab.
2. Enable Headroom — once on, it starts by compressing system prompts.
3. Adjust what it's allowed to touch:
   • Compress system prompts — on by default once Headroom is enabled; turn it off to keep your cache prefix stable.
   • Compress user messages — off by default (user messages are usually short).
   • Min tokens — the minimum block size before a block is compressed.

Good to know

• Lossy — compression rewrites wording to save tokens. It aims to preserve meaning and includes an inflation guard: if a rewrite would come out larger, the original is kept.
• Best-effort — savings vary per request and can be zero. Prompts below Min tokens are left untouched, and if compression fails the original prompt is forwarded unchanged.
• Private — only token-savings metadata (tokens before, after, and saved) is reported to SUPERWISE®. The compressed prompt goes to your chosen LLM provider, never to SUPERWISE®. See Data privacy.