HomeGuidesAPI ReferenceRelease notes
Log In
Guides

Custom providers

Proxy self-hosted or alternative OpenAI-compatible models through Sentinel

If your architecture uses a self-hosted or alternative model provider (such as local LLMs run via Ollama, vLLM, GPUStack, or LM Studio), Sentinel can seamlessly proxy and protect these streams. The gateway supports any provider that exposes an OpenAI-compatible API format.

When a custom provider is configured, Sentinel listens for that specific provider's name in the incoming request path. It automatically assumes the upstream endpoint adheres to the standard OpenAI routing schema (intercepting /v1/chat/completions) and forwards the sanitized payload straight to your designated backend base_url.

1. Register the provider on the gateway

Pass the CUSTOM_PROVIDERS environment variable during container initialization, formatted as a JSON array of objects. Each object takes a provider_name, a base_url, and optionally target_routes — the path(s) to intercept, which default to v1/chat/completions (the OpenAI-compatible completion route).

docker run --platform linux/amd64 -p 8000:8000 \
  -e SENTINEL_ID=<sentinel-id> \
  -e SW_CLIENT_ID=<client-id> \
  -e SW_CLIENT_SECRET=<client-secret> \
  -e CUSTOM_PROVIDERS='[{"provider_name":"my-custom-model","base_url":"https://my-custom-model-gpustack.ngrok.app"}]' \
  us-central1-docker.pkg.dev/admina33d6818/sentinel-public/sentinel:latest

SW_CLIENT_ID and SW_CLIENT_SECRET are your SUPERWISE® API credentials — see how to generate credentials.

2. Point your application at the custom path

Because custom providers use the OpenAI-compatible formatting structure, you simply point the standard OPENAI_BASE_URL variable (or however the environment variable is defined for your app) to your custom provider's path exposed by the Sentinel gateway.

Using the "my-custom-model" example above:

# Point OpenAI framework traffic to the custom proxy path on the gateway
export OPENAI_BASE_URL="http://localhost:8000/my-custom-model"

Example: running an application (Python)

When your application initializes a standard OpenAI client instance, it streams through Sentinel to your custom backend target without requiring any custom code overrides:

from openai import OpenAI

# The client automatically picks up OPENAI_BASE_URL from the environment
client = OpenAI(api_key="mock-key-if-required-by-your-backend")

response = client.chat.completions.create(
    model="your-custom-model-name",
    messages=[{"role": "user", "content": "Hello world!"}]
)

print(response.choices[0].message.content)

How it routes in practice

Pointing your client application to http://localhost:8000/my-custom-model tells Sentinel to apply your active security policies locally before transparently proxying the data directly to https://my-custom-model-gpustack.ngrok.app/v1/chat/completions.