Repository Access
BenchRouter is designed around route snapshots reported by the generated GitHub Actions workflow, not broad source ingestion. GitHub access is scoped through the GitHub App installation selected during setup. The GitHub App does not have Contents:Read; Actions:write is intentionally required so BenchRouter can dispatch customer-owned eval workflows. BenchRouter stores route declarations, fingerprints, path labels, case counts, setup/import metadata, and code-reference path/hash metadata; it does not store source code or authored eval files such as cases and scorer logic.
Live Calls
We never store your prompts or responses. Two exceptions: evals you give us as content, in a chat or through the API (cases, reference outputs, scorer settings, judge and incumbent), which we keep with your account; and eval model outputs, which we keep for up to 90 days (in our replay cache and your eval results), so re-running an unchanged eval doesn't call the models or bill you again, and so you can inspect results. The model providers we route to never train on your data, and they may keep it only for abuse monitoring, for up to 30 days, before deleting it.
Live routed prompts and responses are forwarded to the selected upstream provider and are not persisted by BenchRouter. Live calls create telemetry such as route id, selected model/provider, latency, token counts, cost, status, and error metadata.
Model Providers
Every call BenchRouter sends to a model provider, live or eval, goes only to a provider that does not train on API data, keeps data only for abuse monitoring for at most 30 days, and has every optional retention feature BenchRouter controls (such as caching, stored completions, and request logging) turned off. OpenRouter calls ask OpenRouter to use only providers that do not collect data. A direct provider receives calls only while a dated review of our account with it is on record.
Eval Evidence
Eval run results, scorer outcomes, per-case pass/fail results and scores, and result sets are stored because they are the proof layer for model decisions. Eval model outputs are kept for up to 90 days, in the eval replay cache so an unchanged eval can replay without calling a model or billing you again, and in per-case results so you can inspect them. Authored eval files are not stored by BenchRouter.
Keys and Secrets
- BenchRouter API keys are stored hashed.
- Provider keys are encrypted at rest where BenchRouter stores them.
- Runtime/provider keys are shown only when created or rotated.
- Setup and upgrade tokens are scoped and time-limited.
Infrastructure
- Requests use TLS.
- Access to production systems is limited to operational need.
- Billing/usage metadata is separated from prompt content where practical.
- Upstream providers receive request content only when selected for routing or eval execution.
No Compliance Theater
BenchRouter does not currently claim SOC 2, HIPAA, or GDPR certification. We will publish those claims only if the corresponding controls and reviews exist.
Report a Vulnerability
Email security reports to security@benchrouter.com. Include the affected route/API surface, reproduction steps, impact, and any relevant request ids. Do not include live secrets in the report.