Scope
This policy applies to BenchRouter's hosted app, API, dashboard, setup flow, eval import flow, and billing flow. BenchRouter is operated by BenchRouter, Inc., a Delaware corporation.
Our Promise
We never store your prompts or responses. Two exceptions: evals you give us as content, in a chat or through the API (cases, reference outputs, scorer settings, judge and incumbent), which we keep with your account; and eval model outputs, which we keep for up to 90 days (in our replay cache and your eval results), so re-running an unchanged eval doesn't call the models or bill you again, and so you can inspect results. The model providers we route to never train on your data, and they may keep it only for abuse monitoring, for up to 30 days, before deleting it.
Every call BenchRouter sends to a model provider, live or eval, goes only to a provider that does not train on API data, keeps data only for abuse monitoring for at most 30 days, and has every optional retention feature BenchRouter controls (such as caching, stored completions, and request logging) turned off. OpenRouter calls ask OpenRouter to use only providers that do not collect data. A direct provider receives calls only while a dated review of our account with it is on record.
Data We Collect
- Account data: GitHub login/account identifiers, Google account identifiers, session data, and account metadata.
- We store your Google account email and name to show which account you use.
- GitHub metadata: installation id, repo full name, branch/ref/PR metadata, workflow run URLs, and commit hashes needed for setup and evals.
- Route snapshot metadata: route declarations, file path labels, eval fingerprints, case counts, code-reference path/hash metadata, and setup/import records.
- Live call telemetry: route id, selected model/provider, cost, latency, token counts, status, cache telemetry, and error metadata.
- Eval run evidence: scorer outcomes, per-case pass/fail results and scores, result sets, pass/fail decisions, cost evidence, and latency evidence. Each per-case result also keeps that case's model output for up to 90 days, so you can inspect it.
- Eval replay cache: eval model outputs, for up to 90 days, so an unchanged eval can replay without calling a model or billing you again.
- Secrets and keys: BenchRouter API keys, provider keys, setup/upgrade tokens, and GitHub tokens/secrets where applicable.
- Billing data: balance, ledger, Stripe checkout/webhook references, saved payment method references and limited card details, usage debits, top-ups, automatic reload settings, and tax/accounting metadata.
- Diagnostics and security data: operational logs, abuse signals, IP address, user agent, and support communications.
- Functional cookies, including benchrouter_session for authentication and session management, benchrouter_anon for first-party analytics identity, benchrouter_analytics_optout when you opt out, and benchrouter_do_not_share when downstream analytics sharing is blocked. Google Analytics may set its own measurement cookies when downstream delivery is allowed.
Live Traffic vs. Eval Evidence
BenchRouter does not persist live routed request or response content in its database. Live calls create telemetry records so the service can route, bill, debug, and measure behavior.
Eval traffic is different. Eval run results, scorer outcomes, per-case pass/fail results and scores, and result sets are stored as the evidence layer for route decisions. Eval model outputs are the one kind of model output text we keep, for up to 90 days: in the eval replay cache, so an unchanged eval can replay without calling a model or billing you again, and in each per-case result, so you can inspect it. After 90 days the output text is deleted and the rest of the result stays. Authored eval files in your repository, such as cases, reference outputs, and scorer logic, are not stored by BenchRouter. Evals you create in a chat or submit through the API (cases, reference outputs, scorer settings, judge and incumbent) are stored with your account until the account is deleted.
How We Use Data
BenchRouter does not train models on customer prompts, outputs, or eval run content by default. Any future model-training use of customer content requires explicit opt-in or a separate agreement.
- Provide routing/proxy behavior and dashboard views.
- Run evals, store proof, and decide route model changes.
- Bill usage, apply credits, and prevent fraud or abuse.
- Secure, monitor, debug, and support the service.
- Improve aggregate model routing/catalog quality using non-content telemetry such as model id, provider, latency, cost, token counts, pass rate, error class, and route outcome.
Sharing and Subprocessors
Model providers receive prompts/outputs when selected to serve a live or eval request, and only under the promise above. Direct model providers are preferred where available; OpenRouter is used as a fallback/gap-filler where a model is not served through a direct provider path.
We also use subprocessors for hosting, GitHub integration, billing, and operations. Personal information may be processed or stored in Canada, the United States, and other countries where BenchRouter, subprocessors, or upstream model providers operate.
Retention
| Data category | Retention |
|---|---|
| Live routed request/response content | Not stored by BenchRouter. |
| Live call telemetry and model_calls | Retained while the account is active, until deleted. |
| Eval run results, scorer outcomes, per-case results, and result sets | Retained while the account is active, until deleted. |
| Eval model output text in per-case results | Up to 90 days, then deleted from the result. |
| Eval replay cache response bodies | Up to 90 days in KV for eval traffic only. |
| Authored eval files in your repository, such as cases and scorer logic | Not stored by BenchRouter. |
| Evals created in a chat or through the API (cases, scorer settings, judge, incumbent) | Retained while the account is active, until deleted. |
| Promotion events | Retained as durable route history until account/route deletion or manual deletion where required. |
| Billing ledger | Retained as needed for accounting, tax, fraud, and legal obligations. |
| Provider/API keys | Retained encrypted or hashed as appropriate until revoked or deleted. |
| GitHub integration metadata | Retained while the account is active, until deleted. |
| First-party analytics events and advertising click IDs | Deleted or cleared after 90 days. |
| First-party analytics visitor identity (UTM, landing path, referrer origin) | Up to one year from first collection, or sooner after a successful browser opt-out. |
First-party analytics
BenchRouter measures its own acquisition and activation funnel with a first-party collector. The collector stores a signed HttpOnly anonymous cookie (benchrouter_anon) for one year, a visitor row, optional person link after you sign in, and a closed event set: page_view, cta_click, and auth_started. Fields are limited to pathname, referrer origin, allowlisted UTM values, and click IDs (gclid, gbraid, wbraid, fbclid, twclid). We do not store full URLs, query strings, user agents, or IP addresses in analytics tables.
Collection is on by default for this narrow, non-sensitive first-party purpose. The Global Privacy Control signal (Sec-GPC) is treated as a do-not-sell-or-share request: we record it on the visitor and will never share that visitor's data with advertising platforms. It does not stop first-party analytics. Use the opt-out below to stop first-party analytics on this browser.
After the first-party collector establishes this browser's privacy state, BenchRouter may send the same closed event names, pathnames, and CTA names to Google Analytics for secondary acquisition reporting. We do not send URL query strings or BenchRouter account identifiers. Google may process measurement cookies and device or network data under its privacy terms. We do not send these events when this browser is opted out, sends Global Privacy Control, or has a sticky do-not-share flag.
Opt out of first-party browsing analytics on this browser. This deletes this browser's analytics visitor, events, first-touch information, and person link. It does not delete your account or records needed to provide, secure, bill, or evaluate the service. Aggregate product metrics may still include activity recorded by those required product systems. Contact us separately to request access to or deletion of account information.
Opt-out expires the anonymous cookie and sets benchrouter_analytics_optout for one year. Activation milestones such as a wired route, a completed PPF, or a model promotion are read from existing product records, not copied into the browser analytics event stream.
Your Rights
You can request access, correction, export, or deletion of personal information by emailing privacy@benchrouter.com. We may retain records needed for legal, billing, tax, security, backup, or fraud-prevention reasons.
BenchRouter does not sell personal information. Upstream providers may hold data under their own terms, and you may need to contact them separately for data they process.
Changes
We may update this policy as the product changes. We will update the date above when we make material changes.