Beta operations

Internal beta runbook

Use this page to issue beta keys, guide testers, collect feedback, triage failures, and decide when ZevRouter is ready for trusted external beta.

Internal beta state
Ready after operator setup
Default model
google/gemini-3.8-flash
Tester path
/docs
Operator path
/operator

Internal Beta Launch Checklist

  1. Confirm staging health and Cloud Run smoke passed for the latest revision.
  2. Create 3-10 named beta keys with low balances, daily limits, RPM limits, and explicit model allowlists.
  3. Send testers the migration guide at https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/docs, their zvr_... key, enabled model IDs, and feedback template.
  4. Ask every tester to run one Playground request, one app migration request, one credits check, one activity check, and one generation drilldown.
  5. Monitor /operator, /activity, /credits, and /generation during the first session.
  6. Revoke or throttle keys immediately if a tester reports runaway usage, confusing billing, or unexpected model access.
  7. Collect feedback into the categories below before deciding whether to expand the cohort.

Launch Packet

Create one private launch packet before inviting testers. Do not store plaintext keys in the packet.

Date:
Operator:
Staging URL: https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app
Cloud Run revision:
Support channel:
Feedback doc/channel:
Billing mode: manual / Stripe test / Stripe live

[ ] Cloud Run smoke passed.
[ ] Beta runbook smoke passed.
[ ] Non-admin rehearsal passed.
[ ] Tester roster reviewed.
[ ] Keys issued with low limits.
[ ] Plaintext keys delivered separately from docs.
[ ] Operator console open during first session.
[ ] Revoke/throttle plan confirmed.

Key Issuance Ledger

Track issued keys by key id and status only. Never paste the full zvr_... secret into a shared tracker.

TesterWorkflowKey IDModelsLimitsFirst RequestStatus
NameSDK migrationusr_...google/gemini-3.8-flash$2 balance / $5 hard / $2 daily / 20 RPMpendingactive

Issue A Beta Key

1. Open https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/operator
2. Enter the operator token.
3. Create a named key:
   - name: tester or project name
   - credits: $5 to $20 for internal beta
   - daily limit: $1 to $5
   - RPM: 10 to 30
   - models: google/gemini-3.8-flash
4. Copy the plaintext key once.
5. Send the key separately from the docs link.

Tester Message

You are in the ZevRouter internal beta.

Start here: https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/docs
Base URL: https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/api/v1
Model: google/gemini-3.8-flash
API key: sent separately

Please test:
1. One Playground request.
2. One request from your existing OpenAI/OpenRouter-compatible client.
3. Credits page.
4. Activity page.
5. Generation drilldown.

Send feedback using the template in /beta-runbook.

Feedback Template

Tester:
Date:
Client or app:
Model:
Base URL migrated from:

Result:
- Playground worked: yes/no
- App migration worked: yes/no
- Credits made sense: yes/no
- Activity row appeared: yes/no
- Generation drilldown made sense: yes/no
- Streaming tested: yes/no

Problems:
- Auth/key:
- Model selection:
- Response quality:
- Latency:
- Credits/billing:
- Activity/generation metadata:
- Docs/UI confusion:

Would you use this instead of OpenRouter for this workflow?
What blocked you from switching?

Feedback Collection Artifact

Use one entry per tester session in the private support channel or feedback doc.

[ZevRouter beta feedback]
Tester:
Workflow:
Date:
Model:
Base URL migrated from:
Client, SDK, or agent:

Result:
- Key loaded on /keys: yes/no
- Playground request worked: yes/no
- Existing app migration worked: yes/no
- Credits made sense: yes/no
- Activity row appeared: yes/no
- Generation drilldown made sense: yes/no
- Streaming tested: yes/no

Problems:
- Auth/key:
- Model selection:
- API compatibility:
- Response quality:
- Latency:
- Credits/billing:
- Activity/generation metadata:
- Docs/UI confusion:

Severity:
P0 blocks all testing / P1 blocks this workflow / P2 confusing but usable / P3 nice to have

Feedback Categories

CategorySeverityExamplesAction
Auth and key loadingP0-P1401 loops, revoked key still works, key session confusionFix before adding testers.
Billing and creditsP0-P1wrong balance, unclear debit, Stripe/session mismatchPause paid testing until resolved.
API compatibilityP1-P2SDK cannot call completions, streaming mismatch, provider preference error unclearPatch docs or compatibility behavior.
Model availabilityP1-P2expected OpenRouter model missing, allowlist unclear, 403 confusionAdd model or document replacement.
ObservabilityP2activity missing row, generation detail unclear, fallback metadata confusingImprove UI copy or drilldown.
Product fitP2-P3user would not switch, workflow unclear, pricing expectation mismatchUse for roadmap decisions.

Support And Triage Loop

  1. Keep a named operator in the support channel while the first tester session is active.
  2. Ask testers to include timestamp, model, request path, and generation id when reporting a failure.
  3. For auth or key issues, check active state, allowed models, balance, hard limit, daily limit, and RPM.
  4. For billing or credit confusion, pause paid testing for that key until the transaction path is understood.
  5. For API compatibility issues, capture the client, sanitized request shape, status code, and sanitized response error.
  6. Revoke or throttle immediately for leaked keys, runaway usage, wrong model access, or confusing billing.
  7. Do not add more testers while any P0/P1 issue remains unresolved.

Operator Triage

  1. Check /operator for active key, spend, payment, and recent usage.
  2. Open /activity?key_id=KEY_ID for the tester key.
  3. Open /generation?id=GENERATION_OR_USAGE_ID from the activity row.
  4. Check model detail at /models/:author/:slug and endpoint metadata at /models/:author/:slug/endpoints.
  5. If cost or abuse risk exists, revoke the managed key or lower its limits.

Useful URLs

https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/health
https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/docs
https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/keys
https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/credits
https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/activity
https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/generation?id=GENERATION_ID
https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app/operator

Non-Admin Rehearsal

Run this once before inviting the first cohort. It uses only a tester key or invite-code signup for the tested path.

npm run smoke:beta-rehearsal
ZEVROUTER_BASE_URL=https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app \
ZEVROUTER_REHEARSAL_API_KEY=zvr_REPLACE_ME \
npm run smoke:beta-rehearsal
ZEVROUTER_BASE_URL=https://zusd-testnet---zevrouter-staging-skwx3z2gxa-uc.a.run.app \
ZEVROUTER_BETA_INVITE_CODE=REPLACE_ME \
npm run smoke:beta-rehearsal

Internal Beta Gate

  • Cloud Run smoke passes on latest staging revision.
  • Docs page has migration snippets and known limitations.
  • Key, credits, activity, and generation pages share loaded-key flow.
  • Operator can issue, monitor, and revoke keys.
  • At least one app migration succeeds end to end.
  • Non-admin rehearsal passes with a tester key or invite-code signup.

Trusted External Beta Gate

  • Internal beta has at least 5 successful workflows and no unresolved P0/P1 bugs.
  • Stripe live or manual billing policy is explicit for the cohort.
  • Support and feedback channel is staffed.
  • Usage limits and abuse response are documented.
  • Terms, privacy, refund, and data-retention language is approved enough for beta.

Public Beta Gate

  • Production domain and environment separation are complete.
  • Monitoring and alerting are active.
  • Live billing and webhook delivery are proven.
  • Automated onboarding and key recovery exist.
  • Code review and slimming pass is complete.

Enabled Models