{"openapi":"3.1.0","jsonSchemaDialect":"https://json-schema.org/draft/2020-12/schema","info":{"title":"allevals API","version":"2026-10-28","description":"Evaluate a saved company endpoint with one request. Keys are workspace-scoped: never supply a tenant ID. Native decisions use a distinct text-choice protocol, not a generative leaderboard reproduction. Custom endpoints, OpenRouter with a customer key, and customer-owned Cloudflare AI Gateway bill the configured platform fee per successfully graded row (right or wrong). Customers pay their provider directly; AllEvals does not fund inference or add a provider markup. Real one-row connection checks follow these same rules; only local synthetic endpoints are free. Item admission allowances are not provider dollars. Unknown provider costs remain pending, never estimated from token counts. An endpoint safety circuit stops new calls after HTTP 401/402/403/404/405 or three consecutive infrastructure failures; wrong answers never trigger it and unknown provider outcomes are not retried. /v1 is the external evaluation API; /api is same-origin browser administration, not a public API-key management surface. This contract does not imply that production email, billing, or benchmark packages are configured."},"servers":[{"url":"/","description":"Current deployment origin; no separate API hostname is assumed."}],"tags":[{"name":"Connections","description":"Read saved model connections without credentials."},{"name":"Evaluations","description":"Asynchronous evaluation and immutable item-level evidence."},{"name":"Benchmarks","description":"Public metadata; source rights and execution gates remain explicit."},{"name":"Endpoints","description":"Browser-session administration of company endpoints."},{"name":"API keys","description":"Browser-session administration; secrets shown only once."},{"name":"Account","description":"Browser-bound email sign-in, profile, and session management."},{"name":"Billing","description":"Stripe checkout, portal, legacy purchase claims, and authoritative state."},{"name":"System","description":"Liveness and machine-readable API contract."}],"paths":{"/health":{"get":{"operationId":"getHealth","tags":["System"],"summary":"Check Worker liveness","description":"This is liveness only, not proof that D1, R2, email, Stripe, or a provider is operational.","security":[],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Health"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/openapi.json":{"get":{"operationId":"getOpenApi","tags":["System"],"summary":"Read this API contract","security":[],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/OpenApiDocument"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/benchmarks":{"get":{"operationId":"listBenchmarks","tags":["Benchmarks"],"summary":"List benchmark metadata and current availability","description":"Public catalog metadata does not require a key or billing setup. Supplied Authorization is ignored, including invalid/expired/revoked keys; catalog calls do not increment per-workspace api_requests. Returns only supported installations with a verified manifest receipt. Reads the published D1 catalog in ascending benchmark ID order. Optional category and endpoint_type filters apply before cursor pagination; the default page size of 100 includes all current entries. Publication is reconciled against R2 installation receipts; discovery does not reread R2 on each request. Each entry contains the same descriptive metadata as the detail route. ready means the installation is verified; execution still requires endpoint compatibility, an available executor and account/billing admission. Unknown descriptive facts are null, never inferred from another benchmark edition.","security":[],"parameters":[{"name":"limit","in":"query","schema":{"type":"integer","minimum":1,"maximum":100,"default":100}},{"name":"cursor","in":"query","schema":{"type":"string"},"description":"Copy next_cursor from the preceding page; do not decode or construct cursors. Keep filters unchanged across pages."},{"name":"category","in":"query","schema":{"type":"string","maxLength":100},"description":"Exact category from a catalog entry."},{"name":"endpoint_type","in":"query","schema":{"type":"string","enum":["chat_completions","decisions","completions","responses","loglikelihood","native_predictions","generate_content"]},"description":"Filter by compatible connection protocol, such as chat_completions for chat completion models or decisions for SystemOne decision models. Supply at most once; empty, unknown, or repeated values return 422 invalid_filter."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BenchmarkList"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/benchmarks/{id}":{"get":{"operationId":"getBenchmark","tags":["Benchmarks"],"summary":"Inspect source, license, and protocol caveats","description":"Returns the same provenance, license, metric, capability, configuration and presentation metadata as the list route for one verified installation. Public metadata; supplied Authorization is ignored and not counted as an authenticated workspace API request. Missing or unverified installations return 404. Descriptive metadata does not change the immutable execution manifest or authorize a paid run.","security":[],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Benchmark"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/evaluations":{"post":{"operationId":"createEvaluation","tags":["Evaluations"],"summary":"Evaluate a saved endpoint","description":"Requires evaluations:write for bearer keys. Browser sessions also work and require same-origin Origin for this mutation. The endpoint slug resolves only in your workspace. Freezes the endpoint revision, source/package version, protocol, seed-selected sample IDs, and generation settings. A 202 means durably accepted, not completed. Real package runs enter preparing: chunks are verified and rows materialized asynchronously before inference can start. limit=all requests the full reviewed active corpus (Pro only, at most 20,000 items); numeric limits retain the 1,000 hard cap and plan limits. Admission reserves total_items against the UTC-month sample allowance even if the run later fails or is canceled; this is not provider-dollar accounting. Idempotency-Key is strongly recommended; replaying the same parsed request returns 200, a different request 409. Keys remain reserved for the lifetime of the evaluation. Reuse identical fields/defaults across retries. Optional expected_endpoint_revision and expected_benchmark_version reject changed configuration with 412 before admission/inference. Ordinary real endpoints require expected_collection_mode=prepaid and sufficient wallet funds, a verified saved card and eligible monthly subscription. A separately configured single-run owner verification grant can waive platform fees for its exact private three-item request; it requires an owner browser session and never enables general unpaid execution. Per-call escrow atomically reserves funds before provider work; exhaustion stops new paid work, without replaying completed rows. Reviewed expected_billing_mode and expected_row_unit_price_usd reject changed billing terms with 412 billing_terms_changed; null row rate asserts no platform row fee; it does not waive charges from the customer provider. An existing idempotent replay is returned before checking those preconditions. At most 10 evaluations may be active per workspace; monthly evaluation counts include failed/canceled runs.","security":[{"BearerAuth":[]},{"SessionCookie":[]}],"x-required-scopes":["evaluations:write"],"parameters":[{"name":"Idempotency-Key","in":"header","required":false,"schema":{"type":"string","pattern":"^[A-Za-z0-9._:-]{8,128}$"},"description":"Persist this key before sending so a timeout or process restart can be retried safely."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateEvaluation"}}}},"responses":{"200":{"description":"Existing evaluation returned for an idempotent replay","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Idempotent-Replayed":{"schema":{"type":"string","const":"true"}}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Evaluation"}}}},"202":{"description":"Accepted and durably queued","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Location":{"schema":{"type":"string"},"description":"Relative polling URL."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Evaluation"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"402":{"description":"Insufficient prepaid credit, unavailable billing/card/subscription, plan sample limit, or unavailable full-benchmark access","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"410":{"description":"Retired provider or billing operation; use customer-owned providers and prepaid wallet funding","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"412":{"description":"Endpoint revision or benchmark version precondition no longer matches","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}},"get":{"operationId":"listEvaluations","tags":["Evaluations"],"summary":"List evaluation history","description":"Requires evaluations:read for bearer keys. Newest first, ordered by created_at then opaque ID descending. Filters apply before pagination.","security":[{"BearerAuth":[]},{"SessionCookie":[]}],"x-required-scopes":["evaluations:read"],"parameters":[{"name":"limit","in":"query","schema":{"type":"integer","minimum":1,"maximum":100,"default":25},"description":"Page size, not an evaluation sample count."},{"name":"cursor","in":"query","schema":{"type":"string"},"description":"Copy next_cursor from the preceding page; do not decode or construct cursors. Keep filters unchanged across pages."},{"name":"status","in":"query","schema":{"type":"string","enum":["preparing","queued","running","completed","failed","canceled"]}},{"name":"q","in":"query","schema":{"type":"string","maxLength":100},"description":"Trimmed literal ASCII case-insensitive substring across name, endpoint, benchmark, and evaluation ID. Applied within this workspace before pagination; no wildcard syntax."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/EvaluationList"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/evaluations/{id}":{"get":{"operationId":"getEvaluation","tags":["Evaluations"],"summary":"Read progress and aggregate result","description":"Requires evaluations:read. completed, failed, and canceled are terminal. preparing means chunk verification/materialization is incomplete and no inference may start. prepared_items and attempted_items distinguish materialized rows from attempted execution. score is null until every selected sample completes without infrastructure errors; multilingual Global-MMLU and MuSR also retain null on completion because their authors define only per-language or per-domain metrics, exposed in author_metrics. A wrong but valid answer scores 0, not a provider failure. Test and sampled runs are explicitly marked.","security":[{"BearerAuth":[]},{"SessionCookie":[]}],"x-required-scopes":["evaluations:read"],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Evaluation"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/evaluations/{id}/results":{"get":{"operationId":"listEvaluationResults","tags":["Evaluations"],"summary":"Inspect sample evidence","description":"Requires evaluations:read. Results are ordered by their frozen sample position, including unfinished/failed/canceled rows. During preparing, only already-materialized rows are listed; an empty page does not mean the planned evaluation has no items. Gold references are available only to the owning workspace and never sent to the endpoint. Known endpoint secret echoes are redacted from persisted provider evidence after scoring; redacted=true marks modified evidence. Persisted provider text is limited to 64 KiB of UTF-8; this is not a limit on the complete result-page response.","security":[{"BearerAuth":[]},{"SessionCookie":[]}],"x-required-scopes":["evaluations:read"],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."},{"name":"limit","in":"query","schema":{"type":"integer","minimum":1,"maximum":100,"default":25},"description":"Page size, not an evaluation sample count."},{"name":"cursor","in":"query","schema":{"type":"string"},"description":"Copy next_cursor from the preceding page; do not decode or construct cursors. Keep filters unchanged across pages."},{"name":"outcome","in":"query","schema":{"type":"string","enum":["all","correct","incorrect","failed","pending","canceled"],"default":"all"},"description":"Applied before pagination. correct/incorrect are completed rows with score 1/0; pending includes queued and running. Keep this filter unchanged across pages."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/EvaluationResultList"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/evaluations/{id}/provenance":{"get":{"operationId":"getEvaluationProvenance","tags":["Evaluations"],"summary":"Download frozen source, protocol, and license notices","description":"Requires evaluations:read. Returns the full frozen provenance object directly, not a page or wrapper. Include this object with exports containing source question text. Exact captured license and conversion notices are preserved; source-card excerpts omit sample question/gold sections, with exact UTF-8 byte ranges and full-card hash. Old evaluations without valid frozen notices return 503 benchmark_provenance_unavailable; no replacement attribution is invented. This is not a new grant of source rights.","security":[{"BearerAuth":[]},{"SessionCookie":[]}],"x-required-scopes":["evaluations:read"],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BenchmarkProvenance"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/evaluations/{id}/cancel":{"post":{"operationId":"cancelEvaluation","tags":["Evaluations"],"summary":"Stop new inference requests","description":"Requires evaluations:write. Browser mutations require the exact application Origin. Repeated cancellation returns the canceled resource. Already completed/failed runs return 409. An in-flight request may finish and be billed; cancellation is not a provider refund or proof of upstream cancellation. Preparing runs can also be canceled before inference. Reserved monthly samples are not refunded.","security":[{"BearerAuth":[]},{"SessionCookie":[]}],"x-required-scopes":["evaluations:write"],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Evaluation"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/connections":{"get":{"operationId":"listConnections","tags":["Connections"],"summary":"List active saved connections without secrets","description":"Requires connections:read for bearer keys; dashboard sessions also work. Returns all active saved model endpoints in your workspace, newest first, at most 50; no pagination. Slugs can be passed as endpoint when creating evaluations. Deleted connections are excluded. Credentials and custom header values are never returned. connection_check describes the latest one-item smoke evaluation for the saved revision, not current provider health or model quality. Listing does not invoke inference.","security":[{"BearerAuth":[]},{"SessionCookie":[]}],"x-required-scopes":["connections:read"],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/EndpointList"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/endpoints":{"get":{"operationId":"listEndpoints","tags":["Endpoints"],"summary":"List active saved endpoints without secrets","description":"Returns all active endpoints, at most 50. Deleted slugs remain reserved to preserve historical identity. No secret values, ciphertext, or key hashes are returned. connection_check reflects the latest one-item smoke evaluation for this endpoint revision; passed means that request completed, not that the model is accurate or currently available.","security":[{"SessionCookie":[]}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/EndpointList"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}},"post":{"operationId":"createEndpoint","tags":["Endpoints"],"summary":"Save an endpoint without invoking inference","description":"Save the complete POST URL; paths are not guessed. Credentials and every custom header value are encrypted and write-only. Public HTTPS and safe header rules are checked; DNS is verified before each actual inference request. Saving does not test provider credentials or imply provider health. A workspace can have 50 active endpoints.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateEndpoint"}}}},"responses":{"201":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"ETag":{"schema":{"type":"string","pattern":"^\"[1-9][0-9]*\"$"},"description":"Quoted endpoint revision. Send as If-Match on a later PATCH."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Endpoint"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"410":{"description":"Retired provider or billing operation; use customer-owned providers and prepaid wallet funding","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/endpoints/{id}":{"patch":{"operationId":"updateEndpoint","tags":["Endpoints"],"summary":"Update the endpoint configuration revision","description":"Omitted values are preserved, including model, API key, and headers. Supplied headers replace the entire set; {} clears it. An empty model clears the default. Setting auth_type=none erases the API key unless a conflicting nonempty key is supplied (rejected). Switching to header authentication requires auth_header unless already using header auth. Slug/type are immutable. Changes affect future runs only. If-Match protects against stale browser edits; a concurrent server-side change returns 409 even without it.","security":[{"SessionCookie":[]}],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."},{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."},{"name":"If-Match","in":"header","schema":{"type":"string","pattern":"^\"[1-9][0-9]*\"$"},"description":"Recommended quoted current revision, e.g. \"3\". A stale revision returns 412."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/UpdateEndpoint"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"ETag":{"schema":{"type":"string","pattern":"^\"[1-9][0-9]*\"$"},"description":"Quoted endpoint revision. Send as If-Match on a later PATCH."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Endpoint"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"410":{"description":"Retired provider or billing operation; use customer-owned providers and prepaid wallet funding","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"412":{"description":"Endpoint revision or benchmark version precondition no longer matches","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}},"delete":{"operationId":"deleteEndpoint","tags":["Endpoints"],"summary":"Remove an endpoint and erase its saved credentials","description":"Active runs must finish or be canceled first. Run snapshots remain private historical evidence; terminal snapshot credentials are erased by maintenance after 1 hour. A deleted slug cannot be reused.","security":[{"SessionCookie":[]}],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."},{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"204":{"description":"Endpoint removed; empty response body","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/keys":{"get":{"operationId":"listApiKeys","tags":["API keys"],"summary":"List masked workspace keys","description":"Returns newest-first cursor pagination. revoked=exclude filters before pagination and does not hide older active keys behind revoked rows. Expired non-revoked keys remain visible. Full tokens and hashes are never returned.","security":[{"SessionCookie":[]}],"parameters":[{"name":"limit","in":"query","schema":{"type":"integer","minimum":1,"maximum":100,"default":25},"description":"Page size, not an evaluation sample count."},{"name":"cursor","in":"query","schema":{"type":"string"},"description":"Copy next_cursor from the preceding page; do not decode or construct cursors. Keep filters unchanged across pages."},{"name":"revoked","in":"query","schema":{"type":"string","enum":["include","exclude"],"default":"include"}}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ApiKeyList"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}},"post":{"operationId":"createApiKey","tags":["API keys"],"summary":"Create a scoped key, revealed only once","description":"Store the returned token securely. Later reads return only prefix/last_four. Up to 25 active, unexpired keys and 20 creations/hour per workspace. Omitted expires_in_days means no expiry. benchmarks:read currently grants no additional access: catalog metadata is public and ignores supplied credentials.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateApiKey"}}}},"responses":{"201":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreatedApiKey"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/keys/{id}":{"delete":{"operationId":"revokeApiKey","tags":["API keys"],"summary":"Revoke a key immediately","description":"Repeated revocation of a key owned by this workspace is safe. Does not cancel already accepted evaluations or revoke browser sessions.","security":[{"SessionCookie":[]}],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."},{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Ok"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/auth/start":{"post":{"operationId":"startSignIn","tags":["Account"],"summary":"Request a browser-bound email sign-in link","description":"No account-existence distinction. Production uses configured transactional email; missing delivery configuration returns 503, never a fake successful send. A 15-minute HttpOnly browser nonce cookie is set. Open the link in the same browser. The optional development_link is returned only for explicitly enabled loopback development, never deployed environments. Account request bodies are limited to 16 KiB.","security":[],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/StartSignIn"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Set-Cookie":{"schema":{"type":"string"},"description":"HttpOnly browser-binding nonce; never exposed to JavaScript."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/SignInRequested"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"502":{"description":"Required upstream service returned an invalid response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/auth/verify":{"post":{"operationId":"verifySignIn","tags":["Account"],"summary":"Consume a one-time sign-in token","description":"Requires the nonce cookie set by startSignIn in this browser. The frontend reads the token from the URL fragment, removes it from browser history, then confirms via POST. Invalid, expired, consumed, or browser-mismatched tokens return400. Local challenges cannot authenticate production. Sets an opaque HttpOnly session and clears the nonce. Session lifetime is 30 days with 7-day idle expiry; at most 20 active browser sessions are retained.","security":[{"SignInCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/VerifySignIn"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Set-Cookie":{"schema":{"type":"string"},"description":"Opaque HttpOnly session cookie; Secure and __Host- prefixed outside loopback development."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Session"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/auth/session":{"get":{"operationId":"getSession","tags":["Account"],"summary":"Read the current browser identity and capabilities","security":[{"SessionCookie":[]}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Session"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/auth/logout":{"post":{"operationId":"signOut","tags":["Account"],"summary":"Revoke this browser session","description":"API keys and other sessions are unchanged.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Ok"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/auth/logout-all":{"post":{"operationId":"signOutEverywhere","tags":["Account"],"summary":"Revoke every browser session for this user","description":"Includes the current session. API keys and saved endpoints are unchanged.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Ok"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/auth/sessions":{"get":{"operationId":"listSessions","tags":["Account"],"summary":"List active browser sessions","description":"Includes current=true on this session. Only non-revoked, unexpired, non-idle-expired sessions appear. Returns the complete list without cursor pagination.","security":[{"SessionCookie":[]}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BrowserSessionList"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/auth/sessions/{id}":{"delete":{"operationId":"revokeSession","tags":["Account"],"summary":"Revoke one browser session","description":"Revoking the current session also clears its cookie. A missing/already-revoked session returns 404.","security":[{"SessionCookie":[]}],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."},{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Ok"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/account":{"patch":{"operationId":"updateAccount","tags":["Account"],"summary":"Update profile and workspace names","description":"Both fields are required; this is not a sparse PATCH. Email address and workspace ownership cannot be changed by this operation.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/UpdateAccount"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Session"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/usage":{"get":{"operationId":"getUsage","tags":["Billing"],"summary":"Read current UTC-month usage","description":"Evaluation count includes all admitted runs. items counts completed+failed samples, not queued/canceled ones. Tokens come from validated provider-reported usage; missing usage is 0, not a dollar-cost estimate. reserved_items sums admitted total_items for the UTC month, including canceled/failed work, without refunds; it is the admission allowance, not provider dollars. attempted_items counts claimed execution attempts, not confirmed provider receipt or billing. api_requests counts best-effort authenticated bearer-key evaluation-route requests, excluding public catalog and browser-session calls; it is not an authoritative billing ledger.","security":[{"SessionCookie":[]}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/UsageSummary"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing":{"get":{"operationId":"getBilling","tags":["Billing"],"summary":"Read effective plan and billing readiness","description":"Pro requires an eligible active/trialing configured-price subscription with a future entitlement end. A checkout redirect does not grant access. Usage readiness is separate and stricter: an eligible active monthly automatic-collection subscription, verified saved card, prepaid consent and sufficient available wallet credit. No metered subscription item may remain attached; usage is deducted from the wallet, not added to a monthly usage invoice. Legacy import entitlement is separate from hosted-evaluation Pro access.","security":[{"SessionCookie":[]}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BillingSummary"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/v1/evaluations/{id}/cost":{"get":{"operationId":"getEvaluationCost","tags":["Billing"],"summary":"Read durable evaluation billing evidence","description":"Exact USD decimal strings. New custom, OpenRouter BYOK and Cloudflare totals include platform row fees only; provider inference is billed separately. Retired managed records remain readable, with null totals while historical cost is unresolved. Prepaid collection exposes reserved_usd, settled_usd and wallet_status. Reserved funds are unavailable until settled or released; unresolved historical provider-cost reservations remain held. meter_status is retained only for historical evidence and does not schedule new invoiced usage. Completed rows survive cancellation; unexecuted rows and provider/transport/parsing/scoring errors generate no row fee; correctly graded wrong answers remain billable.","security":[{"BearerAuth":[]},{"SessionCookie":[]}],"parameters":[{"name":"id","in":"path","required":true,"schema":{"type":"string"},"description":"Opaque ID returned by this API."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/InferenceCost"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"404":{"description":"Resource not found in the authenticated workspace","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/inference-budget":{"post":{"operationId":"setInferenceBudget","tags":["Billing"],"summary":"Retired postpaid managed budget","deprecated":true,"security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"410":{"description":"Retired provider or billing operation; use customer-owned providers and prepaid wallet funding","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/enable-usage":{"post":{"operationId":"enableUsageBilling","tags":["Billing"],"summary":"Retired postpaid usage activation","deprecated":true,"security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"410":{"description":"Retired provider or billing operation; use customer-owned providers and prepaid wallet funding","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/wallet/funding":{"get":{"operationId":"getWalletFunding","tags":["Billing"],"summary":"Read verified card and auto-reload settings","security":[{"SessionCookie":[]}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/WalletFundingSummary"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/wallet/topups":{"get":{"operationId":"listWalletTopups","tags":["Billing"],"summary":"Read the latest 50 workspace funding attempts","description":"Newest first. has_more signals older records exist; this bounded route does not expose a pagination cursor. An attempt or Checkout return URL is not proof of credited funds.","security":[{"SessionCookie":[]}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/WalletTopupList"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}},"post":{"operationId":"createWalletTopup","tags":["Billing"],"summary":"Open immediate prepaid credit Checkout","description":"USD cents, $5–$1,000. Requires explicit saved-card consent and a stable Idempotency-Key. Retry the same request/key on ambiguous responses, never open a new attempt to bypass pending payment. Only server-verified successful Stripe payment grants credit. Monthly subscription fees are separate.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."},{"name":"Idempotency-Key","in":"header","required":true,"schema":{"type":"string","minLength":8,"maxLength":128}}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/CreateWalletTopup"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/WalletFundingRedirect"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/wallet/card":{"post":{"operationId":"setupWalletCard","tags":["Billing"],"summary":"Open hosted Stripe card setup","description":"Explicit card-saving consent is required. Card setup never grants wallet credit and does not enable automatic reload. Separate auto-reload consent is required.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."},{"name":"Idempotency-Key","in":"header","required":true,"schema":{"type":"string","minLength":8,"maxLength":128}}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/UsageConsent"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/WalletFundingRedirect"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/wallet/auto-reload":{"put":{"operationId":"updateWalletAutoReload","tags":["Billing"],"summary":"Opt in to or disable bounded automatic credit reload","description":"Amounts are USD cents. amount_cents must exceed threshold_cents and fit daily_cap_cents. Enabling requires explicit consent to off-session charges, a verified card and eligible subscription. expected_revision prevents stale updates. Pending/failed/authentication-required payments do not advance credit. Disabling stops future unsubmitted reloads but cannot undo an already submitted payment.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/UpdateWalletAutoReload"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/WalletFundingSummary"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"402":{"description":"Insufficient prepaid credit, unavailable billing/card/subscription, plan sample limit, or unavailable full-benchmark access","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/stripe/wallet-webhook":{"post":{"operationId":"receiveStripeWalletWebhook","tags":["Billing"],"summary":"Receive signed wallet funding and reversal events","description":"Separate signed raw-body destination with replay-safe payment identity. Verified paid evidence, not a redirect or client assertion, grants credits. Refund/dispute events can block further paid usage for reconciliation.","security":[],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/StripeEvent"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/WebhookReceipt"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/stripe/meter-webhook":{"post":{"operationId":"receiveStripeMeterWebhook","tags":["Billing"],"summary":"Receive signed thin meter-error events","description":"Historical signed Stripe thin-event destination retained for old evidence. Prepaid usage does not create or deliver new usage meter events.","security":[],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/StripeEvent"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/WebhookReceipt"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/checkout":{"post":{"operationId":"createCheckout","tags":["Billing"],"summary":"Open the configured Stripe subscription checkout","description":"No request body. Uses only the deployment-configured monthly platform subscription price, not caller-supplied amounts. Prepaid top-ups are separate immediate payments; no usage-meter price is added. Reuses a persisted open checkout identity and checks existing subscriptions to prevent duplicate purchases. A URL is not proof of payment. Missing billing configuration returns 503. Review Stripe pricing before confirming any payment.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BillingRedirect"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/portal":{"post":{"operationId":"createBillingPortal","tags":["Billing"],"summary":"Open the account-bound Stripe billing portal","description":"No request body. Requires an existing Stripe customer for this workspace. This operation opens a portal; it does not itself cancel or change a subscription.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/BillingRedirect"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/billing/claim":{"post":{"operationId":"claimLegacyPurchase","tags":["Billing"],"summary":"Bind an eligible legacy import purchase to this account","description":"The checkout-session ID is never an authentication credential. Must match the configured original Payment Link, verified purchaser email, paid completed one-time payment, correct environment, one-year eligibility window, and no refund/dispute/revocation. Grants only benchmark_imports, not Pro or an unimplemented download endpoint. Legacy delivery cutover remains separately gated.","security":[{"SessionCookie":[]}],"parameters":[{"name":"Origin","in":"header","required":true,"schema":{"type":"string","format":"uri"},"description":"Exact configured application origin. Cross-site mutations are rejected. Browser JavaScript supplies this automatically."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ClaimLegacyPurchase"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/LegacyPurchaseClaim"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"401":{"description":"Authentication missing, expired, or revoked","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"403":{"description":"Insufficient scope, wrong browser origin, or forbidden operation","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"409":{"description":"Conflicting state, idempotency mismatch, or unavailable benchmark","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"415":{"description":"JSON content type required","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"422":{"description":"Invalid evaluation/endpoint/query fields","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"429":{"description":"Rate, monthly evaluation/sample reservation, or active-work limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."},"Retry-After":{"schema":{"type":"integer","minimum":0},"description":"Seconds until retry when available. Workspace quota errors may not have this header."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}},"/api/stripe/webhook":{"post":{"operationId":"receiveStripeWebhook","tags":["Billing"],"summary":"Receive signed Stripe events","description":"Stripe delivery only, not a browser/key operation. Verify the exact raw UTF-8 body with Stripe-Signature and configured webhook secret (300-second tolerance); do not reserialize JSON before signing. Environment mode must match. Event IDs are deduplicated after successful processing. Subscription state is reconciled from current Stripe state. A processing 503 requests Stripe retry. Maximum 1 MiB raw body.","security":[{"StripeSignature":[]}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"$ref":"#/components/schemas/StripeEvent"}}}},"responses":{"200":{"description":"Successful response","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/WebhookReceipt"}}}},"400":{"description":"Malformed body, invalid account fields, invalid sign-in link, or invalid Stripe signature","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"413":{"description":"Request exceeds its byte limit","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"500":{"description":"Unexpected internal failure","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}},"503":{"description":"Required service unavailable or not configured","headers":{"X-Request-Id":{"schema":{"type":"string"},"description":"Safe support correlation identifier; never a credential."},"X-API-Version":{"schema":{"type":"string"}},"Cache-Control":{"schema":{"type":"string"},"description":"API responses are not cached."}},"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}}}}}}},"components":{"securitySchemes":{"BearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"ae_live_<opaque>","description":"A scoped allevals workspace key, not a provider endpoint credential. Local ae_test_ keys cannot authenticate production."},"SessionCookie":{"type":"apiKey","in":"cookie","name":"__Host-allevals_session","description":"Browser session. Loopback development uses allevals_session. Cookie-authenticated mutations also require the exact application Origin."},"SignInCookie":{"type":"apiKey","in":"cookie","name":"__Host-allevals_signin","description":"Short-lived browser-binding nonce from startSignIn. Loopback development uses allevals_signin."},"StripeSignature":{"type":"apiKey","in":"header","name":"Stripe-Signature","description":"Stripe-generated timestamp and HMAC signature over the exact raw request body. This is not a bearer token."}},"schemas":{"Error":{"type":"object","properties":{"error":{"type":"object","properties":{"code":{"type":"string"},"message":{"type":"string"},"request_id":{"type":"string"},"details":{"description":"Optional safe validation fields or retry timing."}},"required":["code","message","request_id"],"additionalProperties":false}},"required":["error"],"additionalProperties":false},"Ok":{"type":"object","properties":{"ok":{"type":"boolean","const":true}},"required":["ok"],"additionalProperties":false},"Health":{"type":"object","properties":{"status":{"type":"string","const":"ok"},"service":{"type":"string","const":"allevals"},"version":{"type":"string"}},"required":["status","service","version"],"additionalProperties":false},"OpenApiDocument":{"type":"object","required":["openapi","info","paths","components"],"properties":{"openapi":{"type":"string","const":"3.1.0"},"info":{"type":"object"},"paths":{"type":"object"},"components":{"type":"object"}}},"CreateEvaluation":{"type":"object","properties":{"endpoint":{"type":"string","pattern":"^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$","maxLength":64,"description":"Saved endpoint slug in this workspace; no repeated provider secrets.","examples":["company-model-v2"]},"benchmark":{"type":"string","minLength":1,"maxLength":100,"examples":["mmlu-test"]},"expected_collection_mode":{"type":"string","enum":["prepaid","owner_verification"],"description":"Real endpoints require prepaid collection, except an explicit, active operator grant for a single owner verification. owner_verification requires a signed-in owner session and an exact connection, model, benchmark version, sample, and Idempotency-Key match. No general fee waiver is available. Missing or changed terms return 412."},"expected_billing_mode":{"type":"string","enum":["custom","openrouter_byok"]},"expected_row_unit_price_usd":{"type":["string","null"],"maxLength":40,"description":"Optional reviewed row-price precondition; null means no platform row fee. A mismatch returns412 without admitting work."},"expected_endpoint_revision":{"type":"integer","minimum":1,"description":"Optional admission precondition. Reject with 412 if this saved endpoint revision changed; existing idempotent replay takes precedence."},"judge_endpoint":{"type":"string","pattern":"^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$","maxLength":64,"description":"Required for AMO, EnterpriseRAG, Original SimpleQA, SimpleQA Verified, native LiveBench local or August MathArena API: explicitly select a separate saved custom endpoint. AMO requires chat completions with model o4-mini; EnterpriseRAG requires Responses with model gpt-5.4; Original SimpleQA requires chat completions with model gpt-4.1-2025-04-14 and one judge call per row. SimpleQA Verified requires a native generate_content candidate whose exact /v1beta/models/MODEL:generateContent URL matches its model, and a separate custom chat judge using gpt-4.1-2025-04-14; both native calls omit generation overrides and the final score is original notebook F1. Native LiveBench requires chat completions with model gpt-5-mini for its optional native math judge. August MathArena requires chat completions with model gemini-3.8-flash, the original max_tokens65536/high-reasoning judge profile, and four candidate attempts with up to three confirmed judge-format attempts each. June MathArena uses native parsing and no judge endpoint. Candidate and judge usage is billed directly by providers; the platform row fee applies once after successful grading. AMO requires 55 judge calls for its full50-task run; EnterpriseRAG judge calls vary with its native correction pipeline. Credentials are frozen privately, never exported."},"expected_judge_endpoint_revision":{"type":"integer","minimum":1,"description":"AMO or EnterpriseRAG: reject admission with 412 if the selected judge endpoint revision changed."},"expected_benchmark_version":{"type":"string","minLength":1,"maxLength":100,"description":"Optional exact admitted package version from the catalog. Reject with 412 if it changed; this is not a source revision or benchmark ID."},"model":{"type":"string","minLength":1,"maxLength":200,"description":"Optional upstream model override; otherwise endpoint default is required."},"name":{"type":"string","minLength":1,"maxLength":120},"metadata":{"type":"object","maxProperties":50,"additionalProperties":true,"description":"Caller-provided metadata with arbitrary JSON values, retained as provided. At most 50 top-level keys and 1,000 object/array nesting levels including its root object; the total request is limited to 1 MiB. Defaults to an empty object. The platform never fills this field from provider prompts, outputs or traces."},"limit":{"oneOf":[{"type":"integer","minimum":1,"maximum":1000},{"type":"string","const":"all"}],"default":25,"description":"Uniform sampling without replacement. Numeric samples are further limited by plan and corpus size. all requests the full active corpus: Pro only and at most 20,000 reviewed items. Existing releases use seed-shuffled order. Global-MMLU Full has a compact source-order implementation but its 589,764 items exceed this unchanged quota; use numeric samples. MuSR requires all 756 questions and executes the original native order independently of the storage sampling seed. Omission means 8 for allevals-smoke-test, 25 otherwise; MuSR rejects omission. Admitted total_items reserves monthly sample allowance, including failed/canceled runs."},"seed":{"type":"integer","minimum":0,"maximum":4294967295,"description":"Optional sampling seed (unsigned 32-bit integer). Omit for a fresh random seed per new evaluation, returned in configuration.seed. The same seed and limit select the same rows and order for the same benchmark version. Idempotent retries retain the original selection. This does not make provider generation deterministic."},"max_output_tokens":{"type":"integer","minimum":1,"maximum":4096,"default":512,"description":"Requested for text generation; native decisions ignore this setting. Author MMLU always uses one output token; ERQA fixes this to 300. GPQA author editions require 1000 and reject other overrides. AMO requires this field explicitly because its author release is grading-only. Global-MMLU uses native likelihood and rejects generation overrides. AutomationBench, ARC and EnterpriseRAG reject this override and preserve their author requests. ARC sends max_completion_tokens 10000. A custom provider may not enforce this request, so it is not a spend cap."},"temperature":{"type":"number","minimum":0,"maximum":2,"default":0,"description":"Requested for text generation; native decisions ignore it. Author MMLU, ERQA and GPQA fix temperature to zero. AMO requires this field explicitly because its author release is grading-only. Global-MMLU uses native likelihood and rejects generation overrides. AutomationBench, ARC and EnterpriseRAG reject this override and preserve their author requests."}},"required":["endpoint","benchmark"],"additionalProperties":false,"examples":[{"endpoint":"company-model-v2","benchmark":"mmlu-test","limit":25,"expected_collection_mode":"prepaid"},{"endpoint":"company-model-v2","benchmark":"mmlu-test","limit":"all","seed":42,"expected_collection_mode":"prepaid"}]},"EvaluationConfiguration":{"type":"object","properties":{"limit":{"oneOf":[{"type":"integer","minimum":1,"maximum":1000},{"type":"string","const":"all"}]},"max_output_tokens":{"type":["integer","null"],"minimum":1,"maximum":4096,"description":"Null when generation settings are owned by the author runner."},"temperature":{"type":["number","null"],"minimum":0,"maximum":2,"description":"Null when generation settings are owned by the author runner."},"seed":{"type":"integer","minimum":0,"maximum":4294967295},"protocol":{"type":"string"},"source_revision":{"type":"string"},"sample_indices_hash":{"type":["string","null"],"pattern":"^[a-f0-9]{64}$","description":"SHA256 of the ordered frozen sample item IDs. Null until complete chunk preparation and item-identity validation succeeds; never infer a completed sample set from partial prepared rows."}},"required":["limit","max_output_tokens","temperature","seed","protocol","source_revision","sample_indices_hash"],"additionalProperties":false},"Evaluation":{"type":"object","properties":{"billing":{"type":"object","properties":{"collection_mode":{"type":"string","enum":["prepaid","legacy_postpaid","fixture","owner_verification"]},"mode":{"type":"string","enum":["custom","openrouter_byok","openrouter_managed"]},"markup_bps":{"type":"integer","minimum":0},"row_unit_price_usd":{"type":["string","null"]},"per_request_reservation_usd":{"type":["string","null"]}},"required":["collection_mode","mode","markup_bps","row_unit_price_usd"],"additionalProperties":false},"id":{"type":"string","pattern":"^eval_[a-f0-9]{32}$"},"name":{"type":"string"},"metadata":{"type":"object","maxProperties":50,"additionalProperties":true},"benchmark":{"type":"string"},"benchmark_version":{"type":"string"},"model":{"type":"string"},"endpoint":{"type":"string"},"endpoint_id":{"type":"string"},"endpoint_type":{"type":"string","enum":["chat_completions","decisions","completions","responses","loglikelihood","native_predictions","generate_content"]},"endpoint_revision":{"type":"integer","minimum":1},"protocol":{"type":"string"},"source_revision":{"type":"string"},"benchmark_items":{"type":"integer","minimum":0},"sampled":{"type":"boolean"},"status":{"type":"string","enum":["preparing","queued","running","completed","failed","canceled"]},"total_items":{"type":"integer","minimum":0},"prepared_items":{"type":"integer","minimum":0,"description":"Number of materialized and locally validated sample rows. Inference is blocked until all planned samples are prepared and final integrity validation succeeds."},"attempted_items":{"type":"integer","minimum":0,"description":"Execution attempt slots claimed, including failures before confirmed provider delivery. Not a count of confirmed provider charges."},"completed_items":{"type":"integer","minimum":0},"failed_items":{"type":"integer","minimum":0},"score":{"type":["number","null"],"minimum":0,"maximum":1,"description":"Native author aggregate (or smoke mean accuracy) only when every selected sample completed without infrastructure errors. Null for partial, failed, or canceled runs, and for completed multilingual Global-MMLU and MuSR protocols whose authors define no overall aggregate. Their per-language or per-domain metrics remain available in author_metrics."},"created_at":{"type":"string","format":"date-time"},"started_at":{"type":["string","null"],"format":"date-time"},"completed_at":{"type":["string","null"],"format":"date-time"},"input_tokens":{"type":"integer","minimum":0},"output_tokens":{"type":"integer","minimum":0},"duration_ms":{"type":["integer","null"],"minimum":0,"description":"Elapsed wall time from the first worker start to completed_at (or now while active), including preparation and later queue intervals but excluding initial admission queue wait. Null before the worker starts."},"model_time_ms":{"type":["integer","null"],"minimum":0,"description":"Sum of recorded per-sample provider latency, separate from elapsed execution wall time."},"error":{"type":["string","null"]},"is_test":{"type":"boolean","description":"True for a synthetic benchmark or test endpoint. Never use these results as model-quality evidence."},"configuration":{"$ref":"#/components/schemas/EvaluationConfiguration"},"provenance":{"anyOf":[{"$ref":"#/components/schemas/BenchmarkProvenanceSummary"},{"type":"null"}],"description":"Lightweight frozen notice identity. Null for older local evaluations without frozen attribution; full notice export then fails closed."},"author_metrics":{"oneOf":[{"$ref":"#/components/schemas/AuthorRunMetrics"},{"$ref":"#/components/schemas/ErqaRunMetrics"},{"$ref":"#/components/schemas/GlobalMmluRunMetrics"},{"$ref":"#/components/schemas/IfbenchRunMetrics"},{"$ref":"#/components/schemas/Hrm8kRunMetrics"},{"$ref":"#/components/schemas/MultilokoRunMetrics"},{"$ref":"#/components/schemas/MmmluRunMetrics"},{"$ref":"#/components/schemas/KinaRunMetrics"},{"$ref":"#/components/schemas/MedxpertqaRunMetrics"},{"$ref":"#/components/schemas/DropRunMetrics"},{"$ref":"#/components/schemas/MmmuRunMetrics"},{"$ref":"#/components/schemas/WinograndeRunMetrics"},{"$ref":"#/components/schemas/T2RagbenchRunMetrics"},{"$ref":"#/components/schemas/SimpleqaRunMetrics"},{"$ref":"#/components/schemas/SimpleqaVerifiedRunMetrics"},{"$ref":"#/components/schemas/SimplevqaRunMetrics"},{"$ref":"#/components/schemas/MrcrRunMetrics"},{"$ref":"#/components/schemas/MusrRunMetrics"},{"$ref":"#/components/schemas/SobRunMetrics"},{"$ref":"#/components/schemas/LivebenchRunMetrics"},{"$ref":"#/components/schemas/KmmluRunMetrics"},{"$ref":"#/components/schemas/MmluReduxRunMetrics"},{"$ref":"#/components/schemas/MmluProRunMetrics"},{"$ref":"#/components/schemas/PencilRunMetrics"},{"$ref":"#/components/schemas/ArxivmathJuneRunMetrics"},{"$ref":"#/components/schemas/ArxivmathHistoricalRunMetrics"},{"$ref":"#/components/schemas/ArxivmathAugustRunMetrics"},{"$ref":"#/components/schemas/DecisionRunMetrics"}]}},"required":["billing","id","name","metadata","benchmark","benchmark_version","model","endpoint","endpoint_id","endpoint_type","endpoint_revision","protocol","source_revision","benchmark_items","sampled","status","total_items","prepared_items","attempted_items","completed_items","failed_items","score","created_at","started_at","completed_at","input_tokens","output_tokens","duration_ms","model_time_ms","error","is_test","configuration","provenance"],"additionalProperties":false},"EvaluationResult":{"type":"object","properties":{"id":{"type":"string"},"evaluation_id":{"type":"string"},"item_id":{"type":"string"},"position":{"type":"integer","minimum":0},"status":{"type":"string","enum":["queued","running","completed","failed","canceled"]},"score":{"type":["number","null"],"minimum":0,"maximum":1},"latency_ms":{"type":["integer","null"],"minimum":0},"input_tokens":{"type":"integer","minimum":0},"output_tokens":{"type":"integer","minimum":0},"error_code":{"type":["string","null"]},"upstream_status":{"type":["integer","null"],"minimum":100,"maximum":599},"started_at":{"type":["string","null"],"format":"date-time"},"completed_at":{"type":["string","null"],"format":"date-time"},"author_metrics":{"oneOf":[{"$ref":"#/components/schemas/AuthorRowMetrics"},{"$ref":"#/components/schemas/IfbenchRowMetrics"},{"$ref":"#/components/schemas/Hrm8kRowMetrics"},{"$ref":"#/components/schemas/MultilokoRowMetrics"},{"$ref":"#/components/schemas/KinaRowMetrics"},{"$ref":"#/components/schemas/MedxpertqaRowMetrics"},{"$ref":"#/components/schemas/DropRowMetrics"},{"$ref":"#/components/schemas/SimpleqaRowMetrics"},{"$ref":"#/components/schemas/SimpleqaVerifiedRowMetrics"},{"$ref":"#/components/schemas/SimplevqaRowMetrics"},{"$ref":"#/components/schemas/MrcrRowMetrics"},{"$ref":"#/components/schemas/MusrRowMetrics"},{"$ref":"#/components/schemas/SobRowMetrics"},{"$ref":"#/components/schemas/LivebenchRowMetrics"},{"$ref":"#/components/schemas/KmmluRowMetrics"},{"$ref":"#/components/schemas/MmluReduxRowMetrics"},{"$ref":"#/components/schemas/PencilRowMetrics"},{"$ref":"#/components/schemas/ArxivmathJuneRowMetrics"},{"$ref":"#/components/schemas/ArxivmathHistoricalRowMetrics"},{"$ref":"#/components/schemas/ArxivmathAugustRowMetrics"}]}},"required":["id","evaluation_id","item_id","position","status","score","latency_ms","input_tokens","output_tokens","error_code","upstream_status","started_at","completed_at"],"additionalProperties":false},"ArxivmathJuneRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^arxivmath-june-2026:([1-9]|[1-3][0-9]|4[0-8])$"},"score":{"type":"number","enum":[0,0.25,0.5,0.75,1]},"correct":{"type":"array","items":{"type":"boolean"},"minItems":4,"maxItems":4}},"required":["id","score","correct"],"additionalProperties":false},"ArxivmathAugustRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^arxivmath-august-2026:([1-9]|[1-4][0-9]|5[0-7])$"},"score":{"type":"number","enum":[0,0.25,0.5,0.75,1]},"correct":{"type":"array","items":{"type":"integer","enum":[0,1]},"minItems":4,"maxItems":4}},"required":["id","score","correct"],"additionalProperties":false},"ArxivmathJuneRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["arxivmath-june-2026-author-chat-v1"]},"count":{"type":"integer","minimum":1,"maximum":48},"attempts":{"type":"integer","minimum":4,"maximum":192,"multipleOf":4},"accuracy":{"type":"number","minimum":0,"maximum":1}},"required":["protocol","count","attempts","accuracy"],"additionalProperties":false},"ArxivmathHistoricalRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^arxivmath-june-2026-historical49:([1-9]|[1-3][0-9]|4[0-9])$"},"score":{"type":"number","enum":[0,0.25,0.5,0.75,1]},"correct":{"type":"array","items":{"type":"boolean"},"minItems":4,"maxItems":4}},"required":["id","score","correct"],"additionalProperties":false},"ArxivmathHistoricalRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["arxivmath-june-2026-historical49-author-chat-v1"]},"count":{"type":"integer","minimum":1,"maximum":49},"attempts":{"type":"integer","minimum":4,"maximum":196,"multipleOf":4},"accuracy":{"type":"number","minimum":0,"maximum":1}},"required":["protocol","count","attempts","accuracy"],"additionalProperties":false},"ArxivmathAugustRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["arxivmath-august-2026-author-api-v1"]},"count":{"type":"integer","minimum":1,"maximum":57},"attempts":{"type":"integer","minimum":4,"maximum":228,"multipleOf":4},"accuracy":{"type":"number","minimum":0,"maximum":1}},"required":["protocol","count","attempts","accuracy"],"additionalProperties":false},"PencilRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^pencil:(0|[1-9][0-9]{0,2})$"},"index":{"type":"integer","minimum":0,"maximum":299},"puzzle_type":{"type":"string","minLength":1,"maxLength":40},"is_success":{"type":"boolean"},"author_error":{"type":"boolean"}},"required":["id","index","puzzle_type","is_success","author_error"],"additionalProperties":false},"PencilRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["pencil-puzzle-direct-ask-local-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":300},"passed":{"type":"integer","minimum":0,"maximum":300},"accuracy":{"type":"number","minimum":0,"maximum":1}},"required":["protocol","count","passed","accuracy"],"additionalProperties":false},"MmluReduxRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^mmlu-redux-ok:(0|[1-9][0-9]{0,3})$"},"score":{"type":"number","enum":[0,1]}},"required":["id","score"],"additionalProperties":false},"MmluProRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["mmlu-pro-apix-stream-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":12032},"correct":{"type":"integer","minimum":0,"maximum":12032},"accuracy":{"type":"number","minimum":0,"maximum":1},"by_category":{"type":"array","items":{"type":"object","properties":{"category":{"type":"string","enum":["business","law","psychology","biology","chemistry","history","other","health","economics","math","physics","computer science","philosophy","engineering"]},"count":{"type":"integer","minimum":1,"maximum":12032},"correct":{"type":"integer","minimum":0,"maximum":12032},"accuracy":{"type":"number","minimum":0,"maximum":1}},"required":["category","count","correct","accuracy"],"additionalProperties":false},"minItems":1,"maxItems":14}},"required":["protocol","count","correct","accuracy","by_category"],"additionalProperties":false},"MmluReduxRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["mmlu-redux-ok-author-chat-v1"]},"count":{"type":"integer","minimum":1,"maximum":5330},"accuracy":{"type":"number","minimum":0,"maximum":1},"subjects":{"type":"array","items":{"type":"object","properties":{"subject":{"type":"string","maxLength":80},"task":{"type":"string","maxLength":100},"count":{"type":"integer","minimum":1,"maximum":100},"accuracy":{"type":"number","minimum":0,"maximum":1}},"required":["subject","task","count","accuracy"],"additionalProperties":false},"minItems":1,"maxItems":57},"groups":{"type":"array","items":{"type":"object","properties":{"group":{"type":"string","enum":["stem","other","social sciences","humanities"]},"count":{"type":"integer","minimum":1,"maximum":5330},"accuracy":{"type":"number","minimum":0,"maximum":1}},"required":["group","count","accuracy"],"additionalProperties":false},"minItems":1,"maxItems":4}},"required":["protocol","count","accuracy","subjects","groups"],"additionalProperties":false},"KmmluRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^kmmlu(?:-hard)?:(0|[1-9][0-9]{0,4})$"},"score":{"type":"number","enum":[0,1]}},"required":["id","score"],"additionalProperties":false},"KmmluRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["kmmlu-direct-author-chat-v1","kmmlu-hard-direct-author-chat-v1"]},"count":{"type":"integer","minimum":1,"maximum":35030},"accuracy":{"type":"number","minimum":0,"maximum":1},"subject_macro":{"type":"number","minimum":0,"maximum":1},"micro":{"type":"number","minimum":0,"maximum":1},"author_metrics":{"type":"array","items":{"type":"object","properties":{"task":{"type":"string","maxLength":100},"subject":{"type":"string","maxLength":80},"count":{"type":"integer","minimum":1,"maximum":35030},"accuracy":{"type":"number","minimum":0,"maximum":1}},"required":["task","subject","count","accuracy"],"additionalProperties":false},"minItems":1,"maxItems":45}},"required":["protocol","count","accuracy","subject_macro","micro","author_metrics"],"additionalProperties":false},"LivebenchRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^livebench:[a-f0-9]{64}$"},"score":{"type":"number","minimum":0,"maximum":1},"author_eval_error":{"type":"boolean"}},"required":["id","score","author_eval_error"],"additionalProperties":false},"LivebenchRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["livebench-legacy-noncoding-native-local-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":682},"accuracy":{"type":"number","minimum":0,"maximum":1},"author_groups_percent":{"type":"object","properties":{"average":{"type":"number","minimum":0,"maximum":100},"data_analysis":{"type":"number","minimum":0,"maximum":100},"instruction_following":{"type":"number","minimum":0,"maximum":100},"math":{"type":"number","minimum":0,"maximum":100},"reasoning":{"type":"number","minimum":0,"maximum":100},"language":{"type":"number","minimum":0,"maximum":100}},"required":[],"additionalProperties":false,"minProperties":1},"author_tasks_percent":{"type":"object","properties":{"tablereformat":{"type":"number","minimum":0,"maximum":100},"cta":{"type":"number","minimum":0,"maximum":100},"tablejoin":{"type":"number","minimum":0,"maximum":100},"paraphrase":{"type":"number","minimum":0,"maximum":100},"simplify":{"type":"number","minimum":0,"maximum":100},"summarize":{"type":"number","minimum":0,"maximum":100},"story_generation":{"type":"number","minimum":0,"maximum":100},"olympiad":{"type":"number","minimum":0,"maximum":100},"AMPS_Hard":{"type":"number","minimum":0,"maximum":100},"math_comp":{"type":"number","minimum":0,"maximum":100},"zebra_puzzle":{"type":"number","minimum":0,"maximum":100},"spatial":{"type":"number","minimum":0,"maximum":100},"connections":{"type":"number","minimum":0,"maximum":100}},"required":[],"additionalProperties":false,"minProperties":1}},"required":["protocol","count","accuracy","author_groups_percent","author_tasks_percent"],"additionalProperties":false},"SobRunMetrics":{"type":"object","properties":{"protocol":{"$ref":"#/components/schemas/SobNative___schema0"},"count":{"$ref":"#/components/schemas/SobNative___schema1"},"score":{"$ref":"#/components/schemas/SobNative___schema2"},"metrics":{"$ref":"#/components/schemas/SobNative___schema3"}},"required":["protocol","count","score","metrics"],"additionalProperties":false},"SobRowMetrics":{"$schema":"https://json-schema.org/draft/2020-12/schema","type":"object","properties":{"id":{"type":"string","pattern":"^sob:(text|image|audio):(0|[1-9][0-9]*)$"},"index":{"type":"integer","minimum":0,"maximum":5323},"metrics":{"minItems":11,"maxItems":11,"type":"array","items":{"type":"number","minimum":0,"maximum":1}},"missing_gt":{"maxItems":544,"type":"array","items":{"type":"integer","minimum":0,"maximum":543}},"missing_required":{"maxItems":46,"type":"array","items":{"type":"integer","minimum":0,"maximum":45}}},"required":["id","index","metrics","missing_gt","missing_required"],"additionalProperties":false},"SobNative___schema0":{"type":"string","enum":["sob-full-openai-author-v1","sob-text-openai-author-v1","sob-image-openai-author-v1","sob-audio-openai-author-v1"]},"SobNative___schema1":{"type":"integer","minimum":1,"maximum":5324},"SobNative___schema2":{"type":"number","minimum":0,"maximum":1},"SobNative___schema3":{"type":"object","properties":{"modalities":{"type":"object","properties":{"text":{"$ref":"#/components/schemas/SobNative___schema4"},"image":{"$ref":"#/components/schemas/SobNative___schema4"},"audio":{"$ref":"#/components/schemas/SobNative___schema4"}},"additionalProperties":false},"leaderboard":{"type":"object","properties":{"overall_adj":{"$ref":"#/components/schemas/SobNative___schema2"},"overall_raw":{"$ref":"#/components/schemas/SobNative___schema2"},"coverage":{"$ref":"#/components/schemas/SobNative___schema2"},"modalities":{"minItems":1,"maxItems":3,"type":"array","items":{"$ref":"#/components/schemas/SobNative___schema36"}},"value_accuracy":{"$ref":"#/components/schemas/SobNative___schema2"},"faithfulness":{"$ref":"#/components/schemas/SobNative___schema2"},"json_pass":{"$ref":"#/components/schemas/SobNative___schema2"},"path_recall":{"$ref":"#/components/schemas/SobNative___schema2"},"structure_cov":{"$ref":"#/components/schemas/SobNative___schema2"},"type_safety":{"$ref":"#/components/schemas/SobNative___schema2"},"perfect":{"anyOf":[{"$ref":"#/components/schemas/SobNative___schema2"},{"type":"null"}]}},"required":["overall_adj","overall_raw","coverage","modalities","value_accuracy","faithfulness","json_pass","path_recall","structure_cov","type_safety","perfect"],"additionalProperties":false}},"required":["modalities","leaderboard"],"additionalProperties":false},"SobNative___schema4":{"type":"object","properties":{"num_records":{"$ref":"#/components/schemas/SobNative___schema5"},"data_quality":{"$ref":"#/components/schemas/SobNative___schema6"},"summary":{"$ref":"#/components/schemas/SobNative___schema7"}},"required":["num_records","data_quality","summary"],"additionalProperties":false},"SobNative___schema5":{"type":"integer","minimum":1,"maximum":5000},"SobNative___schema6":{"type":"object","properties":{"json_parse_fail_count":{"type":"integer","minimum":0,"maximum":5324},"json_non_structured_root_count":{"type":"integer","minimum":0,"maximum":5324},"invalid_schema_input_count":{"type":"integer","minimum":0,"maximum":5324},"unknown_difficulty_count":{"type":"integer","minimum":0,"maximum":5324},"malformed_jsonl_line_count":{"type":"integer","minimum":0,"maximum":5324}},"required":["json_parse_fail_count","json_non_structured_root_count","invalid_schema_input_count","unknown_difficulty_count","malformed_jsonl_line_count"],"additionalProperties":false},"SobNative___schema7":{"type":"object","properties":{"overall":{"type":"object","properties":{"n":{"$ref":"#/components/schemas/SobNative___schema8"},"metrics":{"$ref":"#/components/schemas/SobNative___schema9"},"category_scores":{"$ref":"#/components/schemas/SobNative___schema20"}},"required":["n","metrics","category_scores"],"additionalProperties":false},"overall_weighted":{"type":"object","properties":{"n":{"$ref":"#/components/schemas/SobNative___schema8"},"metrics":{"$ref":"#/components/schemas/SobNative___schema9"},"category_scores":{"$ref":"#/components/schemas/SobNative___schema20"},"weighting":{"type":"string","const":"schema_complexity"},"weight_field_priority":{"type":"array","prefixItems":[{"type":"string","const":"schema_complexity"},{"type":"string","const":"difficulty"}],"items":false,"minItems":2,"maxItems":2},"difficulty_weights":{"type":"object","properties":{"easy":{"type":"number","const":1},"medium":{"type":"number","const":2},"hard":{"type":"number","const":3}},"required":["easy","medium","hard"],"additionalProperties":false}},"required":["n","metrics","category_scores","weighting","weight_field_priority","difficulty_weights"],"additionalProperties":false}},"required":["overall","overall_weighted"],"additionalProperties":false},"SobNative___schema8":{"type":"integer","minimum":1,"maximum":5000},"SobNative___schema9":{"type":"object","properties":{"json_parse_success":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema10"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"json_root_structured":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema11"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"schema_valid_input":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema12"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"schema_compliance":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema13"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"leaf_value_em":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema14"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"value_token_f1":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema15"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"hier_path_recall":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema16"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"path_set_f1":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema17"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"type_precision":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema18"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false},"strict_json_em":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"metric_name":{"$ref":"#/components/schemas/SobNative___schema19"}},"required":["mean","ci95_low","ci95_high","metric_name"],"additionalProperties":false}},"required":["json_parse_success","json_root_structured","schema_valid_input","schema_compliance","leaf_value_em","value_token_f1","hier_path_recall","path_set_f1","type_precision","strict_json_em"],"additionalProperties":false},"SobNative___schema10":{"type":"string","const":"JSON Parse Success"},"SobNative___schema11":{"type":"string","const":"Structured JSON Root"},"SobNative___schema12":{"type":"string","const":"Schema Valid Input"},"SobNative___schema13":{"type":"string","const":"JSON Pass Rate"},"SobNative___schema14":{"type":"string","const":"Truth Score"},"SobNative___schema15":{"type":"string","const":"Faithfulness Score"},"SobNative___schema16":{"type":"string","const":"Path Recall"},"SobNative___schema17":{"type":"string","const":"Structure Coverage"},"SobNative___schema18":{"type":"string","const":"Type Safety"},"SobNative___schema19":{"type":"string","const":"Perfect Response Rate"},"SobNative___schema20":{"type":"object","properties":{"Long Context Extraction":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"category_name":{"$ref":"#/components/schemas/SobNative___schema21"},"components":{"$ref":"#/components/schemas/SobNative___schema22"}},"required":["mean","ci95_low","ci95_high","category_name","components"],"additionalProperties":false},"Complex Schema Handling":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"category_name":{"$ref":"#/components/schemas/SobNative___schema24"},"components":{"$ref":"#/components/schemas/SobNative___schema25"}},"required":["mean","ci95_low","ci95_high","category_name","components"],"additionalProperties":false},"Multi-Context Linking":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"category_name":{"$ref":"#/components/schemas/SobNative___schema27"},"components":{"$ref":"#/components/schemas/SobNative___schema28"}},"required":["mean","ci95_low","ci95_high","category_name","components"],"additionalProperties":false},"Output Contract Reliability":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"category_name":{"$ref":"#/components/schemas/SobNative___schema30"},"components":{"$ref":"#/components/schemas/SobNative___schema31"}},"required":["mean","ci95_low","ci95_high","category_name","components"],"additionalProperties":false},"Strict Precision":{"type":"object","properties":{"mean":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_low":{"$ref":"#/components/schemas/SobNative___schema2"},"ci95_high":{"$ref":"#/components/schemas/SobNative___schema2"},"category_name":{"$ref":"#/components/schemas/SobNative___schema33"},"components":{"$ref":"#/components/schemas/SobNative___schema34"}},"required":["mean","ci95_low","ci95_high","category_name","components"],"additionalProperties":false}},"required":["Long Context Extraction","Complex Schema Handling","Multi-Context Linking","Output Contract Reliability","Strict Precision"],"additionalProperties":false},"SobNative___schema21":{"type":"string","const":"Long Context Extraction"},"SobNative___schema22":{"type":"array","items":{"$ref":"#/components/schemas/SobNative___schema23"}},"SobNative___schema23":{"type":"string","enum":["json_parse_success","json_root_structured","schema_valid_input","schema_compliance","leaf_value_em","value_token_f1","hier_path_recall","path_set_f1","type_precision","required_key_recall","strict_json_em"]},"SobNative___schema24":{"type":"string","const":"Complex Schema Handling"},"SobNative___schema25":{"type":"array","items":{"$ref":"#/components/schemas/SobNative___schema26"}},"SobNative___schema26":{"type":"string","enum":["json_parse_success","json_root_structured","schema_valid_input","schema_compliance","leaf_value_em","value_token_f1","hier_path_recall","path_set_f1","type_precision","required_key_recall","strict_json_em"]},"SobNative___schema27":{"type":"string","const":"Multi-Context Linking"},"SobNative___schema28":{"type":"array","items":{"$ref":"#/components/schemas/SobNative___schema29"}},"SobNative___schema29":{"type":"string","enum":["json_parse_success","json_root_structured","schema_valid_input","schema_compliance","leaf_value_em","value_token_f1","hier_path_recall","path_set_f1","type_precision","required_key_recall","strict_json_em"]},"SobNative___schema30":{"type":"string","const":"Output Contract Reliability"},"SobNative___schema31":{"type":"array","items":{"$ref":"#/components/schemas/SobNative___schema32"}},"SobNative___schema32":{"type":"string","enum":["json_parse_success","json_root_structured","schema_valid_input","schema_compliance","leaf_value_em","value_token_f1","hier_path_recall","path_set_f1","type_precision","required_key_recall","strict_json_em"]},"SobNative___schema33":{"type":"string","const":"Strict Precision"},"SobNative___schema34":{"type":"array","items":{"$ref":"#/components/schemas/SobNative___schema35"}},"SobNative___schema35":{"type":"string","enum":["json_parse_success","json_root_structured","schema_valid_input","schema_compliance","leaf_value_em","value_token_f1","hier_path_recall","path_set_f1","type_precision","required_key_recall","strict_json_em"]},"SobNative___schema36":{"type":"string","enum":["text","image","audio"]},"MusrRowMetrics":{"type":"object","properties":{"index":{"type":"integer","minimum":0,"maximum":755},"ordinal":{"type":"integer","minimum":0,"maximum":755},"choice":{"type":"integer","minimum":1,"maximum":5},"randomly_selected":{"type":"boolean"},"score":{"type":"integer","enum":[0,1]}},"required":["index","ordinal","choice","randomly_selected","score"],"additionalProperties":false},"MusrRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["musr-cot-plus-gpt4-author-v1"]},"count":{"type":"integer","enum":[756]},"domains":{"type":"array","minItems":3,"maxItems":3,"items":{"type":"object","properties":{"domain":{"type":"integer","minimum":0,"maximum":2},"correct":{"type":"integer","minimum":0,"maximum":256},"total":{"type":"integer","enum":[250,256]},"percentage":{"type":"number","minimum":0,"maximum":100,"description":"Original domain accuracy as a percentage rounded to one decimal place. Runtime validation also checks the native count and rounding relationship."}},"required":["domain","correct","total","percentage"],"additionalProperties":false}}},"required":["protocol","count","domains"],"additionalProperties":false},"MrcrRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^mrcr-v2:(64k_128k|128k_256k):[0-9]+$"},"index":{"type":"integer","minimum":0,"maximum":140},"score":{"type":"number","minimum":0,"maximum":1}},"required":["id","index","score"],"additionalProperties":false},"MrcrRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["mrcr-v2-8needle-64k-128k-genai-author-v1","mrcr-v2-8needle-128k-256k-genai-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":141},"accuracy":{"type":"number","minimum":0,"maximum":1},"display_score":{"type":"string","pattern":"^(?:0\\.\\d{4}|1\\.0000)$"}},"required":["protocol","count","accuracy","display_score"],"additionalProperties":false},"SimpleqaVerifiedRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^simpleqa-verified:(0|[1-9][0-9]*)$"},"index":{"type":"integer","minimum":0,"maximum":999},"is_correct":{"type":"boolean"},"is_incorrect":{"type":"boolean"},"is_not_attempted":{"type":"boolean"}},"required":["id","index","is_correct","is_incorrect","is_not_attempted"],"additionalProperties":false},"SimpleqaVerifiedRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["simpleqa-verified-kaggle-genai-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":1000},"correct":{"type":"integer","minimum":0,"maximum":1000},"incorrect":{"type":"integer","minimum":0,"maximum":1000},"not_attempted":{"type":"integer","minimum":0,"maximum":1000},"accuracy_given_attempted":{"type":"number","minimum":0,"maximum":1},"f1":{"type":"number","minimum":0,"maximum":1}},"required":["protocol","count","correct","incorrect","not_attempted","accuracy_given_attempted","f1"],"additionalProperties":false},"SimplevqaRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^simplevqa:(0|[1-9][0-9]*)$"},"index":{"type":"integer","minimum":0,"maximum":2024},"is_correct":{"type":"boolean"},"is_incorrect":{"type":"boolean"},"is_not_attempted":{"type":"boolean"}},"required":["id","index","is_correct","is_incorrect","is_not_attempted"],"additionalProperties":false},"SimplevqaRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["simplevqa-gpt4o-parallel-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":2025},"metrics":{"type":"object","properties":{"LVLM_name":{"type":"string","enum":["g4o_response"]},"is_correct":{"type":"number","minimum":0,"maximum":1},"is_incorrect":{"type":"number","minimum":0,"maximum":1},"is_not_attempted":{"type":"number","minimum":0,"maximum":1},"is_given_attempted":{"type":"number","minimum":0,"maximum":1},"accuracy_given_attempted":{"type":"number","minimum":0,"maximum":1},"f1":{"type":"number","minimum":0,"maximum":1}},"required":["LVLM_name","is_correct","is_incorrect","is_not_attempted","is_given_attempted","accuracy_given_attempted","f1"],"additionalProperties":false}},"required":["protocol","count","metrics"],"additionalProperties":false},"SimpleqaRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^simpleqa:(0|[1-9][0-9]*)$"},"score":{"type":"number","enum":[0,1]},"metrics":{"type":"object","properties":{"is_correct":{"type":"boolean"},"is_incorrect":{"type":"boolean"},"is_not_attempted":{"type":"boolean"}},"required":["is_correct","is_incorrect","is_not_attempted"],"additionalProperties":false}},"required":["id","score","metrics"],"additionalProperties":false},"SimpleqaNativeMetrics":{"type":"object","properties":{"is_correct":{"type":"number","minimum":0,"maximum":1},"is_correct:std":{"type":"number","minimum":0,"maximum":1},"is_incorrect":{"type":"number","minimum":0,"maximum":1},"is_incorrect:std":{"type":"number","minimum":0,"maximum":1},"is_not_attempted":{"type":"number","minimum":0,"maximum":1},"is_not_attempted:std":{"type":"number","minimum":0,"maximum":1},"score:std":{"type":"number","minimum":0,"maximum":1},"is_given_attempted":{"type":"number","minimum":0,"maximum":1},"accuracy_given_attempted":{"type":"number","minimum":0,"maximum":1},"f1":{"type":"number","minimum":0,"maximum":1}},"required":["is_correct","is_correct:std","is_incorrect","is_incorrect:std","is_not_attempted","is_not_attempted:std","score:std","is_given_attempted","accuracy_given_attempted","f1"],"additionalProperties":false},"SimpleqaRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["simpleqa-openai-chat-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":4326},"accuracy":{"type":"number","minimum":0,"maximum":1},"metrics":{"$ref":"#/components/schemas/SimpleqaNativeMetrics"}},"required":["protocol","count","accuracy","metrics"],"additionalProperties":false},"MmmluNativeMetrics":{"type":"object","properties":{"score:std":{"type":"number","minimum":0,"maximum":0.500000000001},"stem":{"type":"number","minimum":0,"maximum":1},"stem:std":{"type":"number","minimum":0,"maximum":0.500000000001},"humanities":{"type":"number","minimum":0,"maximum":1},"humanities:std":{"type":"number","minimum":0,"maximum":0.500000000001},"social_sciences":{"type":"number","minimum":0,"maximum":1},"social_sciences:std":{"type":"number","minimum":0,"maximum":0.500000000001},"other":{"type":"number","minimum":0,"maximum":1},"other:std":{"type":"number","minimum":0,"maximum":0.500000000001}},"required":["score:std"],"additionalProperties":false,"minProperties":3,"dependentRequired":{"stem":["stem:std"],"stem:std":["stem"],"humanities":["humanities:std"],"humanities:std":["humanities"],"social_sciences":["social_sciences:std"],"social_sciences:std":["social_sciences"],"other":["other:std"],"other:std":["other"]}},"MmmluLocaleMetrics":{"type":"object","properties":{"count":{"type":"integer","minimum":1,"maximum":14042},"score":{"type":"number","minimum":0,"maximum":1},"metrics":{"$ref":"#/components/schemas/MmmluNativeMetrics"}},"required":["count","score","metrics"],"additionalProperties":false},"MmmluRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["mmmlu-openai-chatgpt-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":196588},"metrics":{"type":"object","properties":{"locales":{"type":"object","properties":{"AR-XY":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"BN-BD":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"DE-DE":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"ES-LA":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"FR-FR":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"HI-IN":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"ID-ID":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"IT-IT":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"JA-JP":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"KO-KR":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"PT-BR":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"ZH-CN":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"SW-KE":{"$ref":"#/components/schemas/MmmluLocaleMetrics"},"YO-NG":{"$ref":"#/components/schemas/MmmluLocaleMetrics"}},"required":[],"additionalProperties":false,"minProperties":1}},"required":["locales"],"additionalProperties":false}},"required":["protocol","count","metrics"],"additionalProperties":false},"MmmuReportGroup":{"type":"object","properties":{"num":{"type":"integer","minimum":0,"maximum":900},"acc":{"type":"number","minimum":0,"maximum":1,"description":"Original evaluator accuracy rounded to three decimal places."}},"required":["num","acc"],"additionalProperties":false},"MmmuReport":{"type":"object","properties":{"Overall":{"$ref":"#/components/schemas/MmmuReportGroup"},"Accounting":{"$ref":"#/components/schemas/MmmuReportGroup"},"Agriculture":{"$ref":"#/components/schemas/MmmuReportGroup"},"Architecture_and_Engineering":{"$ref":"#/components/schemas/MmmuReportGroup"},"Art":{"$ref":"#/components/schemas/MmmuReportGroup"},"Art_Theory":{"$ref":"#/components/schemas/MmmuReportGroup"},"Basic_Medical_Science":{"$ref":"#/components/schemas/MmmuReportGroup"},"Biology":{"$ref":"#/components/schemas/MmmuReportGroup"},"Chemistry":{"$ref":"#/components/schemas/MmmuReportGroup"},"Clinical_Medicine":{"$ref":"#/components/schemas/MmmuReportGroup"},"Computer_Science":{"$ref":"#/components/schemas/MmmuReportGroup"},"Design":{"$ref":"#/components/schemas/MmmuReportGroup"},"Diagnostics_and_Laboratory_Medicine":{"$ref":"#/components/schemas/MmmuReportGroup"},"Economics":{"$ref":"#/components/schemas/MmmuReportGroup"},"Electronics":{"$ref":"#/components/schemas/MmmuReportGroup"},"Energy_and_Power":{"$ref":"#/components/schemas/MmmuReportGroup"},"Finance":{"$ref":"#/components/schemas/MmmuReportGroup"},"Geography":{"$ref":"#/components/schemas/MmmuReportGroup"},"History":{"$ref":"#/components/schemas/MmmuReportGroup"},"Literature":{"$ref":"#/components/schemas/MmmuReportGroup"},"Manage":{"$ref":"#/components/schemas/MmmuReportGroup"},"Marketing":{"$ref":"#/components/schemas/MmmuReportGroup"},"Materials":{"$ref":"#/components/schemas/MmmuReportGroup"},"Math":{"$ref":"#/components/schemas/MmmuReportGroup"},"Mechanical_Engineering":{"$ref":"#/components/schemas/MmmuReportGroup"},"Music":{"$ref":"#/components/schemas/MmmuReportGroup"},"Pharmacy":{"$ref":"#/components/schemas/MmmuReportGroup"},"Physics":{"$ref":"#/components/schemas/MmmuReportGroup"},"Psychology":{"$ref":"#/components/schemas/MmmuReportGroup"},"Public_Health":{"$ref":"#/components/schemas/MmmuReportGroup"},"Sociology":{"$ref":"#/components/schemas/MmmuReportGroup"},"Overall-Art and Design":{"$ref":"#/components/schemas/MmmuReportGroup"},"Overall-Business":{"$ref":"#/components/schemas/MmmuReportGroup"},"Overall-Science":{"$ref":"#/components/schemas/MmmuReportGroup"},"Overall-Health and Medicine":{"$ref":"#/components/schemas/MmmuReportGroup"},"Overall-Humanities and Social Science":{"$ref":"#/components/schemas/MmmuReportGroup"},"Overall-Tech and Engineering":{"$ref":"#/components/schemas/MmmuReportGroup"}},"required":["Overall","Overall-Art and Design","Overall-Business","Overall-Science","Overall-Health and Medicine","Overall-Humanities and Social Science","Overall-Tech and Engineering"],"additionalProperties":false},"ErqaRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["erqa-openai-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":400},"correct":{"type":"integer","minimum":0,"maximum":400},"accuracy":{"type":"number","minimum":0,"maximum":1},"single_image":{"type":"object","properties":{"count":{"type":"integer","minimum":0,"maximum":287},"correct":{"type":"integer","minimum":0,"maximum":287}},"required":["count","correct"],"additionalProperties":false},"multi_image":{"type":"object","properties":{"count":{"type":"integer","minimum":0,"maximum":113},"correct":{"type":"integer","minimum":0,"maximum":113}},"required":["count","correct"],"additionalProperties":false},"by_type":{"type":"object","properties":{"Action Reasoning":{"type":"object","properties":{"total":{"type":"integer","minimum":1,"maximum":72},"correct":{"type":"integer","minimum":0,"maximum":72}},"required":["total","correct"],"additionalProperties":false},"Multi-view Reasoning":{"type":"object","properties":{"total":{"type":"integer","minimum":1,"maximum":37},"correct":{"type":"integer","minimum":0,"maximum":37}},"required":["total","correct"],"additionalProperties":false},"Other":{"type":"object","properties":{"total":{"type":"integer","minimum":1,"maximum":14},"correct":{"type":"integer","minimum":0,"maximum":14}},"required":["total","correct"],"additionalProperties":false},"Pointing":{"type":"object","properties":{"total":{"type":"integer","minimum":1,"maximum":34},"correct":{"type":"integer","minimum":0,"maximum":34}},"required":["total","correct"],"additionalProperties":false},"Spatial Reasoning":{"type":"object","properties":{"total":{"type":"integer","minimum":1,"maximum":84},"correct":{"type":"integer","minimum":0,"maximum":84}},"required":["total","correct"],"additionalProperties":false},"State Estimation":{"type":"object","properties":{"total":{"type":"integer","minimum":1,"maximum":55},"correct":{"type":"integer","minimum":0,"maximum":55}},"required":["total","correct"],"additionalProperties":false},"Task Reasoning":{"type":"object","properties":{"total":{"type":"integer","minimum":1,"maximum":38},"correct":{"type":"integer","minimum":0,"maximum":38}},"required":["total","correct"],"additionalProperties":false},"Trajectory Reasoning":{"type":"object","properties":{"total":{"type":"integer","minimum":1,"maximum":66},"correct":{"type":"integer","minimum":0,"maximum":66}},"required":["total","correct"],"additionalProperties":false}},"required":[],"additionalProperties":false}},"required":["protocol","count","correct","accuracy","single_image","multi_image","by_type"],"additionalProperties":false},"T2RagbenchRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["t2-ragbench-convfinqa-oracle-native-author-v1","t2-ragbench-finqa-oracle-native-author-v1","t2-ragbench-tat-dqa-oracle-native-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":11349},"accuracy":{"type":"number","minimum":0,"maximum":1},"metrics":{"type":"object","properties":{"number_match":{"type":"number","minimum":0,"maximum":100,"description":"Unchanged author NumberMatch percent; score equals this value divided by 100."}},"required":["number_match"],"additionalProperties":false}},"required":["protocol","count","accuracy","metrics"],"additionalProperties":false},"WinograndeRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["winogrande-dev-native-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":1267},"accuracy":{"type":"number","minimum":0,"maximum":1},"metrics":{"type":"object","properties":{"acc":{"type":"number","minimum":0,"maximum":1}},"required":["acc"],"additionalProperties":false}},"required":["protocol","count","accuracy","metrics"],"additionalProperties":false},"MmmuRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["mmmu-validation-original-json-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":900},"accuracy":{"type":"number","minimum":0,"maximum":1},"report":{"$ref":"#/components/schemas/MmmuReport"}},"required":["protocol","count","accuracy","report"],"additionalProperties":false},"DropRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^drop:[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}$"},"question_count":{"type":"integer","minimum":1,"maximum":2},"exact_match":{"type":"integer","enum":[0,1]},"f1":{"type":"number","minimum":0,"maximum":1}},"required":["id","question_count","exact_match","f1"],"additionalProperties":false},"DropRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["drop-author-grading-v1"]},"count":{"type":"integer","minimum":1,"maximum":9535},"question_count":{"type":"integer","minimum":1,"maximum":9536},"exact_match":{"type":"number","minimum":0,"maximum":1},"f1":{"type":"number","minimum":0,"maximum":1}},"required":["protocol","count","question_count","exact_match","f1"],"additionalProperties":false},"MedxpertqaRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^Text-(0|[1-9]\\d{0,3})$"},"score":{"type":"number","enum":[0,1]},"author_correct":{"type":"boolean"}},"required":["id","score","author_correct"],"additionalProperties":false},"MedxpertqaGroupMetrics":{"type":"object","properties":{"count":{"type":"integer","minimum":1,"maximum":2450},"correct":{"type":"integer","minimum":0,"maximum":2450},"accuracy_percent":{"type":"number","minimum":0,"maximum":100}},"required":["count","correct","accuracy_percent"],"additionalProperties":false},"MedxpertqaRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["medxpertqa-text-ao-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":2450},"author_correct":{"type":"integer","minimum":0,"maximum":2450},"author_accuracy_percent":{"type":"number","minimum":0,"maximum":100},"accuracy":{"type":"number","minimum":0,"maximum":1},"author_question_types":{"type":"object","properties":{"Reasoning":{"$ref":"#/components/schemas/MedxpertqaGroupMetrics"},"Understanding":{"$ref":"#/components/schemas/MedxpertqaGroupMetrics"}},"required":[],"additionalProperties":false}},"required":["protocol","count","author_correct","author_accuracy_percent","accuracy","author_question_types"],"additionalProperties":false},"KinaRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^kina:(0|[1-9]\\d*)$"},"score":{"type":"number","enum":[0,1]},"empty_prediction":{"type":"boolean"}},"required":["id","score","empty_prediction"],"additionalProperties":false},"KinaRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["kina-author-n1-v1"]},"count":{"type":"integer","minimum":1,"maximum":899},"accuracy":{"type":"number","minimum":0,"maximum":1},"empty_prediction_count":{"type":"integer","minimum":0,"maximum":899},"display_percent":{"type":"string","pattern":"^\\d{1,3}\\.\\d{2}%$"}},"required":["protocol","count","accuracy","empty_prediction_count","display_percent"],"additionalProperties":false},"MultilokoNativeMetrics":{"type":"object","properties":{"em":{"type":"number","minimum":0,"maximum":1},"f1":{"type":"number","minimum":0,"maximum":1},"bleu":{"type":"number","minimum":0,"maximum":100.000000001},"chrf":{"type":"number","minimum":0,"maximum":100.000000001},"edit_distance":{"type":"number","minimum":0,"maximum":524288},"edit_similarity":{"type":"number","minimum":0,"maximum":1},"contains":{"type":"number","minimum":0,"maximum":1}},"required":["em","f1","bleu","chrf","edit_distance","edit_similarity","contains"],"additionalProperties":false},"MultilokoRowMetrics":{"type":"object","properties":{"em":{"type":"number","minimum":0,"maximum":1,"enum":[0,1]},"f1":{"type":"number","minimum":0,"maximum":1},"bleu":{"type":"number","minimum":0,"maximum":100.000000001},"chrf":{"type":"number","minimum":0,"maximum":100.000000001},"edit_distance":{"type":"number","minimum":0,"maximum":524288},"edit_similarity":{"type":"number","minimum":0,"maximum":1},"contains":{"type":"number","minimum":0,"maximum":1,"enum":[0,1]}},"required":["em","f1","bleu","chrf","edit_distance","edit_similarity","contains"],"additionalProperties":false},"MultilokoGroupMetrics":{"type":"object","properties":{"average_em":{"type":"number","minimum":0,"maximum":1},"max_em":{"type":"number","minimum":0,"maximum":1},"min_em":{"type":"number","minimum":0,"maximum":1},"gap_em":{"type":"number","minimum":0,"maximum":1},"average_f1":{"type":"number","minimum":0,"maximum":1},"max_f1":{"type":"number","minimum":0,"maximum":1},"min_f1":{"type":"number","minimum":0,"maximum":1},"gap_f1":{"type":"number","minimum":0,"maximum":1},"average_bleu":{"type":"number","minimum":0,"maximum":100.000000001},"max_bleu":{"type":"number","minimum":0,"maximum":100.000000001},"min_bleu":{"type":"number","minimum":0,"maximum":100.000000001},"gap_bleu":{"type":"number","minimum":0,"maximum":100.000000001},"average_chrf":{"type":"number","minimum":0,"maximum":100.000000001},"max_chrf":{"type":"number","minimum":0,"maximum":100.000000001},"min_chrf":{"type":"number","minimum":0,"maximum":100.000000001},"gap_chrf":{"type":"number","minimum":0,"maximum":100.000000001},"average_edit_distance":{"type":"number","minimum":0,"maximum":524288},"max_edit_distance":{"type":"number","minimum":0,"maximum":524288},"min_edit_distance":{"type":"number","minimum":0,"maximum":524288},"gap_edit_distance":{"type":"number","minimum":0,"maximum":524288},"average_edit_similarity":{"type":"number","minimum":0,"maximum":1},"max_edit_similarity":{"type":"number","minimum":0,"maximum":1},"min_edit_similarity":{"type":"number","minimum":0,"maximum":1},"gap_edit_similarity":{"type":"number","minimum":0,"maximum":1},"average_contains":{"type":"number","minimum":0,"maximum":1},"max_contains":{"type":"number","minimum":0,"maximum":1},"min_contains":{"type":"number","minimum":0,"maximum":1},"gap_contains":{"type":"number","minimum":0,"maximum":1}},"required":["average_em","max_em","min_em","gap_em","average_f1","max_f1","min_f1","gap_f1","average_bleu","max_bleu","min_bleu","gap_bleu","average_chrf","max_chrf","min_chrf","gap_chrf","average_edit_distance","max_edit_distance","min_edit_distance","gap_edit_distance","average_edit_similarity","max_edit_similarity","min_edit_similarity","gap_edit_similarity","average_contains","max_contains","min_contains","gap_contains"],"additionalProperties":false},"MultilokoRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["multiloko-dev-chat-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":7750},"metrics":{"type":"object","properties":{"group_metrics":{"$ref":"#/components/schemas/MultilokoGroupMetrics"},"languages":{"type":"object","properties":{"arabic":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"bengali":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"cantonese":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"czech":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"dutch":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"english":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"french":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"hebrew":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"hindi":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"italian":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"indonesian":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"farsi":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"german":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"japanese":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"khmer":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"korean":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"malay":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"marathi":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"polish":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"portuguese":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"romanian":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"russian":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"simplified_mandarin":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"spanish":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"swedish":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"tagalog":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"thai":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"traditional_mandarin":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"turkish":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"vietnamese":{"$ref":"#/components/schemas/MultilokoNativeMetrics"},"urdu":{"$ref":"#/components/schemas/MultilokoNativeMetrics"}},"required":[],"additionalProperties":false,"minProperties":1,"maxProperties":31}},"required":["group_metrics","languages"],"additionalProperties":false}},"required":["protocol","count","metrics"],"additionalProperties":false},"Hrm8kRowMetrics":{"type":"object","properties":{"accuracy":{"type":"number","enum":[0,1]},"parse_failure_rate":{"type":"number","enum":[0,1]},"verify_failure_rate":{"type":"number","enum":[0,1]},"language_penalizer_average":{"type":"number","enum":[0,1]}},"required":["accuracy","parse_failure_rate","verify_failure_rate","language_penalizer_average"],"additionalProperties":false},"Hrm8kNativeMetrics":{"type":"object","properties":{"accuracy":{"type":"number","minimum":0,"maximum":1},"parse_failure_rate":{"type":"number","minimum":0,"maximum":1},"verify_failure_rate":{"type":"number","minimum":0,"maximum":1},"language_penalizer_average":{"type":"number","minimum":0,"maximum":1}},"required":["accuracy","parse_failure_rate","verify_failure_rate","language_penalizer_average"],"additionalProperties":false},"Hrm8kRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["hrm8k-hret-math-match-mixed-author-v1"]},"count":{"type":"integer","minimum":1,"maximum":8011},"metrics":{"$ref":"#/components/schemas/Hrm8kNativeMetrics"}},"required":["protocol","count","metrics"],"additionalProperties":false},"IfbenchRowMetrics":{"type":"object","properties":{"id":{"type":"string","pattern":"^ifbench:\\d+$"},"strict":{"type":"array","items":{"type":"boolean"},"minItems":1,"maxItems":2},"loose":{"type":"array","items":{"type":"boolean"},"minItems":1,"maxItems":2}},"required":["id","strict","loose"],"additionalProperties":false},"IfbenchReport":{"type":"object","properties":{"prompt_accuracy":{"type":"number","minimum":0,"maximum":1},"instruction_accuracy":{"type":"number","minimum":0,"maximum":1},"tier0":{"type":"object","additionalProperties":{"type":"number","minimum":0,"maximum":1}},"tier1":{"type":"object","additionalProperties":{"type":"number","minimum":0,"maximum":1}}},"required":["prompt_accuracy","instruction_accuracy","tier0","tier1"],"additionalProperties":false},"IfbenchRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["ifbench-author-driver-v1"]},"count":{"type":"integer","minimum":1,"maximum":300},"accuracy":{"type":"number","minimum":0,"maximum":1},"strict":{"$ref":"#/components/schemas/IfbenchReport"},"loose":{"$ref":"#/components/schemas/IfbenchReport"}},"required":["protocol","count","accuracy","strict","loose"],"additionalProperties":false},"GlobalMmluRunMetrics":{"type":"object","properties":{"protocol":{"type":"string","enum":["global-mmlu-lite-author-likelihood-v1","global-mmlu-full-author-likelihood-v1","global-mmlu-en-author-likelihood-v1","global-mmlu-te-author-likelihood-v1"]},"count":{"type":"integer","minimum":1,"maximum":589764},"author_metrics":{"type":"array","items":{"type":"object","properties":{"task":{"type":"string"},"language":{"type":"string"},"accuracy":{"type":"number","minimum":0,"maximum":1},"count":{"type":"integer","minimum":1,"maximum":14042}},"required":["task","language","accuracy","count"],"additionalProperties":false},"minItems":1,"maxItems":42}},"required":["protocol","count","author_metrics"],"additionalProperties":false},"AuthorRowMetrics":{"type":"object","properties":{"question_id":{"type":"string","pattern":"^qst_\\d{4}$"},"question_type":{"type":"string"},"corrected":{"type":"boolean"},"answer_correct":{"type":"boolean"},"completeness_pct":{"type":"number","minimum":0,"maximum":100},"document_recall_pct":{"type":["number","null"],"minimum":0,"maximum":100},"invalid_extra_docs":{"type":["integer","null"],"minimum":0,"maximum":10}},"required":["question_id","question_type","corrected","answer_correct","completeness_pct","document_recall_pct","invalid_extra_docs"],"additionalProperties":false},"AuthorGroupMetrics":{"type":"object","properties":{"count":{"type":"integer","minimum":0},"average_correctness_pct":{"type":"number"},"average_completeness_pct":{"type":"number"},"combined_correctness_completeness_score":{"type":"number"},"average_recall_pct":{"type":"number"},"average_invalid_extra_docs":{"type":"number"}},"required":["count","average_correctness_pct","average_completeness_pct","combined_correctness_completeness_score","average_recall_pct","average_invalid_extra_docs"],"additionalProperties":false},"AuthorRunMetrics":{"type":"object","properties":{"aggregate_stats":{"type":"object","properties":{"total_questions":{"type":"integer","minimum":0},"completed_questions":{"type":"integer","minimum":0},"skipped_rows":{"type":"integer","minimum":0},"num_corrected_questions":{"type":"integer","minimum":0},"average_correctness_pct":{"type":"number"},"average_completeness_pct":{"type":"number"},"combined_correctness_completeness_score":{"type":"number"},"average_recall_pct":{"type":"number"},"average_invalid_extra_docs":{"type":"number"}},"required":["total_questions","completed_questions","skipped_rows","num_corrected_questions","average_correctness_pct","average_completeness_pct","combined_correctness_completeness_score","average_recall_pct","average_invalid_extra_docs"],"additionalProperties":false},"question_type_stats":{"type":"object","additionalProperties":{"$ref":"#/components/schemas/AuthorGroupMetrics"}}},"required":["aggregate_stats","question_type_stats"],"additionalProperties":false},"BenchmarkProvenanceSummary":{"type":"object","properties":{"source_url":{"type":"string","format":"uri","maxLength":2048},"license":{"type":"string","minLength":1,"maxLength":200},"notice_sha256":{"type":"string","pattern":"^[a-f0-9]{64}$"}},"required":["source_url","license","notice_sha256"],"additionalProperties":false},"NoticeDocument":{"type":"object","properties":{"filename":{"type":"string","minLength":1,"maxLength":80},"source_url":{"type":["string","null"],"format":"uri","maxLength":2048},"sha256":{"type":"string","pattern":"^[a-f0-9]{64}$"},"text":{"type":"string","maxLength":120000}},"required":["filename","source_url","sha256","text"],"additionalProperties":false},"SourceCardExcerpt":{"type":"object","properties":{"start_byte":{"type":"integer","minimum":0},"end_byte":{"type":"integer","minimum":1},"sha256":{"type":"string","pattern":"^[a-f0-9]{64}$"},"text":{"type":"string","maxLength":120000}},"required":["start_byte","end_byte","sha256","text"],"additionalProperties":false},"SourceCardNotice":{"type":"object","properties":{"filename":{"type":"string","const":"SOURCE-CARD.md"},"source_url":{"type":"string","format":"uri","maxLength":2048},"sha256":{"type":"string","pattern":"^[a-f0-9]{64}$","description":"SHA256 of the complete captured source card, including sections omitted from this export."},"git_blob":{"type":"string","pattern":"^[a-f0-9]{40}$"},"format":{"type":"string","const":"exact-byte-excerpts"},"omitted_sections":{"type":"array","items":{"type":"string","maxLength":500},"minItems":1,"maxItems":10},"excerpts":{"type":"array","items":{"$ref":"#/components/schemas/SourceCardExcerpt"},"minItems":1,"maxItems":10}},"required":["filename","source_url","sha256","git_blob","format","omitted_sections","excerpts"],"additionalProperties":false},"BenchmarkProvenance":{"type":"object","properties":{"schema_version":{"type":"integer","const":1},"benchmark_id":{"type":"string","minLength":1,"maxLength":100},"benchmark_version":{"type":"string","minLength":1,"maxLength":100},"source_revision":{"type":"string","minLength":1,"maxLength":200},"source_url":{"type":"string","format":"uri","maxLength":2048},"source_dataset":{"type":["string","null"],"maxLength":200},"license":{"type":"string","minLength":1,"maxLength":200},"item_count":{"type":"integer","minimum":1,"maximum":589764},"source_item_count":{"type":"integer","minimum":1,"maximum":589764},"items_sha256":{"type":["string","null"],"pattern":"^[a-f0-9]{64}$"},"manifest_sha256":{"type":["string","null"],"pattern":"^[a-f0-9]{64}$"},"protocol":{"type":"string","minLength":1,"maxLength":200},"notice_sha256":{"type":"string","pattern":"^[a-f0-9]{64}$","description":"SHA256 of this object without notice_sha256, serialized with recursively sorted object keys and unchanged array order."},"documents":{"type":"object","properties":{"license":{"anyOf":[{"$ref":"#/components/schemas/NoticeDocument"},{"type":"null"}]},"conversion_notice":{"$ref":"#/components/schemas/NoticeDocument"},"source_card":{"anyOf":[{"$ref":"#/components/schemas/SourceCardNotice"},{"type":"null"}]}},"required":["license","conversion_notice","source_card"],"additionalProperties":false},"changes":{"type":"array","items":{"type":"string","maxLength":2000},"minItems":1,"maxItems":30}},"required":["schema_version","benchmark_id","benchmark_version","source_revision","source_url","source_dataset","license","item_count","source_item_count","items_sha256","manifest_sha256","protocol","notice_sha256","documents","changes"],"additionalProperties":false},"EndpointConnectionCheck":{"type":"object","properties":{"status":{"type":"string","enum":["unverified","queued","running","passed","failed","canceled"]},"evaluation_id":{"type":["string","null"]},"checked_at":{"type":["string","null"],"format":"date-time","description":"Completion timestamp of the check; null before a terminal state or when unverified."}},"required":["status","evaluation_id","checked_at"],"additionalProperties":false},"BenchmarkFewshotConfig":{"type":"object","properties":{"count":{"type":["integer","null"],"minimum":0,"description":"Reviewed number of prompt examples, or null when not established."},"seed":{"type":["integer","null"],"description":"Seed for selecting prompt examples, distinct from evaluation sampling."},"sampler":{"type":["string","null"],"description":"Reviewed prompt-example selection method, or null when unspecified."}},"required":["count","seed","sampler"],"additionalProperties":false},"BenchmarkGenerationConfig":{"type":"object","properties":{"temperature":{"type":["number","null"]},"max_output_tokens":{"type":["integer","null"],"minimum":0},"sampling":{"type":["string","null"]},"output_budget":{"type":["string","null"]},"reasoning_policy":{"type":["string","null"]},"target_output_mode":{"type":["string","null"]},"reasoning":{"type":["string","null"]}},"required":[],"additionalProperties":false},"BenchmarkPresentation":{"type":"object","properties":{"revision":{"type":"string","description":"Descriptive metadata revision; independent of the execution version."},"name":{"type":"string"},"description":{"type":"string"},"count_unit":{"type":"string"},"split":{"type":["string","null"]},"exemplars":{"type":["string","null"],"description":"Description of prompt examples; never their contents or answers."},"maintainer":{"type":"object","properties":{"id":{"type":"string","enum":["allevals"]},"name":{"type":"string","enum":["AllEvals"]}},"required":["id","name"],"additionalProperties":false}},"required":["revision","name","description","count_unit","split","exemplars","maintainer"],"additionalProperties":false},"BenchmarkAvailabilityDetails":{"type":"object","properties":{"status":{"type":"string","enum":["verified_installation"]},"execution_check_required":{"type":"boolean","const":true},"message":{"type":"string"}},"required":["status","execution_check_required","message"],"additionalProperties":false},"Benchmark":{"type":"object","properties":{"id":{"type":"string"},"slug":{"type":"string","description":"The stable AllEvals identifier, equal to id. Distinct protocols retain distinct IDs."},"name":{"type":"string"},"description":{"type":"string"},"category":{"type":"string"},"tags":{"type":"array","items":{"type":"string"}},"item_count":{"type":"integer","minimum":0},"version":{"type":"string"},"protocol":{"type":"string"},"scoring":{"type":"string","enum":["choice","exact","numeric","llm-judge"]},"availability":{"type":"string","enum":["ready","requires_package","restricted","unsupported"]},"license":{"type":"string"},"source_url":{"type":"string","format":"uri"},"caveat":{"type":"string"},"is_test":{"type":"boolean"},"endpoint_types":{"type":"array","items":{"type":"string","enum":["chat_completions","decisions","completions","responses","loglikelihood","native_predictions","generate_content"]}},"runner":{"type":["string","null"],"description":"Reviewed evaluator family; null when not established."},"manifest_sha256":{"type":"string","pattern":"^[a-f0-9]{64}$","description":"SHA-256 of the reviewed canonical manifest JSON, matching the installation receipt. This is not the hash of arbitrary JSON formatting or of descriptive metadata."},"source_revision":{"type":"string","description":"Exact source_revision from the admitted immutable manifest."},"data_license":{"type":["string","null"],"description":"Dataset license when separately established by reviewed evidence. Null does not waive the existing combined license terms."},"code_license":{"type":["string","null"],"description":"Evaluator code license when separately established by reviewed evidence."},"license_url":{"type":["string","null"],"format":"uri","description":"Source URL of the reviewed license document, or null when unavailable."},"display_policy":{"type":"string","enum":["metadata_only"],"description":"Catalog responses expose descriptive metadata only, never benchmark sample content, prompts, answers, customer outputs or traces."},"required_capabilities":{"type":["array","null"],"items":{"type":"string"},"description":"Reviewed model/adapter capabilities beyond the endpoint wire format; null when not established. This does not prove compatibility with a specific endpoint."},"metrics":{"type":["array","null"],"items":{"type":"string"},"description":"Precise metric names for this evaluator edition; null when not established. The existing scoring field remains the coarse scoring category."},"fewshot_config":{"anyOf":[{"$ref":"#/components/schemas/BenchmarkFewshotConfig"},{"type":"null"}]},"generation_config":{"anyOf":[{"$ref":"#/components/schemas/BenchmarkGenerationConfig"},{"type":"null"}],"description":"Reviewed generation recipe, not defaults for a new evaluation request. Missing or null values mean unknown or unspecified, never an implicit zero. The original evaluator owns its execution settings."},"sampling_algorithm":{"type":"string","description":"mulberry32-partial-fisher-yates-v1 selects evaluation items without replacement. source-order-all-or-mulberry32-partial-fisher-yates-v1 preserves source order for a complete global-mmlu-full-author run and uses the same seeded selection for a partial run. This is distinct from few-shot prompt-example sampling."},"presentation":{"$ref":"#/components/schemas/BenchmarkPresentation"},"availability_details":{"$ref":"#/components/schemas/BenchmarkAvailabilityDetails"}},"required":["id","slug","name","description","category","tags","item_count","version","protocol","scoring","availability","license","source_url","caveat","is_test","endpoint_types","runner","manifest_sha256","source_revision","data_license","code_license","license_url","display_policy","required_capabilities","metrics","fewshot_config","generation_config","sampling_algorithm","presentation","availability_details"],"additionalProperties":false},"Endpoint":{"type":"object","properties":{"provider":{"type":"string","enum":["custom","openrouter","cloudflare"]},"billing_mode":{"type":"string","enum":["custom","openrouter_byok","openrouter_managed"]},"id":{"type":"string"},"slug":{"type":"string"},"name":{"type":"string"},"type":{"type":"string","enum":["chat_completions","decisions","completions","responses","loglikelihood","native_predictions","generate_content"]},"url":{"type":"string","format":"uri"},"model":{"type":"string"},"auth_type":{"type":"string","enum":["bearer","header","none"]},"auth_header":{"type":["string","null"]},"has_api_key":{"type":"boolean"},"header_names":{"type":"array","items":{"type":"string"}},"connection_check":{"$ref":"#/components/schemas/EndpointConnectionCheck"},"revision":{"type":"integer","minimum":1},"created_at":{"type":"string","format":"date-time"},"updated_at":{"type":"string","format":"date-time"},"is_test":{"type":"boolean"}},"required":["provider","billing_mode","id","slug","name","type","url","model","auth_type","auth_header","has_api_key","header_names","connection_check","revision","created_at","updated_at","is_test"],"additionalProperties":false},"CreateEndpoint":{"type":"object","properties":{"slug":{"type":"string","pattern":"^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$","maxLength":64,"minLength":2},"type":{"type":"string","enum":["chat_completions","decisions","completions","responses","loglikelihood","native_predictions","generate_content"]},"provider":{"type":"string","enum":["custom","openrouter","cloudflare"]},"billing_mode":{"type":"string","enum":["custom","openrouter_byok"]},"name":{"type":"string","minLength":1,"maxLength":80},"url":{"type":"string","minLength":1,"maxLength":2048,"format":"uri","description":"Full public HTTPS POST URL on port 443. No query, fragment, credentials, IP literal, private hostname, or redirect. For generate_content, use the exact /v1beta/models/MODEL:generateContent path matching the saved model and custom billing. This is not a base URL."},"model":{"type":"string","maxLength":200,"default":""},"auth_type":{"type":"string","enum":["bearer","header","none"]},"auth_header":{"type":"string","minLength":1,"maxLength":80,"description":"Required for header authentication. Omit for bearer/none. Reserved transport headers are forbidden."},"api_key":{"type":"string","maxLength":4096,"writeOnly":true,"description":"Endpoint-provider credential, not an allevals key. Never returned. Nonempty for bearer/header; omit or empty for none."},"headers":{"type":"object","maxProperties":16,"additionalProperties":{"type":"string","maxLength":4096},"writeOnly":true,"description":"All values are encrypted and write-only. Case-insensitive names must be unique and cannot override authentication or reserved headers. Combined names/values limited to 8,192 characters; controls forbidden. The exact Cloudflare Access pair CF-Access-Client-Id and CF-Access-Client-Secret is allowed only together with nonempty values; all other cf-* headers remain forbidden.","default":{}}},"required":["slug","name","type","url"],"additionalProperties":false,"examples":[{"slug":"company-model-v2","name":"Company model v2","type":"chat_completions","url":"https://model.example.org/v1/chat/completions","model":"company-v2","auth_type":"none"},{"slug":"openrouter-model","name":"My OpenRouter model","provider":"openrouter","billing_mode":"openrouter_byok","type":"chat_completions","url":"https://openrouter.ai/api/v1/chat/completions","model":"provider/model","api_key":"CUSTOMER_OPENROUTER_KEY"},{"slug":"cloudflare-route","name":"My Cloudflare route","provider":"cloudflare","billing_mode":"custom","type":"chat_completions","url":"https://gateway.ai.cloudflare.com/v1/0123456789abcdef0123456789abcdef/my-gateway/compat/chat/completions","model":"dynamic/my-route","api_key":"CUSTOMER_CLOUDFLARE_TOKEN"}],"description":"provider and billing_mode are immutable. OpenRouter uses the canonical https://openrouter.ai/api/v1/chat/completions URL, chat_completions type, billing_mode openrouter_byok and the customer API key. Cloudflare uses https://gateway.ai.cloudflare.com/v1/<account_id>/<gateway_id>/compat/chat/completions, chat_completions and billing_mode custom; the encrypted customer token is sent as Bearer in cf-aig-authorization. Omitted Cloudflare authentication fields normalize to header/cf-aig-authorization. Router providers accept no custom headers and never use an AllEvals-funded provider key. Custom retains the existing protocol/authentication options. Retired openrouter_managed requests return 410 managed_provider_removed. Billing readiness is checked before evaluation admission."},"UpdateEndpoint":{"type":"object","properties":{"name":{"type":"string","minLength":1,"maxLength":80},"url":{"type":"string","minLength":1,"maxLength":2048,"format":"uri","description":"Full public HTTPS POST URL on port 443. No query, fragment, credentials, IP literal, private hostname, or redirect. For generate_content, use the exact /v1beta/models/MODEL:generateContent path matching the saved model and custom billing. This is not a base URL."},"model":{"type":"string","maxLength":200},"auth_type":{"type":"string","enum":["bearer","header","none"]},"auth_header":{"type":"string","minLength":1,"maxLength":80,"description":"Required for header authentication. Omit for bearer/none. Reserved transport headers are forbidden."},"api_key":{"type":"string","maxLength":4096,"writeOnly":true,"description":"Endpoint-provider credential, not an allevals key. Never returned. Nonempty for bearer/header; omit or empty for none."},"headers":{"type":"object","maxProperties":16,"additionalProperties":{"type":"string","maxLength":4096},"writeOnly":true,"description":"All values are encrypted and write-only. Case-insensitive names must be unique and cannot override authentication or reserved headers. Combined names/values limited to 8,192 characters; controls forbidden. The exact Cloudflare Access pair CF-Access-Client-Id and CF-Access-Client-Secret is allowed only together with nonempty values; all other cf-* headers remain forbidden."}},"required":[],"additionalProperties":false,"minProperties":1,"description":"Sparse update; no defaults apply. Existing credentials cannot be read back. Provider and billing mode cannot be changed. Authentication consistency also depends on the saved provider; Cloudflare uses its fixed cf-aig-authorization header."},"CreateApiKey":{"type":"object","properties":{"name":{"type":"string","minLength":1,"maxLength":80},"scopes":{"type":"array","items":{"type":"string","enum":["evaluations:read","evaluations:write","benchmarks:read","connections:read"]},"minItems":1,"maxItems":4,"uniqueItems":true},"expires_in_days":{"type":"integer","minimum":1,"maximum":365}},"required":["name","scopes"],"additionalProperties":false,"examples":[{"name":"Release CI","scopes":["evaluations:read","evaluations:write"],"expires_in_days":90}]},"ApiKey":{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"prefix":{"type":"string"},"last_four":{"type":"string"},"scopes":{"type":"array","items":{"type":"string","enum":["evaluations:read","evaluations:write","benchmarks:read","connections:read"]},"minItems":1,"maxItems":4,"uniqueItems":true},"created_at":{"type":"string","format":"date-time"},"expires_at":{"type":["string","null"],"format":"date-time"},"last_used_at":{"type":["string","null"],"format":"date-time"},"revoked_at":{"type":["string","null"],"format":"date-time"}},"required":["id","name","prefix","last_four","scopes","created_at","expires_at","last_used_at","revoked_at"],"additionalProperties":false},"CreatedApiKey":{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"prefix":{"type":"string"},"last_four":{"type":"string"},"scopes":{"type":"array","items":{"type":"string","enum":["evaluations:read","evaluations:write","benchmarks:read","connections:read"]},"minItems":1,"maxItems":4,"uniqueItems":true},"created_at":{"type":"string","format":"date-time"},"expires_at":{"type":["string","null"],"format":"date-time"},"last_used_at":{"type":["string","null"],"format":"date-time"},"revoked_at":{"type":["string","null"],"format":"date-time"},"token":{"type":"string","pattern":"^ae_(live|test)_[A-Za-z0-9_-]{43}$","description":"Shown only on creation; copy to a server-side secret manager now."}},"required":["id","name","prefix","last_four","scopes","created_at","expires_at","last_used_at","revoked_at","token"],"additionalProperties":false},"StartSignIn":{"type":"object","properties":{"email":{"type":"string","format":"email","maxLength":254},"name":{"type":"string","maxLength":80},"next":{"type":"string","maxLength":2048,"description":"Optional internal /app or /app/* destination after sign-in. External, malformed, or non-dashboard values fall back to /app; fragments are discarded. The browser validates it again."}},"required":["email"],"additionalProperties":false},"SignInRequested":{"type":"object","properties":{"ok":{"type":"boolean","const":true},"development_link":{"type":"string","format":"uri","description":"Only explicit loopback development. Contains a disposable local verification token; never available in production."}},"required":["ok"],"additionalProperties":false},"VerifySignIn":{"type":"object","properties":{"token":{"type":"string","pattern":"^[A-Za-z0-9_-]{43}$","writeOnly":true}},"required":["token"],"additionalProperties":false},"User":{"type":"object","properties":{"id":{"type":"string"},"email":{"type":"string","format":"email"},"name":{"type":"string"},"created_at":{"type":"string","format":"date-time"}},"required":["id","email","name","created_at"],"additionalProperties":false},"Workspace":{"type":"object","properties":{"id":{"type":"string"},"name":{"type":"string"},"created_at":{"type":"string","format":"date-time"},"plan":{"type":"string","enum":["free","pro"]}},"required":["id","name","created_at","plan"],"additionalProperties":false},"Session":{"type":"object","properties":{"user":{"$ref":"#/components/schemas/User"},"workspace":{"$ref":"#/components/schemas/Workspace"},"environment":{"type":"string"},"capabilities":{"type":"object","properties":{"email":{"type":"boolean"},"billing":{"type":"boolean"},"local_auth":{"type":"boolean"}},"required":["email","billing","local_auth"],"additionalProperties":false}},"required":["user","workspace","environment","capabilities"],"additionalProperties":false},"BrowserSession":{"type":"object","properties":{"id":{"type":"string"},"created_at":{"type":"string","format":"date-time"},"expires_at":{"type":"string","format":"date-time"},"last_seen_at":{"type":"string","format":"date-time"},"user_agent":{"type":["string","null"]},"current":{"type":"boolean"}},"required":["id","created_at","expires_at","last_seen_at","user_agent","current"],"additionalProperties":false},"UpdateAccount":{"type":"object","properties":{"name":{"type":"string","minLength":1,"maxLength":80},"workspace_name":{"type":"string","minLength":1,"maxLength":80}},"required":["name","workspace_name"],"additionalProperties":false},"UsageSummary":{"type":"object","properties":{"evaluations":{"type":"integer","minimum":0},"items":{"type":"integer","minimum":0},"reserved_items":{"type":"integer","minimum":0},"attempted_items":{"type":"integer","minimum":0},"input_tokens":{"type":"integer","minimum":0},"output_tokens":{"type":"integer","minimum":0},"api_requests":{"type":"integer","minimum":0},"period_start":{"type":"string","format":"date-time"},"daily":{"type":"array","items":{"type":"object","properties":{"date":{"type":"string","format":"date"},"evaluations":{"type":"integer","minimum":0},"items":{"type":"integer","minimum":0}},"required":["date","evaluations","items"],"additionalProperties":false}}},"required":["evaluations","items","reserved_items","attempted_items","input_tokens","output_tokens","api_requests","period_start","daily"],"additionalProperties":false},"BillingSummary":{"type":"object","properties":{"wallet":{"$ref":"#/components/schemas/WalletSummary"},"wallet_funding":{"$ref":"#/components/schemas/WalletFundingSummary"},"inference":{"$ref":"#/components/schemas/InferenceBillingSummary"},"row_billing":{"$ref":"#/components/schemas/RowBillingSummary"},"plan":{"type":"string","enum":["free","pro"]},"status":{"type":"string"},"configured":{"type":"boolean"},"mode":{"type":"string","enum":["live","test","unconfigured"]},"current_period_end":{"type":["string","null"],"format":"date-time"},"cancel_at_period_end":{"type":"boolean"},"legacy_access":{"type":"boolean"},"legacy_access_expires_at":{"type":["string","null"],"format":"date-time"},"limits":{"type":"object","properties":{"evaluations_per_month":{"type":"integer","minimum":0},"items_per_evaluation":{"type":"integer","minimum":0},"reserved_items_per_month":{"type":"integer","minimum":0},"full_benchmark_items":{"type":"integer","minimum":0}},"required":["evaluations_per_month","items_per_evaluation","reserved_items_per_month","full_benchmark_items"],"additionalProperties":false}},"required":["wallet","wallet_funding","inference","row_billing","plan","status","configured","mode","current_period_end","cancel_at_period_end","legacy_access","legacy_access_expires_at","limits"],"additionalProperties":false},"InferenceBillingSummary":{"type":"object","properties":{"managed_configured":{"type":"boolean","const":false,"deprecated":true},"managed_available":{"type":"boolean","const":false,"deprecated":true},"markup_bps":{"type":"integer","const":0,"deprecated":true},"budget_usd":{"type":"string"},"credit_limit_usd":{"type":"string"},"spent_usd":{"type":"string"},"reserved_usd":{"type":"string"},"pending_cost_items":{"type":"integer","minimum":0},"reconciliation_required":{"type":"boolean"},"available_models":{"type":"array","items":{"type":"object","properties":{"model":{"type":"string"},"max_output_tokens":{"type":"integer","minimum":0},"per_request_reservation_usd":{"type":"string"}},"required":["model","max_output_tokens"],"additionalProperties":false}}},"required":["managed_configured","managed_available","markup_bps","budget_usd","credit_limit_usd","spent_usd","reserved_usd","pending_cost_items","reconciliation_required","available_models"],"additionalProperties":false},"RowBillingSummary":{"type":"object","properties":{"configured":{"type":"boolean"},"available":{"type":"boolean"},"unit":{"type":"string","const":"successfully_evaluated_row"},"unit_price_usd":{"type":["string","null"]},"reconciliation_required":{"type":"boolean"}},"required":["configured","available","unit","unit_price_usd","reconciliation_required"],"additionalProperties":false},"InferenceCost":{"type":"object","properties":{"collection_mode":{"type":"string","enum":["prepaid","legacy_postpaid","fixture","owner_verification"]},"wallet_status":{"type":"string","enum":["not_applicable","reserved","debited","reconciliation_required"]},"reserved_usd":{"type":"string"},"settled_usd":{"type":"string"},"mode":{"type":"string","enum":["custom","openrouter_byok","openrouter_managed"]},"currency":{"type":"string","const":"usd"},"markup_bps":{"type":"integer","minimum":0},"status":{"type":"string","enum":["not_billed","pending","known","reconciliation_required"]},"provider_cost_usd":{"type":["string","null"]},"platform_fee_usd":{"type":["string","null"]},"total_usd":{"type":["string","null"]},"pending_items":{"type":"integer","minimum":0},"billable_rows":{"type":"integer","minimum":0},"meter_status":{"type":"string","enum":["not_applicable","pending","accepted","reconciliation_required"]}},"required":["mode","currency","markup_bps","status","provider_cost_usd","platform_fee_usd","total_usd","pending_items","meter_status"],"additionalProperties":false},"WalletSummary":{"type":"object","properties":{"collection_mode":{"type":"string","const":"prepaid"},"currency":{"type":"string","const":"usd"},"configured":{"type":"boolean"},"balance_usd":{"type":"string"},"reserved_usd":{"type":"string"},"available_usd":{"type":"string"},"spent_usd":{"type":"string"},"blocked":{"type":"boolean"}},"required":["collection_mode","currency","configured","balance_usd","reserved_usd","available_usd","spent_usd","blocked"],"additionalProperties":false},"WalletFundingSummary":{"type":"object","properties":{"configured":{"type":"boolean"},"card":{"anyOf":[{"type":"object","properties":{"brand":{"type":"string"},"last4":{"type":"string","pattern":"^[0-9]{4}$"}},"required":["brand","last4"],"additionalProperties":false},{"type":"null"}]},"auto_reload":{"type":"object","properties":{"enabled":{"type":"boolean"},"threshold_cents":{"type":"integer","minimum":0},"amount_cents":{"type":"integer","minimum":0},"daily_cap_cents":{"type":"integer","minimum":0},"revision":{"type":"integer","minimum":0},"last_error":{"type":["string","null"]}},"required":["enabled","threshold_cents","amount_cents","daily_cap_cents","revision","last_error"],"additionalProperties":false},"pending_topup":{"type":"boolean"}},"required":["configured","card","auto_reload","pending_topup"],"additionalProperties":false},"WalletTopupList":{"type":"object","properties":{"data":{"type":"array","items":{"type":"object","properties":{"id":{"type":"string"},"kind":{"type":"string","enum":["manual","auto","card"]},"amount_cents":{"type":"integer","minimum":0},"status":{"type":"string","enum":["created","pending","succeeded","failed","requires_action","review"]},"created_at":{"type":"string","format":"date-time"}},"required":["id","kind","amount_cents","status","created_at"],"additionalProperties":false}},"has_more":{"type":"boolean"}},"required":["data","has_more"],"additionalProperties":false},"WalletFundingRedirect":{"type":"object","properties":{"id":{"type":"string"},"status":{"type":"string","enum":["created","pending","succeeded","failed","requires_action","review"]},"url":{"anyOf":[{"type":"string","format":"uri"},{"type":"null"}]}},"required":["id","status","url"],"additionalProperties":false},"CreateWalletTopup":{"type":"object","properties":{"amount_cents":{"type":"integer","minimum":500,"maximum":100000},"save_card_consent":{"type":"boolean","const":true}},"required":["amount_cents","save_card_consent"],"additionalProperties":false},"UpdateWalletAutoReload":{"type":"object","properties":{"enabled":{"type":"boolean"},"threshold_cents":{"type":"integer","minimum":0,"maximum":100000},"amount_cents":{"type":"integer","minimum":500,"maximum":100000},"daily_cap_cents":{"type":"integer","minimum":500,"maximum":100000},"expected_revision":{"type":"integer","minimum":0},"consent":{"type":"boolean","const":true}},"required":["enabled","threshold_cents","amount_cents","daily_cap_cents","expected_revision"],"additionalProperties":false},"InferenceBudget":{"type":"object","properties":{"budget_usd":{"type":"string","maxLength":40},"consent":{"type":"boolean","const":true}},"required":["budget_usd","consent"],"additionalProperties":false},"UsageConsent":{"type":"object","properties":{"consent":{"type":"boolean","const":true}},"required":["consent"],"additionalProperties":false},"BillingRedirect":{"type":"object","properties":{"url":{"type":"string","format":"uri"},"mode":{"type":"string","enum":["live","test"]}},"required":["url","mode"],"additionalProperties":false},"ClaimLegacyPurchase":{"type":"object","properties":{"checkout_session_id":{"type":"string","pattern":"^cs_(?:test_|live_)?[A-Za-z0-9]{8,255}$","writeOnly":true}},"required":["checkout_session_id"],"additionalProperties":false},"LegacyPurchaseClaim":{"type":"object","properties":{"ok":{"type":"boolean","const":true},"scope":{"type":"string","const":"benchmark_imports"},"expires_at":{"type":"string","format":"date-time"}},"required":["ok","scope","expires_at"],"additionalProperties":false},"StripeEvent":{"type":"object","required":["id","object","type","created","livemode","data"],"properties":{"id":{"type":"string"},"object":{"type":"string","const":"event"},"type":{"type":"string"},"created":{"type":"integer","minimum":0},"livemode":{"type":"boolean"},"data":{"type":"object","required":["object"],"properties":{"object":{"type":"object"}}}},"additionalProperties":true,"description":"Stripe event envelope; event-specific objects follow Stripe schemas. Signature verification uses raw bytes, not this JSON Schema."},"WebhookReceipt":{"type":"object","properties":{"received":{"type":"boolean","const":true}},"required":["received"],"additionalProperties":false},"BenchmarkList":{"type":"object","properties":{"data":{"type":"array","items":{"$ref":"#/components/schemas/Benchmark"}},"has_more":{"type":"boolean"},"next_cursor":{"type":["string","null"]}},"required":["data","has_more","next_cursor"],"additionalProperties":false},"EvaluationList":{"type":"object","properties":{"data":{"type":"array","items":{"$ref":"#/components/schemas/Evaluation"}},"has_more":{"type":"boolean"},"next_cursor":{"type":["string","null"]}},"required":["data","has_more","next_cursor"],"additionalProperties":false},"EvaluationResultList":{"type":"object","properties":{"data":{"type":"array","items":{"$ref":"#/components/schemas/EvaluationResult"}},"has_more":{"type":"boolean"},"next_cursor":{"type":["string","null"]}},"required":["data","has_more","next_cursor"],"additionalProperties":false},"EndpointList":{"type":"object","properties":{"data":{"type":"array","items":{"$ref":"#/components/schemas/Endpoint"}},"has_more":{"type":"boolean"},"next_cursor":{"type":["string","null"]}},"required":["data","has_more","next_cursor"],"additionalProperties":false},"ApiKeyList":{"type":"object","properties":{"data":{"type":"array","items":{"$ref":"#/components/schemas/ApiKey"}},"has_more":{"type":"boolean"},"next_cursor":{"type":["string","null"]}},"required":["data","has_more","next_cursor"],"additionalProperties":false},"BrowserSessionList":{"type":"object","properties":{"data":{"type":"array","items":{"$ref":"#/components/schemas/BrowserSession"}},"has_more":{"type":"boolean"},"next_cursor":{"type":["string","null"]}},"required":["data","has_more","next_cursor"],"additionalProperties":false},"DecisionRunMetrics":{"type":"object","properties":{"protocol":{"$ref":"#/components/schemas/DecisionNative___schema0"},"count":{"$ref":"#/components/schemas/DecisionNative___schema1"},"score":{"$ref":"#/components/schemas/DecisionNative___schema3"},"metrics":{"$ref":"#/components/schemas/DecisionNative___schema4"}},"required":["protocol","count","score","metrics"],"additionalProperties":false},"DecisionNative___schema0":{"type":"string","const":"decision-index-0.3-author-v1"},"DecisionNative___schema1":{"minimum":1,"$ref":"#/components/schemas/DecisionNative___schema2"},"DecisionNative___schema2":{"type":"integer","minimum":0,"maximum":100000},"DecisionNative___schema3":{"type":"number","minimum":0,"maximum":1},"DecisionNative___schema4":{"type":"object","properties":{"native":{"anyOf":[{"type":"object","properties":{"catalog_id":{"$ref":"#/components/schemas/DecisionNative___schema5"},"dataset":{"$ref":"#/components/schemas/DecisionNative___schema6"},"requests":{"$ref":"#/components/schemas/DecisionNative___schema2"},"answered":{"$ref":"#/components/schemas/DecisionNative___schema2"},"unsupported":{"$ref":"#/components/schemas/DecisionNative___schema2"},"errors":{"$ref":"#/components/schemas/DecisionNative___schema2"},"abstained":{"$ref":"#/components/schemas/DecisionNative___schema2"},"pending":{"$ref":"#/components/schemas/DecisionNative___schema2"},"metric":{"$ref":"#/components/schemas/DecisionNative___schema7"},"score":{"$ref":"#/components/schemas/DecisionNative___schema8"},"median_ms":{"$ref":"#/components/schemas/DecisionNative___schema9"},"scored_requests":{"$ref":"#/components/schemas/DecisionNative___schema2"},"reference_same_cases":{"$ref":"#/components/schemas/DecisionNative___schema8"},"tracks":{"type":"object","propertyNames":{"type":"string","pattern":"^[A-Za-z0-9_. /+-]{1,100}$"},"additionalProperties":{"type":"object","properties":{"metric":{"anyOf":[{"$ref":"#/components/schemas/DecisionNative___schema6"},{"type":"null"}]},"score":{"$ref":"#/components/schemas/DecisionNative___schema8"},"scored_requests":{"$ref":"#/components/schemas/DecisionNative___schema2"}},"required":["metric","score","scored_requests"],"additionalProperties":false}},"detail":{"type":"object","properties":{"field_accuracy":{"$ref":"#/components/schemas/DecisionNative___schema8"},"case_exact_accuracy":{"$ref":"#/components/schemas/DecisionNative___schema8"},"macro_f1":{"$ref":"#/components/schemas/DecisionNative___schema8"},"cluster_macro_accuracy":{"$ref":"#/components/schemas/DecisionNative___schema8"},"custom_metrics":{"$ref":"#/components/schemas/DecisionNative___schema10"},"subgroups":{"type":"object","propertyNames":{"type":"string","pattern":"^[A-Za-z0-9_. /+-]{1,100}$"},"additionalProperties":{"type":"object","properties":{"n":{"$ref":"#/components/schemas/DecisionNative___schema2"},"mean":{"$ref":"#/components/schemas/DecisionNative___schema8"}},"required":["n","mean"],"additionalProperties":false}},"positive_f1_by_field":{"$ref":"#/components/schemas/DecisionNative___schema10"},"category_macro_f1":{"$ref":"#/components/schemas/DecisionNative___schema8"},"positive_micro_f1":{"$ref":"#/components/schemas/DecisionNative___schema8"}},"additionalProperties":false}},"required":["catalog_id","dataset","requests","answered","unsupported","errors","abstained","pending","metric","score","median_ms","scored_requests","reference_same_cases"],"additionalProperties":false},{"type":"object","properties":{"catalog_id":{"$ref":"#/components/schemas/DecisionNative___schema5"},"dataset":{"$ref":"#/components/schemas/DecisionNative___schema6"},"requests":{"$ref":"#/components/schemas/DecisionNative___schema2"},"answered":{"$ref":"#/components/schemas/DecisionNative___schema2"},"unsupported":{"$ref":"#/components/schemas/DecisionNative___schema2"},"errors":{"$ref":"#/components/schemas/DecisionNative___schema2"},"abstained":{"$ref":"#/components/schemas/DecisionNative___schema2"},"pending":{"$ref":"#/components/schemas/DecisionNative___schema2"},"metric":{"$ref":"#/components/schemas/DecisionNative___schema7"},"score":{"$ref":"#/components/schemas/DecisionNative___schema8"},"median_ms":{"$ref":"#/components/schemas/DecisionNative___schema9"},"scored_fields":{"$ref":"#/components/schemas/DecisionNative___schema2"},"field_accuracy":{"$ref":"#/components/schemas/DecisionNative___schema8"},"chance":{"$ref":"#/components/schemas/DecisionNative___schema8"}},"required":["catalog_id","dataset","requests","answered","unsupported","errors","abstained","pending","metric","score","median_ms","scored_fields","field_accuracy","chance"],"additionalProperties":false}]},"index_component":{"type":"object","properties":{"raw":{"$ref":"#/components/schemas/DecisionNative___schema3"},"skill":{"$ref":"#/components/schemas/DecisionNative___schema3"},"coverage":{"$ref":"#/components/schemas/DecisionNative___schema3"},"random":{"$ref":"#/components/schemas/DecisionNative___schema8"},"rule":{"type":"string","enum":["track","chance","vs baseline","shown, not counted"]},"tracks":{"maxItems":100,"type":"array","items":{"$ref":"#/components/schemas/DecisionNative___schema11"}}},"required":["raw","skill","coverage","random","rule"],"additionalProperties":false}},"required":["native","index_component"],"additionalProperties":false},"DecisionNative___schema5":{"type":"integer","minimum":1,"maximum":99},"DecisionNative___schema6":{"type":"string","minLength":1,"maxLength":160},"DecisionNative___schema7":{"anyOf":[{"$ref":"#/components/schemas/DecisionNative___schema6"},{"type":"null"}]},"DecisionNative___schema8":{"anyOf":[{"$ref":"#/components/schemas/DecisionNative___schema3"},{"type":"null"}]},"DecisionNative___schema9":{"anyOf":[{"type":"number","minimum":0},{"type":"null"}]},"DecisionNative___schema10":{"type":"object","propertyNames":{"type":"string","pattern":"^[A-Za-z0-9_. /@+-]{1,100}$"},"additionalProperties":{"type":["number","null"]}},"DecisionNative___schema11":{"type":"object","properties":{"track":{"$ref":"#/components/schemas/DecisionNative___schema6"},"raw":{"$ref":"#/components/schemas/DecisionNative___schema3"},"random":{"$ref":"#/components/schemas/DecisionNative___schema3"},"skill":{"$ref":"#/components/schemas/DecisionNative___schema3"},"planned_cases":{"$ref":"#/components/schemas/DecisionNative___schema2"},"coverage":{"$ref":"#/components/schemas/DecisionNative___schema3"},"pending":{"$ref":"#/components/schemas/DecisionNative___schema3"}},"required":["track","raw","random","skill","planned_cases","coverage","pending"],"additionalProperties":false}}}}