Developer Hub
Welcome to the ModelVantage Developer Hub. Our platform actively pings the world's leading Large Language Model (LLM) endpoints from multiple AWS regions every single minute. Rather than relying on vendor status pages—which are often self-reported and lag real incidents by 30 to 60 minutes—ModelVantage acts as an independent, neutral third-party monitor, ensuring you know about outages and regional latency spikes before your users do.
REST API Guide
Explore the base URL, request headers, and rate limits.
Endpoint Reference
Get detailed request and response formats for our live endpoints.
Real-Time Alerts
Configure Slack alerts and generic webhooks with signature security.
Error Directory
Reference our standardized HTTP error codes and troubleshooting guidelines.
API Capabilities
ModelVantage exposes robust programmatic access to both current and historical performance data:
- Real-Time Snapshots: Access end-to-end latency and HTTP status codes for every active model/region combination within seconds.
- Historical Archive: Retrieve granular historical poll records with up to a 90-day lookback window for in-depth regional analysis.
- Webhook Push Alerts: Dispatch real-time notifications to your servers or Slack channels the instant a model degrades or recovers.
- SLA Certifications: Systematically download timestamped compliance PDF reports to substantiate vendor credit claims. Every account can download each monthly report up to five times.
Onboarding
Getting integrated with the ModelVantage API takes less than five minutes. Follow these simple steps to register, generate secure credentials, and issue your first authorized API call:
- Create a free account: Create an account in the Developer Portal — an email code is the whole signup, and nothing is charged. API keys, alerts and reports live there.
- Authenticate on the Developer Portal: Navigate to the ModelVantage Developer Portal. Enter your account's email address. A 6-digit One-Time Password (OTP) will be sent to your inbox. Enter the code to log-in to the portal.
-
Generate Your API Key:
In the portal dashboard, locate the "API Keys" widget and click "Create New Key". Provide a descriptive label (e.g.,
Production Gateway).Security Boundary: Your newly generated API key (prefixed with
mv_live_) is displayed exactly once. If you lose this key, you must generate a new one. -
Pass the Key in Request Headers:
Keyed calls send your API key in the
X-API-Keyrequest header. See the example below:GET /v1/snapshot HTTP/1.1 Host: api.modelvantage.com X-API-Key: mv_live_your_actual_key_token_here Accept: application/json
REST API Guide
All request endpoints are served over HTTPS relative to our global API gateway:
https://api.modelvantage.com/v1
Rate Limits
To maintain high availability and prevent resource exhaustion, the ModelVantage REST API implements sliding-window rate limiting:
- Rate Limit:
60 requests per minute(evaluated using a sliding window per API key).
Usage Response Headers
Every API response reports your live rate-limit window; an authenticated response also reports the monthly quota. The telemetry headers are:
| Response Header | Type | Description |
|---|---|---|
X-RateLimit-Limit |
Integer | The maximum requests allowed per 60-second sliding window (always 60). |
X-RateLimit-Remaining |
Integer | The number of requests remaining in the current sliding window. |
X-RateLimit-Reset |
Integer | Unix epoch timestamp (seconds UTC) when the current rate limit window resets. |
Retry-After |
Integer | Returned only on 429 RATE_LIMIT_EXCEEDED responses. Specifies the number of seconds to wait before retrying. |
X-Quota-Limit |
Integer | The monthly request allowance for the account — 10000 on a free account. Sent on authenticated responses only — a 401 has no account to measure. |
X-Quota-Remaining |
Integer | Requests remaining in the current monthly quota window. Reaching 0 means the next call returns 429 QUOTA_EXCEEDED. Sent on authenticated responses only. |
Sentinel Values
Note on Null Safety: To simplify parsing in strongly-typed client languages (such as Rust, Go, Java, and C#) and prevent runtime null pointer exceptions, ModelVantage never returns null primitives for numerical telemetry fields.
Instead, when end-to-end latency or HTTP statuses are unavailable (due to a connection timeout, DNS resolution failure, skipped probe, or vendor downtime), the API returns a sentinel value of -1 in the latency_ms and http_status fields.
Endpoint Reference
GET /snapshot
Retrieves the latest probe telemetry records for every monitored model and AWS region combination. This endpoint acts as a live, real-time pulse of the AI ecosystem.
Caching Policy: Responses are cached globally for 30 seconds. Consecutive queries within the 30-second window will return identical payload data and timestamp fields, but will still consume rate limits.
| Header Parameter | Type | Required | Description |
|---|---|---|---|
X-API-Key |
String | Yes | Your secure API credential token. Format: mv_live_... |
Response Schema
Returns a roster version, the generation timestamp, and a list of live status entries:
roster_id(string): A content hash identifying the monitored-model roster this snapshot was built against. When it changes, refetchGET /models.snapshot_at(string, ISO-8601): The generation timestamp (UTC) of this snapshot data.items(array): List of live status entries.model_id(string): The model ID (e.g.,gpt-5.4,claude-sonnet-5-5,gemini-2.5-flash). Join againstGET /modelsto resolve provider and tier.region(string): The AWS region from which the minute check was initiated (e.g.,us-east-1,us-west-2,eu-west-1,ap-southeast-1,us-east-2).http_status(integer): The raw HTTP response status code. Returns-1on timeouts/failures.latency_ms(integer): The raw end-to-end execution latency in milliseconds. Returns-1on timeouts/failures.polled_at(string, ISO-8601): The exact UTC timestamp when this model probe was completed.
Example JSON Response
{
"roster_id": "a3f9c1e2d4b5",
"snapshot_at": "2026-06-21T22:00:00Z",
"items": [
{
"model_id": "gpt-5.4",
"region": "us-east-1",
"http_status": 200,
"latency_ms": 342,
"polled_at": "2026-06-21T21:59:47Z"
},
{
"model_id": "claude-sonnet-5-5",
"region": "eu-west-1",
"http_status": 200,
"latency_ms": 512,
"polled_at": "2026-06-21T21:59:41Z"
},
{
"model_id": "gemini-2.5-flash",
"region": "us-west-2",
"http_status": -1,
"latency_ms": -1,
"polled_at": "2026-06-21T21:59:39Z"
}
]
}
GET /models
Retrieves the monitored-model roster: one entry per model with its
provider, tier and poll_interval_minutes,
versioned by a roster_id.
Caching Policy: Responses are cacheable for a long time — the roster only
changes when the monitored model set changes (signalled by a new roster_id).
Fetch it once, cache it, and refetch only when a /snapshot
response advertises a different roster_id.
Response Schema
roster_id(string): Content hash identifying this roster version.models(array): One entry per monitored model.provider(string):openai,anthropic, orgoogle.model_id(string): The model ID. Join key against/snapshotitems.tier(string):standard(higher-capability/thinking) orfast(low-latency).poll_interval_minutes(integer): Minutes between scheduled probes. Grade freshness against it — a poll older than twice this plus two minutes (capped at 20) is stale.
Example JSON Response
{
"roster_id": "a3f9c1e2d4b5",
"models": [
{ "provider": "openai", "model_id": "gpt-5.4", "tier": "standard", "poll_interval_minutes": 5 },
{ "provider": "openai", "model_id": "gpt-5.4-mini", "tier": "fast", "poll_interval_minutes": 1 },
{ "provider": "anthropic", "model_id": "claude-sonnet-5-5", "tier": "standard", "poll_interval_minutes": 5 },
{ "provider": "google", "model_id": "gemini-2.5-flash", "tier": "fast", "poll_interval_minutes": 1 }
]
}
GET /archive
Queries historically archived poll records from our database. Telemetry records are ordered chronologically by polled_at descending (newest first). This endpoint utilizes cursor-based keyset pagination to handle massive query result sets efficiently.
| Query Parameter | Type | Required | Description |
|---|---|---|---|
from |
String (ISO-8601) | Yes | Inclusive starting UTC timestamp of the range. Must not exceed the 90-day lookback limit. |
to |
String (ISO-8601) | Yes | Exclusive ending UTC timestamp of the range. |
provider |
String | No | Filter by AI provider: openai, anthropic, or google. |
model_id |
String | No | Filter by specific model ID (e.g., gpt-5.4-mini, claude-haiku-4-5). |
region |
String | No | Filter by specific monitoring AWS region. |
http_status |
Integer | No | Filter by exact HTTP status response. Pass -1 to query timed-out/failed polls. |
limit |
Integer | No | Number of records returned per page. Default is 100, maximum is 500. |
cursor |
String | No | Opaque keyset base64 cursor token to retrieve the next page of results. |
Keyset Pagination Mechanics
ModelVantage utilizes keyset pagination.
- Extracting Cursor: When executing a query, if additional records exist, the response object returns
pagination.has_more = trueand exposes a token underpagination.next_cursor. - Fetching the Next Page: For the subsequent query, supply this cursor token unmodified in your query parameters as
?cursor=.... Cursors are encrypted, tokens—so do not attempt to construct them manually. When no more records are available,has_morewill returnfalseandnext_cursorwill evaluate tonull.
Example JSON Response
{
"data": [
{
"id": 4829301,
"provider": "openai",
"model_id": "gpt-5.4",
"region": "us-east-1",
"polled_at": "2026-06-21T21:59:47Z",
"latency_ms": 342,
"http_status": 200
},
{
"id": 4829298,
"provider": "openai",
"model_id": "gpt-5.4",
"region": "us-east-1",
"polled_at": "2026-06-21T21:54:51Z",
"latency_ms": 298,
"http_status": 200
},
{
"id": 4829195,
"provider": "openai",
"model_id": "gpt-5.4",
"region": "us-east-1",
"polled_at": "2026-06-21T21:49:53Z",
"latency_ms": -1,
"http_status": -1
}
],
"pagination": {
"limit": 100,
"has_more": true,
"next_cursor": "eyJwb2xsZWRfYXQiOiAiMjAyNi0wNi0yMVQyMTo0NDo0OFoiLCAiaWQiOiA0ODI5MTAyfQ==",
"returned_count": 3
}
}
Multi-Region Failover Recipes
ModelVantage is the health signal; your application stays in control of the switch. These drop-in recipes read our status data and hand you a healthy model and region, so you can call the provider yourself with your own key. We never sit in your request path.
How it works
- Fetch the roster once:
GET /v1/modelsreturns the monitored models with theirproviderandtier, versioned by aroster_id. - Poll for status:
GET /v1/snapshotreturns livehttp_statusandlatency_msper model and region, cached server-side for 30 seconds. - Join on
model_id: combine the two to know which (provider, model, region) combinations are healthy right now. - Pick and call: choose a candidate with a strategy, then call your provider SDK as usual.
- Self-updating: when this set of models changes,
roster_idchanges — refetch the roster and your mapping is current. No hard-coded model lists to maintain.
Read the sentinel: http_status and latency_ms are
-1 (not null) when a poll timed out or was skipped. Treat
-1 and any status ≥ 400 as unhealthy.
Budget: snapshots are cached for 30 seconds, and the API allows 60 requests/minute per key. Both recipes below cache locally, so a high-traffic app makes at most one snapshot call every 30 seconds.
Pull recipe — fail over on each request
Wrap your provider client. Before each call, ask ModelVantage for a healthy candidate and use it. Common decision paths, pick one or pass your own callback:
sameProviderFastest— stay on your chosen provider; use its fastest healthy model.anyProviderFastest— accept any provider; use the fastest healthy model overall.preferenceOrder— follow your own explicit list of (provider, model, region).
"""ModelVantage drop-in failover router — Python reference implementation.
Read this alongside the docs page at /docs/#failover. It is the canonical,
tested source for the Python recipe shown there.
Design (advisory, not a proxy — see decisions.md, 2026-10-02):
- Fetch the model *roster* from ``GET /v1/models`` (provider + tier per model),
cached by ``roster_id``.
- Poll ``GET /v1/snapshot`` for live status (``http_status`` / ``latency_ms`` per
model+region), cached for a short TTL.
- Pick a healthy (provider, model_id, region) from those two, then call the
provider yourself with your own key. ModelVantage never sits in your request
path.
Key ideas:
- ``roster_id`` is a content hash: refetch the roster only when a snapshot's
``roster_id`` changes, so there is no hard-coded client-side model metadata.
- A snapshot item is *healthy* when ``http_status != -1`` (and, if you like,
``< 400``); ``-1`` is the "no data / timeout" sentinel.
- A tiny TTL cache with single-flight keeps you well inside the 60 req/min rate
limit; some staleness (the server's 30s cache) is expected and fine.
"""
from __future__ import annotations
import threading
import time
from collections.abc import Callable
from dataclasses import dataclass
from typing import Optional, Protocol
import httpx
# ── Types ─────────────────────────────────────────────────────────────────────
class _Response(Protocol):
"""The slice of ``httpx.Response`` the router uses."""
def raise_for_status(self) -> "_Response": ...
def json(self) -> dict: ...
class _HttpClient(Protocol):
"""The slice of ``httpx.Client`` the router uses.
Structural (not the concrete class) so tests can inject a fake; a real
``httpx.Client`` satisfies it.
"""
def get(self, path: str) -> _Response: ...
@dataclass(frozen=True)
class Candidate:
"""One selectable (model, region) with its roster metadata and live status."""
provider: str
model_id: str
tier: str
region: str
http_status: int
latency_ms: float
class AllModelsDownError(RuntimeError):
"""Raised when no healthy candidate satisfies the caller's constraints."""
# ── Decision strategies ───────────────────────────────────────────────────────
Strategy = Callable[[list[Candidate], list[Candidate]], Optional[Candidate]]
def same_provider_fastest(preferred: list[Candidate], healthy: list[Candidate]) -> Optional[Candidate]:
"""Stay on the preferred model's provider; pick its fastest healthy model.
``preferred[0]`` is the model you wanted. Among healthy candidates from the
same provider, return the lowest-latency one.
"""
if not preferred:
return None
provider = preferred[0].provider
pool = [c for c in healthy if c.provider == provider]
if not pool:
return None
return min(pool, key=lambda c: c.latency_ms)
def any_provider_fastest(_preferred: list[Candidate], healthy: list[Candidate]) -> Optional[Candidate]:
"""Any provider: return the lowest-latency healthy candidate."""
if not healthy:
return None
return min(healthy, key=lambda c: c.latency_ms)
def preference_order(order: list[tuple[str, str, str]]) -> Strategy:
"""Build a strategy that follows an explicit (provider, model_id, region) order.
The first entry in ``order`` that is healthy wins. This is the "enterprise"
mode: fully explicit, self-owned, and it fails visibly (a stale id simply
won't match) rather than silently misrouting.
"""
def _strategy(_preferred: list[Candidate], healthy: list[Candidate]) -> Optional[Candidate]:
by_key = {(c.provider, c.model_id, c.region): c for c in healthy}
for key in order:
match = by_key.get(key)
if match is not None:
return match
return None
return _strategy
# ── TTL cache (single-flight, fail-open) ──────────────────────────────────────
class _TTLCache:
"""A minimal single-flight TTL cache.
- One entry, refreshed after ``ttl`` seconds.
- Concurrent refreshes share a single in-flight fetch (single-flight), so a
burst of requests makes one upstream call.
- On refresh failure, the last good value is returned (fail-open).
"""
def __init__(self, fetch: Callable[[], dict], ttl: float) -> None:
self._fetch = fetch
self._ttl = ttl
self._lock = threading.Lock()
self._value: Optional[dict] = None
self._expires = 0.0
def get(self, ttl_override: Optional[float] = None) -> dict:
with self._lock:
now = time.monotonic()
if self._value is not None and now < self._expires:
return self._value
try:
fresh = self._fetch()
except Exception:
if self._value is not None:
return self._value # fail-open: serve the last good value
raise
self._value = fresh
self._expires = now + (self._ttl if ttl_override is None else ttl_override)
return fresh
def get_if_matches_roster(self, roster_id: str) -> Optional[dict]:
"""Return the cached roster only if it matches ``roster_id`` (else None)."""
with self._lock:
if self._value is not None and self._value.get("roster_id") == roster_id:
return self._value
return None
def refresh(self) -> dict:
"""Force a refetch now (used when a new roster_id is observed)."""
with self._lock:
fresh = self._fetch()
self._value = fresh
self._expires = time.monotonic() + self._ttl
return fresh
# ── The router ────────────────────────────────────────────────────────────────
class FailoverRouter:
"""Selects a healthy (provider, model, region) from ModelVantage's data.
Args:
api_key: Your ``mv_live_...`` key.
client: Optional HTTP client (defaults to ``httpx.Client``; any object
exposing ``get(path)`` returning a response works, which tests use).
base_url: Customer API base, defaults to production.
snapshot_ttl: Seconds to cache a snapshot (default 30, matching the server).
roster_ttl: Seconds to cache the roster (default 6h; it also self-refreshes
immediately when a snapshot advertises a new ``roster_id``).
"""
def __init__(
self,
api_key: str,
*,
client: Optional[_HttpClient] = None,
base_url: str = "https://api.modelvantage.com/v1",
snapshot_ttl: float = 30.0,
roster_ttl: float = 6 * 3600.0,
) -> None:
self._client: _HttpClient = client or httpx.Client(
base_url=base_url,
headers={"X-API-Key": api_key},
timeout=17.0, # belt-and-braces; the server caches for 30s
)
self._snapshot = _TTLCache(self._fetch_snapshot, snapshot_ttl)
self._roster = _TTLCache(self._fetch_roster, roster_ttl)
# -- HTTP --
def _fetch_snapshot(self) -> dict:
return self._client.get("/snapshot").raise_for_status().json()
def _fetch_roster(self) -> dict:
return self._client.get("/models").raise_for_status().json()
# -- Public API --
def choose(
self,
preferred: list[tuple[str, str, str]],
strategy: Strategy,
*,
allow_providers: Optional[set[str]] = None,
require_region: Optional[str] = None,
allow_tiers: Optional[set[str]] = None,
) -> Candidate:
"""Pick a healthy candidate, or raise ``AllModelsDownError``.
Args:
preferred: Ordered (provider, model_id, region) you'd like first.
strategy: One of the built-in strategies (or your own callable).
allow_providers: Optional provider allow-list.
require_region: Optional single-region constraint.
allow_tiers: Optional tier filter, e.g. ``{"fast"}``.
"""
snapshot = self._snapshot.get()
roster = self._roster_for(snapshot)
# Join: snapshot items carry model_id only; resolve provider+tier from the roster.
meta = {m["model_id"]: m for m in roster.get("models", [])}
items_by_pair = {(i["model_id"], i["region"]): i for i in snapshot.get("items", [])}
def passes_filters(c: Candidate) -> bool:
if c.http_status == -1 or c.http_status >= 400:
return False
if allow_providers is not None and c.provider not in allow_providers:
return False
if require_region is not None and c.region != require_region:
return False
if allow_tiers is not None and c.tier not in allow_tiers:
return False
return True
def build(item: dict) -> Optional[Candidate]:
info = meta.get(item["model_id"])
if info is None:
return None # not in the roster (should not happen; be safe)
return Candidate(
provider=info["provider"],
model_id=item["model_id"],
tier=info.get("tier", ""),
region=item["region"],
http_status=int(item.get("http_status", -1)),
latency_ms=float(item.get("latency_ms", -1)),
)
healthy = [c for c in (build(i) for i in snapshot.get("items", [])) if c and passes_filters(c)]
# Preferred candidates are NOT health-filtered: a strategy like
# same_provider_fastest needs the preferred *provider* even when the
# preferred model itself is down, so it can switch within that provider.
preferred_candidates: list[Candidate] = []
for (provider, model_id, region) in preferred:
item = items_by_pair.get((model_id, region))
if item is None:
continue
candidate = build(item)
if candidate is not None and candidate.provider == provider:
preferred_candidates.append(candidate)
if not healthy:
raise AllModelsDownError("No healthy model matched the given constraints.")
chosen = strategy(preferred_candidates, healthy)
if chosen is None:
raise AllModelsDownError(
"The requested strategy found no healthy candidate among the constraints."
)
return chosen
def _roster_for(self, snapshot: dict) -> dict:
"""Return the roster matching the snapshot's ``roster_id``.
If the cached roster's id differs, force a refetch (the roster changed);
a TTL-valid but wrong-id cache must never be served.
"""
cached = self._roster.get_if_matches_roster(snapshot.get("roster_id", ""))
if cached is not None:
return cached
return self._roster.refresh()
/**
* ModelVantage drop-in failover router — TypeScript reference implementation.
*
* Read this alongside the docs page at /docs/#failover. It is the canonical,
* tested source for the TypeScript recipe shown there.
*
* Design (advisory, not a proxy — see decisions.md, 2026-10-02):
* - Fetch the model *roster* from `GET /v1/models` (provider + tier per model),
* cached by `rosterId`.
* - Poll `GET /v1/snapshot` for live status (`httpStatus` / `latencyMs` per
* model+region), cached for a short TTL.
* - Pick a healthy (provider, modelId, region) from those two, then call the
* provider yourself with your own key. ModelVantage never sits in your request
* path.
*
* Zero dependencies: uses the global `fetch` (Node 18+).
*/
export interface ModelInfo {
provider: string;
model_id: string;
tier: string;
}
export interface ModelsResponse {
roster_id: string;
models: ModelInfo[];
}
export interface SnapshotItem {
model_id: string;
region: string;
http_status: number;
latency_ms: number;
polled_at: string;
}
export interface SnapshotResponse {
roster_id: string;
snapshot_at: string;
items: SnapshotItem[];
}
export interface Candidate {
provider: string;
modelId: string;
tier: string;
region: string;
httpStatus: number;
latencyMs: number;
}
export type Strategy = (preferred: Candidate[], healthy: Candidate[]) => Candidate | undefined;
export class AllModelsDownError extends Error {}
/** Stay on the preferred model's provider; pick its fastest healthy model. */
export const sameProviderFastest: Strategy = (preferred, healthy) => {
if (preferred.length === 0) return undefined;
const provider = preferred[0].provider;
const pool = healthy.filter((c) => c.provider === provider);
if (pool.length === 0) return undefined;
return pool.reduce((best, c) => (c.latencyMs < best.latencyMs ? c : best));
};
/** Any provider: return the lowest-latency healthy candidate. */
export const anyProviderFastest: Strategy = (_preferred, healthy) => {
if (healthy.length === 0) return undefined;
return healthy.reduce((best, c) => (c.latencyMs < best.latencyMs ? c : best));
};
/** Follow an explicit [provider, modelId, region] order; first healthy wins. */
export function preferenceOrder(order: Array<[string, string, string]>): Strategy {
return (_preferred, healthy) => {
const byKey = new Map(healthy.map((c) => [`${c.provider}|${c.modelId}|${c.region}`, c]));
for (const [provider, modelId, region] of order) {
const match = byKey.get(`${provider}|${modelId}|${region}`);
if (match) return match;
}
return undefined;
};
}
/** A minimal single-flight TTL cache with fail-open on refresh error. */
class TTLCache<T extends { roster_id: string }> {
private value?: T;
private expires = 0;
private inflight?: Promise<T>;
private readonly fetchFn: () => Promise<T>;
private readonly ttlMs: number;
constructor(fetchFn: () => Promise<T>, ttlMs: number) {
this.fetchFn = fetchFn;
this.ttlMs = ttlMs;
}
async get(): Promise<T> {
const now = Date.now();
if (this.value !== undefined && now < this.expires) return this.value;
if (this.inflight !== undefined) return this.inflight; // single-flight
this.inflight = this.fetchFn()
.then((fresh) => {
this.value = fresh;
this.expires = Date.now() + this.ttlMs;
return fresh;
})
.catch((err) => {
if (this.value !== undefined) return this.value; // fail-open
throw err;
})
.finally(() => {
this.inflight = undefined;
});
return this.inflight;
}
/** Return the cached value only if its roster_id matches. */
getIfMatches(rosterId: string): T | undefined {
return this.value !== undefined && this.value.roster_id === rosterId ? this.value : undefined;
}
/** Force a refetch now (used when a new roster_id is observed). */
async refresh(): Promise<T> {
const fresh = await this.fetchFn();
this.value = fresh;
this.expires = Date.now() + this.ttlMs;
return fresh;
}
}
export interface ChooseOptions {
allowProviders?: Set<string>;
requireRegion?: string;
allowTiers?: Set<string>;
}
export class FailoverRouter {
private readonly snapshotCache: TTLCache<SnapshotResponse>;
private readonly rosterCache: TTLCache<ModelsResponse>;
private readonly apiKey: string;
constructor(
apiKey: string,
options: {
baseUrl?: string;
snapshotTtlMs?: number;
rosterTtlMs?: number;
fetchFn?: typeof fetch;
} = {},
) {
this.apiKey = apiKey;
const baseUrl = options.baseUrl ?? "https://api.modelvantage.com/v1";
const fetchFn =
options.fetchFn ??
((input: RequestInfo | URL, init?: RequestInit) =>
fetch(input, {
...init,
headers: { ...(init?.headers ?? {}), "X-API-Key": this.apiKey },
}));
const get = async <T>(path: string): Promise<T> => {
const res = await fetchFn(`${baseUrl}${path}`);
if (!res.ok) throw new Error(`ModelVantage ${path} returned ${res.status}`);
return (await res.json()) as T;
};
this.snapshotCache = new TTLCache(() => get<SnapshotResponse>("/snapshot"), options.snapshotTtlMs ?? 30_000);
this.rosterCache = new TTLCache(() => get<ModelsResponse>("/models"), options.rosterTtlMs ?? 6 * 3600_000);
}
async choose(
preferred: Array<[string, string, string]>,
strategy: Strategy,
options: ChooseOptions = {},
): Promise<Candidate> {
const snapshot = await this.snapshotCache.get();
const roster = await this.rosterFor(snapshot.roster_id);
const meta = new Map(roster.models.map((m) => [m.model_id, m]));
const itemsByPair = new Map(snapshot.items.map((i) => [`${i.model_id}|${i.region}`, i]));
const passes = (c: Candidate): boolean => {
if (c.httpStatus === -1 || c.httpStatus >= 400) return false;
if (options.allowProviders && !options.allowProviders.has(c.provider)) return false;
if (options.requireRegion && c.region !== options.requireRegion) return false;
if (options.allowTiers && !options.allowTiers.has(c.tier)) return false;
return true;
};
const build = (item: SnapshotItem): Candidate | undefined => {
const info = meta.get(item.model_id);
if (info === undefined) return undefined;
return {
provider: info.provider,
modelId: item.model_id,
tier: info.tier,
region: item.region,
httpStatus: item.http_status,
latencyMs: item.latency_ms,
};
};
const healthy = snapshot.items
.map(build)
.filter((c): c is Candidate => c !== undefined && passes(c));
// Preferred candidates are NOT health-filtered: a strategy like
// sameProviderFastest needs the preferred *provider* even when the preferred
// model itself is down, so it can switch within that provider.
const preferredCandidates: Candidate[] = [];
for (const [provider, modelId, region] of preferred) {
const item = itemsByPair.get(`${modelId}|${region}`);
if (item === undefined) continue;
const candidate = build(item);
if (candidate !== undefined && candidate.provider === provider) {
preferredCandidates.push(candidate);
}
}
if (healthy.length === 0) {
throw new AllModelsDownError("No healthy model matched the given constraints.");
}
const chosen = strategy(preferredCandidates, healthy);
if (chosen === undefined) {
throw new AllModelsDownError("The requested strategy found no healthy candidate among the constraints.");
}
return chosen;
}
private async rosterFor(rosterId: string): Promise<ModelsResponse> {
const cached = this.rosterCache.getIfMatches(rosterId);
if (cached !== undefined) return cached;
return this.rosterCache.refresh();
}
}
Push recipe — react to an outage alert
If you would rather be told the moment something breaks than poll for it, register a
webhook in the Developer Portal.
We deliver a signed model.outage event; your handler flips a routing flag
or clears a cached client, so the very next request already avoids the outage. Each
payload includes a data.alternative_models list of healthy models in that
region, ready to use as your fallback.
Verify every request: webhooks are signed with
X-ModelVantage-Signature (HMAC-SHA256). See
Signature Security for the verification snippet.
"""ModelVantage webhook failover handler — Python reference implementation.
Read this alongside the docs page at /docs/#failover-webhook. It is the canonical,
tested source for the webhook recipe shown there.
Push-based failover: instead of polling /v1/snapshot, you register a webhook in
the Developer Portal and react to the signed ``model.outage`` event. The payload
carries a ``data.alternative_models`` list (healthy models in the affected
region), which this handler uses to update a routing table so the *next* request
already avoids the degraded model.
Verify every request with ``X-ModelVantage-Signature`` before trusting it.
"""
from __future__ import annotations
import hashlib
import hmac
import time
from typing import Optional
def verify_signature(
payload: bytes,
secret: str,
header: str,
tolerance_sec: int = 300,
) -> bool:
"""Constant-time verification of ``X-ModelVantage-Signature``.
The header format is ``t={unix_seconds},v1={hex_hmac_sha256}`` and the signed
message is ``b"t={timestamp}." + payload``. Reject stale timestamps to blunt
replay attacks.
"""
if not header or not secret:
return False
parts = dict(item.split("=", 1) for item in header.split(",") if "=" in item)
try:
timestamp = int(parts["t"])
received = parts["v1"]
except (KeyError, ValueError):
return False
if abs(int(time.time()) - timestamp) > tolerance_sec:
return False
message = f"t={timestamp}.".encode("utf-8") + payload
expected = hmac.new(secret.encode("utf-8"), message, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, received)
class RoutingTable:
"""A tiny in-memory routing table the webhook handler mutates.
In production this might be Redis, a feature flag, or a shared dict behind
your web framework. The important part: an outage event pins the affected
(model_id, region) to a healthy alternative from the payload.
"""
def __init__(self) -> None:
self._pinned: dict[tuple[str, str], str] = {}
def pin(self, model_id: str, region: str, alternative_model_id: str) -> None:
self._pinned[(model_id, region)] = alternative_model_id
def unpin(self, model_id: str, region: str) -> None:
self._pinned.pop((model_id, region), None)
def alternative_for(self, model_id: str, region: str) -> Optional[str]:
return self._pinned.get((model_id, region))
def make_handler(secret: str, table: RoutingTable):
"""Return a handler that verifies, then updates ``table`` from the event.
Signature: ``handler(raw_body: bytes, signature_header: str) -> int`` — returns
an HTTP status code (204 on success, 401 on a bad signature).
"""
def handler(raw_body: bytes, signature_header: str) -> int:
if not verify_signature(raw_body, secret, signature_header):
return 401
import json
event = json.loads(raw_body)
data = event.get("data", {})
model_id = data.get("model")
region = data.get("region", "")
status = data.get("status")
if not model_id:
return 204 # nothing actionable
if status == "outage":
alternatives = data.get("alternative_models", [])
healthy_id = _first_healthy_id(alternatives)
if healthy_id:
table.pin(model_id, region, healthy_id)
elif status == "recovered":
table.unpin(model_id, region)
return 204
return handler
def _first_healthy_id(alternatives: list[dict]) -> Optional[str]:
"""Pick the first alternative's model_id (the list is already healthy models)."""
for alt in alternatives:
model_id = alt.get("model_id")
if model_id:
return model_id
return None
/**
* ModelVantage webhook failover handler — TypeScript reference implementation.
*
* Read this alongside the docs page at /docs/#failover-webhook. It is the
* canonical, tested source for the webhook recipe shown there.
*
* Push-based failover: register a webhook in the Developer Portal and react to
* the signed `model.outage` event. The payload carries a
* `data.alternative_models` list (healthy models in the affected region), which
* this handler uses to update a routing table so the *next* request already
* avoids the degraded model.
*
* Verification uses Web Crypto (available in Node 18+ and edge runtimes).
*/
export interface WebhookData {
model: string;
region?: string;
status: string;
alternative_models?: Array<{ model_id: string; provider: string; region: string }>;
}
export interface WebhookEvent {
event: string;
data: WebhookData;
}
/** Constant-time hex comparison (fold XOR over equal-length strings). */
function timingSafeEqualHex(a: string, b: string): boolean {
if (a.length !== b.length) return false;
let diff = 0;
for (let i = 0; i < a.length; i++) diff |= a.charCodeAt(i) ^ b.charCodeAt(i);
return diff === 0;
}
/** Verify `X-ModelVantage-Signature: t=<unix>,v1=<hex>` over `t=<unix>.<body>`. */
export async function verifySignature(
payload: Uint8Array,
secret: string,
header: string,
toleranceSec = 300,
): Promise<boolean> {
if (!header || !secret) return false;
const parts: Record<string, string> = {};
for (const kv of header.split(",")) {
const idx = kv.indexOf("=");
if (idx !== -1) parts[kv.slice(0, idx)] = kv.slice(idx + 1);
}
const timestamp = Number.parseInt(parts["t"] ?? "", 10);
const received = parts["v1"];
if (!Number.isFinite(timestamp) || !received) return false;
if (Math.abs(Math.floor(Date.now() / 1000) - timestamp) > toleranceSec) return false;
const enc = new TextEncoder();
const prefix = enc.encode(`t=${timestamp}.`);
const message = new Uint8Array(prefix.length + payload.length);
message.set(prefix, 0);
message.set(payload, prefix.length);
const key = await globalThis.crypto.subtle.importKey(
"raw",
enc.encode(secret),
{ name: "HMAC", hash: "SHA-256" },
false,
["sign"],
);
const sigBuf = await globalThis.crypto.subtle.sign("HMAC", key, message);
const expected = [...new Uint8Array(sigBuf)].map((b) => b.toString(16).padStart(2, "0")).join("");
return timingSafeEqualHex(expected, received);
}
/** A tiny in-memory routing table the webhook handler mutates. */
export class RoutingTable {
private pinned = new Map<string, string>();
private key(modelId: string, region: string): string {
return `${modelId}|${region}`;
}
pin(modelId: string, region: string, alternativeModelId: string): void {
this.pinned.set(this.key(modelId, region), alternativeModelId);
}
unpin(modelId: string, region: string): void {
this.pinned.delete(this.key(modelId, region));
}
alternativeFor(modelId: string, region: string): string | undefined {
return this.pinned.get(this.key(modelId, region));
}
}
/** Build a handler that verifies then updates `table`; returns an HTTP status. */
export function makeHandler(secret: string, table: RoutingTable) {
return async (rawBody: Uint8Array, signatureHeader: string): Promise<number> => {
if (!(await verifySignature(rawBody, secret, signatureHeader))) return 401;
const event = JSON.parse(new TextDecoder().decode(rawBody)) as WebhookEvent;
const data = event.data ?? ({} as WebhookData);
const modelId = data.model;
const region = data.region ?? "";
if (!modelId) return 204;
if (data.status === "outage") {
const healthyId = firstHealthyId(data.alternative_models ?? []);
if (healthyId) table.pin(modelId, region, healthyId);
} else if (data.status === "recovered") {
table.unpin(modelId, region);
}
return 204;
};
}
function firstHealthyId(alternatives: Array<{ model_id?: string }>): string | undefined {
for (const alt of alternatives) {
if (alt.model_id) return alt.model_id;
}
return undefined;
}
Real-Time Alerts
Slack Integration & Webhook Setup
ModelVantage can directly notify your teams directly of outages, performance degradations, and system recoveries, in real time.
Slack Integration
Integrate directly with your engineering or operations channels in seconds:
- Navigate to your Slack Workspace App configuration page and generate an "Incoming Webhook" URL (e.g.,
https://hooks.slack.com/services/...). - In the ModelVantage Developer Portal, locate the Slack Alerts module. Click "Add Webhook" and paste your Slack webhook URL.
- Click "Send Test Event" to dispatch a simulated notification message directly into your Slack channel to verify the pipeline. (Each configured alert supports up to 5 mock test events before locking).
Generic Webhook Payload Schemas
For custom backend services or automation pipelines, configure generic webhook endpoints in the portal. ModelVantage fires structured JSON payloads via POST requests with standard content headers.
Event Types
model.outage: Dispatched immediately when a monitored model stops responding completely (returns non-200 or times out).model.degraded: Fired when 1-hour average latency exceeds 1,000 ms or error rate rises above 1%.model.recovered: Triggered the moment a degraded or down model resumes normal, stable telemetry values.
Webhook Payload JSON Schema
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "WebhookEventPayload",
"type": "object",
"required": ["event", "timestamp", "data"],
"additionalProperties": false,
"properties": {
"event": {
"type": "string",
"enum": ["model.outage", "model.degraded", "model.recovered"]
},
"timestamp": {
"type": "integer",
"description": "Unix epoch timestamp when the webhook event was generated."
},
"data": {
"type": "object",
"required": ["model", "provider", "status", "message"],
"additionalProperties": false,
"properties": {
"model": {
"type": "string",
"description": "The identifier of the affected AI model."
},
"provider": {
"type": "string",
"enum": ["openai", "anthropic", "google"]
},
"status": {
"type": "string",
"enum": ["outage", "degraded", "operational"]
},
"message": {
"type": "string",
"description": "Human-readable summary of the performance event."
}
}
}
}
}
Example Payload Body
{
"event": "model.outage",
"timestamp": 1750550460,
"data": {
"model": "gpt-5.4",
"provider": "openai",
"status": "outage",
"message": "Model gpt-5.4 is currently experiencing an outage."
}
}
Webhook Security & Cryptographic Signatures
ModelVantage signs every webhook payload cryptographically. All webhook POST requests include an X-ModelVantage-Signature header.
Signing Logic
When you register a webhook in our portal, we generate a unique signing secret prefixed with whsec_. Every outgoing webhook POST includes an X-ModelVantage-Signature header containing:
- Timestamp (
t): The Unix epoch time (seconds UTC) when the webhook was dispatched. - HMAC-SHA256 Signature (
v1): Computed over the message"t={timestamp}." + payload_bytesusing your signing secret as the HMAC key.
The full header format is:
X-ModelVantage-Signature: t=1750550460,v1=9e3a6bc42f8832bd1a886ee90432bc977e2ffbc77daef60667e6ee3bc9192931
To verify, re-compute the HMAC-SHA256 over the timestamp-prefixed payload and compare using a constant-time equality function.
Replay Protection: Always verify the timestamp in the signature. Reject any request whose timestamp differs from your server's current time by more than 5 minutes (300 seconds).
Verification Example (Python)
import hmac
import hashlib
import time
def verify_signature(payload: bytes, secret: str, header: str, tolerance_sec: int = 300) -> bool:
try:
parts = dict(item.split("=") for item in header.split(","))
timestamp = int(parts["t"])
received_sig = parts["v1"]
except (KeyError, ValueError):
return False
if abs(int(time.time()) - timestamp) > tolerance_sec:
return False
message = f"t={timestamp}.".encode() + payload
expected_sig = hmac.new(secret.encode(), message, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected_sig, received_sig)
Service Level Agreements (SLAs)
Track whether your AI providers are meeting their uptime commitments. ModelVantage independently measures availability across every monitored model and region, giving you objective data to support SLA credit claims.
Monthly Compliance Reports
ModelVantage generates a signed, timestamped PDF for each month of measured uptime:
- Measured Uptime: Reports state the regional uptime measured for every endpoint. There is no customer-specific target to configure and no automatic verdict — compare the measured month against your own contract terms.
- Dispute Evidence: Use these independently verified reports as supporting documentation when filing SLA credit requests with OpenAI, Anthropic, or Google.
- Download Access: Every account can download each monthly report up to five times; the portal shows the remaining allowance on each month. The portal lists the current month plus the four before it, floored by the archive's first month.
Developer Support
We pride ourselves on providing high-resolution developer-to-developer customer support. Every account gets priority queueing:
- Response Expectation: Guaranteed next-business-day response for all support tickets.
- Direct Email Contact: For any integration hurdles, custom endpoint requests or volume questions, email our core engineering staff directly at support@modelvantage.com.
Error Code Reference
The ModelVantage Customer API returns standard HTTP response codes alongside structural JSON error envelopes.