afterburnby cogniphicOpen the console

afterburnPlatform

The whole loop,
in one console.

Endpoints, benchmarks, policies, routes, a gateway and the evidence they leave behind. Afterburn keeps all six on one record, so a routing decision can always be traced back to the measurements that made it.

Open the consoleRead the API

Afterburn · fleet view Sample data

endpoints
6 connected
requests
218 / s
first token p95
180 ms
route
chat · r4
fallbacks
0.4%
cost / 1M out
$0.167
  • Load per endpoint
  • Latency surface
  • Route candidates
  • OpenAI-compatible
  • Streams timed, not sampled
  • Every change a revision
  • Self-hosted

Six parts

Six parts,
one record.

Each part reads from and writes to the same history. Nothing in the console is estimated; it is measured by the requests Afterburn sends or forwards.

  • Endpoints

    Any OpenAI-compatible base URL, the model it serves, its key and its price. Health is checked against its own model list.

  • Benchmarks

    The same prompts to every endpoint at a set concurrency, streamed and timed: first token, between tokens, the whole answer, errors.

  • Policies

    A latency limit and an error budget. Only endpoints that keep them are ranked, cheapest first, with the reasons for the rest.

  • Routes

    A name your apps call as the model, a primary endpoint and its fallbacks. Every change is a revision you can roll back.

  • Gateway

    One OpenAI-compatible URL with its own keys. Streams pass through, failures fall through, every call is logged.

  • Evidence

    Live latency, tokens and cost per route and endpoint, and the before and after of every route change, kept with it.

A decision

How a route
changes hands.

Five steps, all visible. You can stop at any one of them, and the console shows what each step decided and why.

  1. Connect

    Add the endpoint. Afterburn lists its models to prove it answers.

  2. Measure

    Run a benchmark on your prompts, or let live traffic speak.

  3. Qualify

    Drop every endpoint that breaks the latency limit or the error budget.

  4. Rank

    Order the rest by measured cost per million output tokens.

  5. Apply

    Make the winner the route’s primary, keep the old one as fallback.

Telemetry

What the engines say,
beside what users feel.

vLLM and SGLang publish Prometheus metrics. Point Afterburn at them and it reads the queue, the cache and the throughput next to the latencies it measures itself.

Engine metrics · vllm-h100Sample data

requests running
14
requests waiting
3
KV cache in use
64%
prefix cache hit rate
41%
generation throughput
1,180 tok/s

Measured by Afterburn · vllm-h100Sample data

first token, p95
175 ms
between tokens, p50
11 ms
total, p95
1.9 s
errors, last hour
0
cost per 1M output
$0.21

Self-hosted

Runs where
your endpoints run.

A FastAPI service and a Next.js console. SQLite on a laptop, PostgreSQL in a cluster. Endpoint keys never leave your network.

Terminaldocker compose

$ docker compose up --build
postgres   ready · 5432
api        alembic upgrade head · ok
api        uvicorn · 0.0.0.0:8000
web        next start · 0.0.0.0:3200

console    http://localhost:3200/app
gateway    http://localhost:8000/v1
  • Your database

    SQLite by default, PostgreSQL with one setting. Migrations run with Alembic.

  • Your network

    The gateway calls your endpoints directly. Nothing routes through us.

  • Your keys

    Endpoint keys are sealed with a key only your deployment holds.

  • Your rules

    Roles and an audit trail on every change, from the first sign-in.

See your own
endpoints first.

Connect one, run a benchmark, and read what it actually does.

Open the consoleRead the API