Skip to content
costhelm

Quickstart

1. Boot the gateway

Section titled “1. Boot the gateway”
Terminal window
uvx costhelm serve

The gateway listens on http://localhost:8111 and serves its dashboard at /. No provider key yet? If Ollama is running locally, set OLLAMA_MODEL and you have a fully working, $0.00 gateway:

Terminal window
OLLAMA_MODEL=phi4:latest COSTHELM_ORDER=ollama uvx costhelm serve

For hosted providers, copy the .env.example next to where you run the gateway and fill in only the keys you have — a provider registers only when its key variable is set.

2. Make a call

Section titled “2. Make a call”
Terminal window
curl -s localhost:8111/v1/chat -H 'Content-Type: application/json' \
  -d '{"prompt": "hello", "max_tokens": 50}'

The response is the completion plus the economics envelopes:

{
  "text": "Hello! How can I help?",
  "provider": "ollama",
  "cost":   { "total_usd": 0.0, "price_source": "provider", "...": "..." },
  "budget": { "allowed": true, "enforced": true, "...": "..." },
  "cache":  null,
  "router_decision": null
}

Every call also writes one priced row to the ledger — see it at /v1/calls, or on the dashboard.

3. Arm a budget and spend it

Section titled “3. Arm a budget and spend it”

The example agent arms a $0.05 lifetime ceiling for one session, asks questions until the controller refuses, and prints the refusal arithmetic:

Terminal window
uv run python examples/budget_agent.py
ceiling armed: $0.05 for session:budget-agent-demo


call 1: ollama/phi4:latest — $0.005800 (total $0.005800) — 'An LLM gateway is...'
...
call 5: REFUSED — session:budget-agent-demo would exceed its lifetime budget:
  spent $0.047000 of $0.050000, this call projects $0.016000
  Nothing was sent to a provider; the refusal cost $0.

The whole example is ~50 lines:

examples/budget_agent.py
"""A budget-governed agent loop in ~50 lines.


Arms a $0.05 lifetime ceiling for one session, then keeps asking questions
until the gateway refuses. Every call prints what it actually cost; the final
402 prints the numbers the controller refused on. Nothing here holds a
provider key — the gateway owns all of that.


Run the gateway first (`costhelm serve`), then:


    uv run python examples/budget_agent.py
"""


from __future__ import annotations


import asyncio


from costhelm.client import BudgetExceeded, CosthelmClient


SESSION = "budget-agent-demo"
CEILING_USD = 0.05


QUESTIONS = [
    "In one sentence: what is an LLM gateway?",
    "Name three reasons to meter LLM spend per agent.",
    "What is cascade routing, in two sentences?",
    "Why should budget checks run before the provider call?",
] * 25  # more questions than the ceiling will allow




async def main() -> None:
    async with CosthelmClient() as gw:
        await gw.health()
        await gw.set_budget(f"session:{SESSION}", CEILING_USD, period="lifetime")
        print(f"ceiling armed: ${CEILING_USD:.2f} for session:{SESSION}\n")


        spent = 0.0
        for i, q in enumerate(QUESTIONS, start=1):
            try:
                r = await gw.chat(q, request={"session": SESSION, "max_tokens": 150})
            except BudgetExceeded as refusal:
                env = refusal.envelope
                print(f"\ncall {i}: REFUSED — {env.get('message')}")
                print(
                    f"  limit ${env.get('limit_usd')}, spent ${env.get('spent_usd')}, "
                    f"projected ${env.get('projected_usd')} (shortfall ${env.get('shortfall_usd')})"
                )
                print("  Nothing was sent to a provider; the refusal cost $0.")
                break
            cost = (r.get("cost") or {}).get("total_usd") or 0.0
            spent += cost
            print(
                f"call {i}: {r['provider']}/{r['model']} — ${cost:.6f} "
                f"(total ${spent:.6f}) — {r['text'][:60]!r}"
            )
        else:
            print("\nran out of questions before the ceiling — raise QUESTIONS or lower CEILING_USD")




if __name__ == "__main__":
    asyncio.run(main())

Where next

Section titled “Where next”