Route coding tasks with the Chalk Model Gateway
This demo shows a judge model reading a GitHub issue and selecting a small, mid-tier, or frontier model using written criteria. Choose an issue to inspect the routing decision, then compare the estimated costs.
Which model should handle this issue?
Choose a sample issue or paste a public GitHub issue URL. Compare the issue with the judge’s criteria and selected tier: level 0 selects gpt-5-nano, level 1 gpt-5.4-mini, and level 2 gpt-5.5.
Choose a GitHub issue to see which model Chalk selects, why it fits the task, and its estimated cost.
The demo is rate limited per visitor. Issues are fetched from GitHub's public API, and only the decision is shown.
Built on Chalk
Chalk runs the judge, applies the route policy, and records the chosen model. The example supplies the issue-triage criteria and public GitHub issues.
Classify each request
A judge on the Chalk Model Gateway returns one label using the criteria stored on the route. The label selects a configured model.
Keep a fallback target
If the judge times out or returns an unexpected label, the route serves its default target. Each outcome can also have a fallback chain.
Inspect decisions before switching
Shadow mode keeps the default model serving while recording the model the judge would select. Compare those decisions with your own labels.
Scope access and trace usage
The demo key can call only its route, with rate and token budgets. Request traces record the routed model, latency, tokens, and cost.
Add a judge to a route
Use uv, the Chalk CLI, and a Chalk environment with an OpenAI connection using the openai/ prefix. That connection must support the models named in route_issue.py.
Save the Python tab as route_issue.py in an empty directory, then run Quickstart. The script uses your selected Chalk login; no token or connection ID needs to be copied into the code.
It creates chalk/issue-triage, routes one sample issue, prints the selected model, and revokes its temporary route-only key. The route remains; choose another route_name before a repeat run.
# Prerequisites: uv, Chalk CLI, and an OpenAI provider connection in Chalk.
chalk login
chalk environment YOUR_ENVIRONMENT
# In an empty directory, save the Python tab as route_issue.py, then run:
uv run --no-project --python 3.12 --script route_issue.py
# Prints the selected model and a short response. The temporary key is revoked.Stop paying frontier prices for typo fixes
Routing, keys, budgets and traces for every model your agents call. Talk to an engineer about putting a judge in front of yours.