Serve typed decision models on Chalk GPUs
This demo shows how to run laya-typed-decisions on a Chalk GPU to answer typed questions about support tickets, invoices, alerts, and agent traces. Edit a sample record to try it, then explore the code to see how it works.
Change the record. Compare the decisions.
Start with the support ticket: the model predicts an action, category, need for a human, churn risk, and urgency. Change the customer’s final message, select Run decisions, and compare the probabilities. These are model predictions; your application decides which actions to take.
The customer was charged twice and threatens to leave. Change the final customer message, rerun, and compare churn risk and urgency.
One model call returns named choices, yes/no answers, and scores from the input record. Confidence values are model estimates.
What should the assistant do next with this conversation?
What is this customer conversation primarily about?
This conversation requires a human agent rather than automated handling.
How likely is this customer to stop doing business with us because of this interaction?
How time-sensitive is this conversation?
The model is laya-typed-decisions, a 421M-parameter ModernBERT-large specialist fine-tuned on four workflows: support, invoices, security alerts and agent traces. On the 2,000 held-out decisions in its test set it gives the reference answer 77% of the time (86% on yes/no questions), against 48% for always picking each question’s most common answer. Outside those four workflows, expect weaker answers. Its calibration temperatures were fitted on training data, so treat the probabilities as indicative and validate them on your own held-out decisions.
Built on Chalk
Chalk builds the model image and serves the decision endpoint on an NVIDIA L4. The example encodes a record once and returns choices, probabilities, and scores for the questions you supply.
A model ready to use
Deploy the model and its web app together. Chalk provisions the GPU and returns an HTTPS URL.
Deploy from Python
Define the container and its model weights in Python. Later builds can reuse cached image layers.
Model weights ready at startup
Chalk builds the pinned model weights into the container image. New replicas load them from the image without downloading them again.
Typed answers from your app
The example endpoint accepts choice, yes/no, and ordered-score questions. It returns a probability distribution for each answer so your application can apply its own decision rules.
Run it on your own
Clone the example, log in to Chalk, and run the deploy script. Chalk builds the image with the model weights, provisions an NVIDIA L4, and prints the demo URL.
You will need uv and a Chalk environment with GPU capacity. Send a record and typed questions to the /decide endpoint. Change the questions in each request, and evaluate the model on your own held-out decisions before using its probabilities as thresholds.
import httpx
response = httpx.post(
f"{web_url}/decide",
json={
"state": {
"ticket": "Charged twice for our annual renewal. Refund it today or we leave.",
},
"questions": {
"team": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Charges, invoices and refunds",
"technical": "The product is not working",
"account": "Login, plan and access",
},
},
"needs_human": {
"type": "noul", # a statement; the answer is P(true)
"instructions": "A human agent should take over this conversation.",
},
"churn_risk": {
"type": "score",
"instructions": "How likely is this customer to leave?",
"criteria": ["low", "moderate", "high", "imminent"],
},
},
},
)
answers = response.json()["answers"]
answers["team"]["choice"] # "billing" (p = 0.86)
answers["needs_human"]["noul"] # P(true) = 0.68
answers["churn_risk"]["score"] # expected level on 0..3 = 2.48Build typed decisions into your workflow
Route support tickets, triage invoices, or score alerts with a model you deploy. Talk to an engineer about serving decision models on Chalk GPUs.