Recorded run · Data agents

Answer business questions with ChalkSQL and agent sandboxes

This demo shows a recorded data agent looking up business definitions in documents and tickets, then querying the billing data with ChalkSQL. Inspect the sources and final query to see why definitions matter as much as the numbers.

Check the definitions behind the answer

Step through the recorded run and open its tool calls. Look for the product, account, and revenue rules found in documents and tickets, then check that the final ChalkSQL query applies them. This is a replay, not a new live agent run.

data agent · chalk sandbox
Question

What is our revenue per product for enterprise customers split by whether or not they're "AI native"?

Sandbox egress policy
$NOTION_TOKEN in the sandbox__chalk-secr... (placeholder)
GET https://api.github.comblocked (ConnectError)
GET https://pypi.org/simple/blocked (ConnectError)
GET <notion>/v1/pages/x (no token)401
Tool calls16 / 16
Recorded answer

Net revenue for the 28 real enterprise accounts, by product, split by AI-native status. Forge is the largest line for AI-native accounts ($3.60M vs $1.35M for the others). Services is shown separately and is not a product.

AI-nativeNot AI-native
Forge
$3.60M
$1.35M
Pulse
$889K
$1.14M
Relay
$592K
$744K
Vault
$694K
$969K
Services
$115K
$134K
Definitions it applied
  • Revenue is net: charges and credits summed (Notion: Revenue reporting conventions, rule 1).
  • Excluded customers whose domain ends in .internal (2 enterprise test/demo accounts) (Revenue reporting conventions, rule 2).
  • Enterprise = segment column on customers table, not re-derived from ARR (Revenue reporting conventions, rule 4).
  • SKU to product mapping from Notion Product catalog; SUP-PREM and ONB-PRO are rolled into a separate Services line, not a product (Revenue reporting conventions rule 3, Product catalog).
  • STRM-LEGACY counted as Relay for all revenue reporting (Linear BILL-118, overriding the catalog's 'retired' listing).
  • AI-native flags come from the Notion Enterprise account review database (source of truth per the 'Definition: AI-native customer' page): 10 accounts marked Yes.
  • Three accounts marked 'Not reviewed' were resolved from Linear GTM-214: Helix Bio = AI-native, Nimbus Health = AI-native, Granite Logistics = not AI-native.
  • No date filter applied, so all invoice history is included (fiscal year = calendar year, but no period was requested).
Answered in 25s with 16 tool calls
The final ChalkSQL query
WITH c AS (
  SELECT customer_id, CASE WHEN domain IN ('latticelabs.ai','synthex.ai','corticalsystems.com','mosaicvision.io','parrotspeech.ai','quill.law','inferenceworks.dev','heliorobotics.com','embersearch.ai','cobaltsec.com','helixbio.com','nimbushealth.com') THEN 1 ELSE 0 END ai
  FROM 'customers.parquet'
  WHERE segment='enterprise' AND domain NOT LIKE '%.internal'
), l AS (
  SELECT CASE WHEN sku IN ('PUL-STD','PUL-ENT') THEN 'Pulse' WHEN sku IN ('VLT-STORE','VLT-BACKUP') THEN 'Vault' WHEN sku IN ('RLY-MSG','RLY-CONN','STRM-LEGACY') THEN 'Relay' WHEN sku IN ('FRG-GPU','FRG-SEAT') THEN 'Forge' WHEN sku IN ('SUP-PREM','ONB-PRO') THEN 'Services' ELSE 'UNMAPPED' END product, customer_id, amount_usd
  FROM 'invoice_lines.parquet')
SELECT l.product, round(sum(CASE WHEN c.ai=1 THEN l.amount_usd ELSE 0 END),2) ai_native_usd, round(sum(CASE WHEN c.ai=0 THEN l.amount_usd ELSE 0 END),2) not_ai_native_usd
FROM l JOIN c ON l.customer_id=c.customer_id GROUP BY l.product ORDER BY l.product
What Notion and Linear saw
+0.0snotionGET /v1/pages/x401
+2.7snotionPOST /v1/search200
+2.7snotionPOST /v1/search200
+2.7snotionPOST /v1/search200
+2.7snotionPOST /v1/search200
+2.7slinearGET /v1/issues?query=AI+native200
+5.2snotionGET /v1/pages/a41f6c0e-ai-native-definition200
+5.2snotionGET /v1/pages/3b8d04f1-revenue-conventions200
+5.2snotionGET /v1/pages/7c1e9a52-catalog200
+5.3snotionPOST /v1/databases/d6c3b8e2-enterprise-review/query200
+5.3slinearGET /v1/issues/GTM-214200
+5.3snotionGET /v1/pages/e90b2d77-q3-offsite200
+7.9slinearGET /v1/issues/BILL-118200

Fieldstone, its accounts, documents and tickets are fictional. The Notion and Linear APIs here are small stand-ins served from fixtures; the sandbox, the egress policy, ChalkSQL and the model router are real.

Built on Chalk

Chalk supplies the sandbox, network policy, SQL engine, and model endpoint. The example supplies the agent loop, fixture-backed Notion and Linear services, and reference answer.

  • Keep service credentials outside the agent

    The sandbox receives placeholder credentials. The network proxy substitutes real credentials only on requests to their configured hosts.

  • Restrict the reachable services

    The example’s network policy allows three service hosts. Other destinations, including package indexes and paste sites, cannot be reached from the sandbox.

  • Query data with ChalkSQL

    The agent queries billing tables in object storage with joins, CTEs, and CASE expressions. Its final SQL encodes the business rules it found.

  • Use the model router

    The agent calls an OpenAI-compatible endpoint on Chalk’s model router. The example records tool arguments, results, citations, and an independently computed reference answer.

Deploy your own

Build it in three files

agent.py is a plain tool-calling loop with seven tools. run.py starts it in a sandbox with the access policy. deploy_workspace.py stands up the fake Notion and Linear.

To point it at your own workspace, swap two base URLs and two tokens. The tools use a handful of read endpoints, and the sandbox policy stays exactly the same.

chalk-ai/data-agent
git clone https://github.com/chalk-ai/data-agent
cd data-agent

# Build the demo warehouse and put it where your environment's ChalkSQL can read it.
./data/generate.py --truth
gcloud storage cp data/out/*.parquet gs://<your-bucket>/data-agent/

# Deploy the fake Notion and Linear, each with its own token.
./deploy_workspace.py

# Ask a question.
./run.py --data-prefix gs://<your-bucket>/data-agent

Build agents that know your business

Analytics agents, support agents, or anything that needs your data and your definitions. Talk to an engineer about running them next to both.