Replay agents and models against production data as it actually was, with one knowledge cutoff on every tool call. Score with functions or LLM judges.

Agent Evals: Test Agents on Real Production Data | Chalk

Accurate evals built on real data, for agents and models

database

Production data is the eval set

clock

One cutoff, whole trajectory

lock

Immutable definitions

gavel

Scorers & judges

listTree

Read the whole run

shieldCheck

Keep data in your cloud

Rich Pearce

Upload a dataset or point at a revision, name the task function and the scorers, and run it. Every tool the agent reaches is held to the same cutoff, so the run reflects a real moment rather than today's data.

Live Oct 8: Give Agents Context for Evals and Production

We built a Slack agent on Chalk Sandboxes that can reach Zerocater, Slack and OpenAI, and nothing else. Then we tried to make it leak. Network policies and auth injection held.

How Chalk Uses Chalk | MealBot

A workspace for agentic ML. Your agent investigates against the data your models actually see, and the notebook it leaves behind is the evidence you review before deciding what ships.

Introducing Chalk Notebooks: where your ML agent does its work

Chalk MCP Server connects AI agents directly to the Chalk primitives your team already uses and Chalk Assistant brings that agent inside the Chalk interface.

Introducing Chalk MCP Server and Chalk Assistant: Run Agentic Machine Learning on Chalk

Agents now run the modeling loop end to end. What decides whether you can trust it is the data layer underneath.

What is Agentic Machine Learning?

Why real-time, windowed context beats static retrieval for agents — illustrated with a poker example that becomes a fraud detector.

Context Has a Timestamp

Why agents fail on live decisions, and the system that fixes it.

An agent only knows what's in its prompt. A context engine computes the values that describe a customer's situation right now and assembles them at inference time, so the agent stops guessing at facts the business already has.

What is a Context Engine?

What Is a Feature Store in Machine Learning?

A feature store is more than a tool – it is a secret weapon that empowers teams to innovate faster, collaborate better, and deliver production grade machine learning.

What Is a Feature Store?

Build or Buy? Why Melio Picked Chalk

What we heard at Snowflake Summit, and what we built to answer it.

What Snowflake Summit Was Really About

Run agents against the exact data your production system would have served, in your cloud.

Introducing Chalk Compute: Time-Traveling Agent Sandboxes in Your Cloud

A hundred people, one new headquarters, and a lot of competitive mahjong. Chalk's San Francisco office is officially open.

Chalk celebrated the opening of its new San Francisco headquarters with 100+ engineers and builders from the Bay Area AI and ML infrastructure community.

Welcome Home, West Coast

We're celebrating our new office — and the timing couldn't be better. We've got a packed spring ahead, from Snowflake Summit to AI Council and more.

Chalk's Spring Fling is its most ambitious events season yet. From an SF office warming party to Bronze sponsorship at AI Council, a first-time presence at Snowflake Summit (booth #2611, June 1–4), and something in the works at Databricks Data & AI Summit, Chalk is showing up everywhere AI and data infrastructure is being built.

Chalk's Spring Fling: Where to Find Us This Season

Learn what data drift is in machine learning, how it differs from concept and model drift, and how teams detect and manage drift in production ML systems.

What Is Data Drift in Machine Learning (and How to Detect It)

Most feature stores are built around batch pipelines and materialized storage, which creates an inherent freshness ceiling. Tightening staleness settings cannot overcome architectural limits. This article explains why traditional feature stores struggle with real-time freshness and how a compute-first approach enables on-demand feature computation, lower latency, and improved correctness for time-sensitive ML use cases like fraud detection and underwriting.

Why Your Feature Store Has a Freshness Ceiling

How Turo Built a Self-Serve ML Feature Platform for Search and Pricing With Chalk

A roundup of Winter 2025’s latest Chalk product updates, covering performance improvements, new integrations, and infrastructure enhancements for running fast, reliable real time data pipelines in production.

Quarterly Product Update: Winter

Recap from ODSC's first SF meetup of 2026, where we did a deep dive into real-time feature computation for marketplace recommender systems.

Chalk x ODSC Meetup Recap 2026

MLOps enables teams to train and run models at scale. Learn why production ML is challenging, from feature inconsistency to fragmented data and how to solve it.

7 Common MLOps Challenges (and How to Solve Them)

Medely staffs critical healthcare in real-time with Chalk

This quarter, we expanded Chalk across three core dimensions: flexibility, visibility, and efficiency. Teams can now define and serve models alongside features, trace query behavior for diagnosis, and adapt feature logic dynamically without redeploying.

Quarterly Product Update: Fall

How Mission Lane manages thousands of ML features in production using Chalk. Demo recap covering feature reusability, live decisioning, and batch workflows. 

Mission Lane demo: Scaling ML with thousands of features

Powering Reliable, Explainable Data for Credit Decisioning at iwoca with Chalk

Chalk named to Fast Company's Next Big Things in Tech 2025 for helping AI teams get fresh data to their models in real-time.

Chalk Named to Fast Company's Next Big Things in Tech 2025

Chalk runs inside your VPC on Kubernetes so sensitive data stays resident, latency drops, and ML teams move faster. Hear the full story on the podcast.

Deploy where your data lives: Why Chalk runs in your cloud

Your notebook code is production code with Chalk. Deploy directly from notebooks and experiment safely with branches.

Chalk for Data Scientists

MoneyLion delivers AI-powered personal finance products with Chalk

CEO Marc on The Information TV unpacks why winning at AI isn't about bigger models but smarter infrastructure that keeps data and compute in your cloud.

CEO Marc on The Information TV

Our CEO Marc Freed-Finnegan joins This Week in Startups (TWiST) to explain why AI needs fresh data fast. Learn how Chalk eliminates the speed vs. freshness tradeoff.

Chalk on TWiST500

 Data engineers waste weeks managing ML pipelines. Learn how to escape backfill nightmares and tool sprawl with declarative features on Chalk.

Chalk for Data Engineers

How Chalk fuels Whatnot’s live shopping platform

The Chalk team went to SciPy 2025 in Tacoma and came back inspired. Here's our recap of all the advances we saw in Python's data ecosystem.

SciPy 2025 Recap

This quarter, we delivered platform-level improvements that extend Chalk’s ability to support complex, production-grade ML systems.

Quarterly Product Update: Summer

Verisoul shows how to catch sophisticated fraudsters by using Chalk to combine ML features with LLM intelligence. See how they rapidly iterated through three different techniques in under an hour.

Verisoul Demo Recap

Discover the critical differences between feature stores and feature engines. Learn when to use each approach and how feature engines enable real-time computation and rapid experimentation for ML teams.

Feature store vs. Feature engine

Mission Lane’s Source of Truth for Credit Decisioning with Chalk

Learn how AI engineers use Chalk to eliminate stale context, reduce tooling complexity, and ship AI features that work consistently in production.

Chalk for AI Engineers

What is MLOps? Learn the definition, meaning, and benefits of MLOps, plus key components, challenges, and best practices for deploying and maintaining ML models.

What is MLOps? Definition, Challenges, and Best Practices

Insights from Data Council 2025 on the shift toward real-time inference, declarative data architectures, Python-first interfaces, and simplified data stacks—plus a look at the community shaping the future of data.

What Data Council 2025 taught us about the future of data

We put Chalk’s native prompt evaluation framework to the test by benchmarking top models on a custom Christopher Nolan trivia dataset—showcasing how teams can evaluate and deploy LLM workflows with production-grade rigor.

Which LLM Wins at Nolan Trivia? Chalk’s Prompt Evaluation in Production

I'm incredibly excited to share that Chalk has raised a $50 million Series A at a $500 million valuation, led by Felicis, with participation from Triatomic Capital and our existing investors General Catalyst, Unusual Ventures, and Xfund. This funding marks a huge milestone and a powerful validation of our vision for real-time AI infrastructure.

Chalk Raises $50M Series A to Power AI Inference

Chalk's team recaps their April 2025 tour across major AI conferences, sharing what they learnt on real-time ML infrastructure and LLM integration.

Chalk on the Move: April Events Recap

This quarter’s updates help ML teams move faster in production with better performance, observability, and flexibility.

Quarterly Product Update: Spring

To eliminate Python’s performance bottlenecks in real-time ML, Chalk built a Symbolic Python Interpreter that converts flexible Python resolver code into optimized Velox-native expressions. This post dives into the technical journey and impact of enabling fast, parallel execution without sacrificing developer ergonomics.

Accelerating Real-Time ML with Symbolic Python Interpreter

How Apartment List uses Chalk to power personalized apartment recommendations

Verisoul stops fake accounts with Chalk’s real‑time feature platform

A roundup of Chalk's latest product releases, feature updates, and new documentation for the end of the year 2024.

Vital predicts hospital wait times with Chalk’s feature platform

A roundup of Fall 2024’s latest product releases, feature updates, and new documentation and tutorials.

Chalk and Elliot Marx won Best Technology at The GenAI Collective's Demo Day by showing how Chalk's feature store can be used to unify traditional machine learning features with generative AI for parsing unstructured data.

Chalk wins Best Technology at The GenAI Collective's Demo Day

A deep dive into how we built support for PartiQL and DynamoDB on Chalk by using DuckDB's SQL parser.

Deep Dive: Building PartiQL Support at Chalk

New product releases and changelog updates from Chalk for summer 2024.

A Tutorial on Fraud and Risk

Machine learning has long been deeply embedded in the field of fraud detection. A deep dive into how feature stores empower ML teams to build fraud models.

Feature Store at Work

Today we’re thrilled to announce Chalk's expansion with the opening of a New York City office.

Introducing Chalk NY

Today we're proud to announce CB Insights recognized Chalk alongside Databricks in its 2024 AI 100 as one of the leading ML / AI Development Platforms globally.

Chalk and Databricks selected as leading ML / AI Platforms in 2024 CB Insights AI 100

We’re proud to announce we’ve raised $10 million in seed funding led by General Catalyst, Unusual Ventures, and Xfund to build the data platform for machine learning and generative AI.

Announcing our $10M seed to accelerate machine learning

Chalk is proud to announce the availability of our SOC 2 Type 1 Report.

Chalk Completes SOC 2 Type 1 Report

Serve production features at inference time with single-digit millisecond latency. 

Model Training

Governance & Observability

Functions

Notebooks

Agent Evals

MCP Gateway

Sandboxes

Model Gateway

Model Serving

Federated Queries

Chalk for AI Platform Engineers

Chalk For Startups

Data Engineers

MLOps

Data Scientists

AI/ML

Trust & Safety Agents

Customer support agents

Workflow Automation Agents

Serve pricing models fresh demand, supply, and behavioral features at inference-time.

Dynamic Pricing

Deploy Agents in Sandboxes

Replay agents against the state your systems actually served.

Evaluate Agent Workflows

Evaluate & Improve Agents Workflow

Train and fine-tune open models on your proprietary data with Chalk.

Fine-tune Models on Your Data

Fine-tune Models

Reduce the cost of model calls and maintain data sovereignty. 


Serve Open Models

Open Model Serving

Recursive self-improvement loops for the ML lifecycle.

Autoresearch Agents

ML Autoresearch Agent

Adapt results instantly based on user context, behavior, and intent, while maintaining full control over ranking logic and data.


Agent Evals

Accurate evals built on real data, for agents and models

Explore Chalk Evaluations

Production data is the eval set

One cutoff, whole trajectory

Immutable definitions

Scorers & judges

Read the whole run

Keep data in your cloud

Compared to some problems, we make relatively few, very high-value decisions. Throughput isn't the problem. Trust is.

How Agent Evals work

Pin evaluation to a moment

Score with functions or judges

Compare runs, then open one

Replay the world as it was

Keep up with Chalk

Most Innovative Companies

Register for the upcoming webinar

Raising the Talent Bar

Introducing Chalk Notebooks

Know which agent change is shippable