Every swipe, counted
before the next one lands.
Card transactions stream from Kafka into Chalk windowed features: distinct cards, top countries, spend and p90 amounts per merchant and per cardholder, materialized into 10-second buckets and read back from the online store as they change.
Start an attack, watch the windows move
The dots are transactions on the Kafka topic, one row per merchant. The numbers below are read from the Chalk online store every two seconds. Press Card-testing attack and watch distinct cards and top countries jump at Tessel Electronics, or start an account takeover and watch one cardholder's distinct merchants and countries climb. Everything here is fictional.
How it works
Recomputing “distinct cards at this merchant in the last five minutes” from raw events on every request does not scale. Materialized aggregations do the work once, as events arrive, and keep it in buckets that any window can reuse.
Produce
A fictional card network writes about three transactions a second to a Kafka topic, plus the bursts you trigger: a card-testing attack on one merchant or an account takeover of one cardholder.
Resolve
A native Chalk stream resolver consumes the topic and projects each JSON message onto a CardTransaction row, keyed to its cardholder and its merchant, with no Python in the loop.
Materialize
Every windowed feature folds the row into a 10-second bucket per cardholder and per merchant: counts and sums, plus mergeable sketches for distinct counts, top-k and percentiles.
Serve
An online query for a 1-minute, 5-minute or 1-hour window merges the buckets under it, so the answer never rescans raw transactions. The demo reads six entities every two seconds.
Declare the windows, not the pipeline
The whole demo is two files in a Chalk project: features that declare the windows and how they are bucketed, and a stream resolver that maps a Kafka message onto a transaction. Chalk runs the consumers, maintains the buckets, and answers window queries from the online store.
The same feature definitions answer offline queries too, so a model trained on these windows sees the values it would have seen at each point in time.
Windowed aggregations in the docsfrom chalk import DataFrame, FeatureTime, Windowed, _, windowed
from chalk.features import feature, features
# 10s buckets roll up into the 1m and 5m windows; 1m buckets roll up into the hour.
BUCKETS = {"bucket_durations": {"10s": ["1m", "5m"], "1m": "1h"}}
WINDOWS = ("1m", "5m", "1h")
@features
class CardTransaction:
id: str
cardholder_id: "Cardholder.id"
merchant_id: "Merchant.id"
card_id: str
amount: float
country: str
category: str
ts: FeatureTime
@features
class Merchant:
id: str
transactions: DataFrame[CardTransaction] = feature(max_staleness="1h")
txn_count: Windowed[int] = windowed(
*WINDOWS, materialization=BUCKETS, expression=_.transactions.count()
)
distinct_cards: Windowed[int] = windowed(
*WINDOWS, materialization=BUCKETS,
expression=_.transactions[_.card_id].approx_count_distinct(),
)
top_countries: Windowed[list[str]] = windowed(
*WINDOWS, materialization=BUCKETS,
expression=_.transactions[_.country].approx_top_k(k=3),
)
p90_amount: Windowed[float] = windowed(
*WINDOWS, materialization=BUCKETS,
expression=_.transactions[_.amount].approx_percentile(quantile=0.9),
)What Chalk handles
You write the aggregation once. The consumers, buckets, sketches and window reads are Chalk's job.
Sketches that merge
approx_count_distinct, approx_top_k and approx_percentile keep a small sketch per bucket, and windows combine sketches instead of storing every value.
One definition, many windows
windowed("1m", "5m", "1h") declares all three windows at once, and bucket_durations decides which bucket size backs each window.
Per entity, both sides
The same stream feeds cardholder windows and merchant windows. Joining a row to two entities is a type annotation, not a second pipeline.
Built for fraud signals
Distinct cards per merchant and distinct merchants per cardholder are the classic card-testing and account-takeover signals, ready at query time.
Native or Python
This resolver is declared as expressions and runs natively. When a message needs real logic, a Python @stream resolver plugs into the same windows.
Runs in your cloud
The stream consumers, the online store and the backend for this page all run in a Chalk environment, next to your Kafka and your data.
Real-time features from your streams
Kafka, Kinesis or Pub/Sub in, windowed features out. Talk to an engineer about your fraud, risk or personalization signals.