Streaming windowed aggregations.
From Kafka.
This live demo streams card transactions from Kafka into Chalk windowed features and serves them from the online store as they change. It is for fraud, risk and ML engineers who need streaming signals like distinct cards per merchant without building and running a stream-processing pipeline. You will trigger a card-testing attack or an account takeover and watch distinct counts, top countries, spend and p90 amounts update across 1-minute, 5-minute and 1-hour windows, read every two seconds.
Connect an event burst to changing feature windows
The stream shows transactions; the feature values are read from the online store every two seconds.
Try this experiment · expected results
1Start a card-testing burst
Choose Card-testing attack and watch Tessel Electronics.
Expected: Distinct-card counts and top countries change in the merchant windows.
2Compare the account-takeover scenario
Start an account takeover and inspect that cardholder’s merchants and countries.
Expected: The same event stream updates different entity windows; short and long windows summarize different histories.
Compare an event burst with the selected merchant’s or cardholder’s rolling counts. Chalk updates the online features as events arrive from Kafka.
Built on Chalk
You write the aggregation once. The consumers, buckets, sketches and window reads are Chalk's job.
Sketches that merge
approx_count_distinct, approx_top_k and approx_percentile keep a small sketch per bucket, and windows combine sketches instead of storing every value.
One definition, many windows
windowed("1m", "5m", "1h") declares all three windows at once, and bucket_durations decides which bucket size backs each window.
Per entity, both sides
The same stream feeds cardholder windows and merchant windows. Joining a row to two entities is a type annotation, not a second pipeline.
Built for fraud signals
Distinct cards per merchant and distinct merchants per cardholder are the classic card-testing and account-takeover signals, ready at query time.
Declare the windows, not the pipeline
The whole demo is two files in a Chalk project: features that declare the windows and how they are bucketed, and a stream resolver that maps a Kafka message onto a transaction. Chalk runs the consumers, maintains the buckets, and answers window queries from the online store.
The same feature definitions answer offline queries too, so a model trained on these windows sees the values it would have seen at each point in time.
from chalk import DataFrame, FeatureTime, Windowed, _, windowed
from chalk.features import feature, features
# 10s buckets roll up into the 1m and 5m windows; 1m buckets roll up into the hour.
BUCKETS = {"bucket_durations": {"10s": ["1m", "5m"], "1m": "1h"}}
WINDOWS = ("1m", "5m", "1h")
@features
class CardTransaction:
id: str
cardholder_id: "Cardholder.id"
merchant_id: "Merchant.id"
card_id: str
amount: float
country: str
category: str
ts: FeatureTime
@features
class Merchant:
id: str
transactions: DataFrame[CardTransaction] = feature(max_staleness="1h")
txn_count: Windowed[int] = windowed(
*WINDOWS, materialization=BUCKETS, expression=_.transactions.count()
)
distinct_cards: Windowed[int] = windowed(
*WINDOWS, materialization=BUCKETS,
expression=_.transactions[_.card_id].approx_count_distinct(),
)
top_countries: Windowed[list[str]] = windowed(
*WINDOWS, materialization=BUCKETS,
expression=_.transactions[_.country].approx_top_k(k=3),
)
p90_amount: Windowed[float] = windowed(
*WINDOWS, materialization=BUCKETS,
expression=_.transactions[_.amount].approx_percentile(quantile=0.9),
)Implementation details
How the example works
Produce
A fictional card network writes about three transactions a second to a Kafka topic, plus the bursts you trigger: a card-testing attack on one merchant or an account takeover of one cardholder.
Resolve
A native Chalk stream resolver consumes the topic and projects each JSON message onto a CardTransaction row, keyed to its cardholder and its merchant, with no Python in the loop.
Materialize
Every windowed feature folds the row into a 10-second bucket per cardholder and per merchant: counts and sums, plus mergeable sketches for distinct counts, top-k and percentiles.
Serve
An online query for a 1-minute, 5-minute or 1-hour window merges the buckets under it, so the answer never rescans raw transactions. The demo reads six entities every two seconds.
More capabilities
Native or Python
This resolver is declared as expressions and runs natively. When a message needs real logic, a Python @stream resolver plugs into the same windows.
Runs in your cloud
The stream consumers, the online store and the backend for this page all run in a Chalk environment, next to your Kafka and your data.
Real-time features from your streams
Kafka, Kinesis or Pub/Sub in, windowed features out. Talk to an engineer about your fraud, risk or personalization signals.