Live, or on a break?
Ask the video.
A vision pipeline on a Chalk GPU watches a livestream four times a second and plots how confident it is that the streamer is at the desk and engaged. Breaks, empty chairs and be-right-back screens show up as the line drops.
Play the stream
Press play and watch the confidence line follow the streamer: talking, gaming, stepping away, a be-right-back screen, and a stretch scrolling a phone. The top strip is the script the recording was made from; the second is what the pipeline decided. Click the plot to jump, or analyze a short clip of your own.
The streamer is fictional: generated with LTX-Video and voiced with Kokoro TTS.
How it works
No single model knows what a break is. An empty chair, a silent streamer and a be-right-back card each look different, so the pipeline listens to several weak signals and lets a small, explainable model weigh them. On footage it was never fit on, its verdict matches the script 94% of the time.
Look and listen
Four times a second, an object detector checks for a person, CLIP compares the frame with descriptions of a streamer at the desk and of an empty chair or a be-right-back card, and a voice-activity model scores the audio.
Fuse
Presence signals are averaged over two seconds; speech is summarized as whether the streamer spoke in the last six, which is what separates scrolling a phone from a pause between sentences. A five-weight logistic model, fit on a separate labeled recording, turns them into one probability.
Decide
An exponential moving average and two thresholds keep the verdict from flickering: it switches to a break below 0.4 and back to active above 0.6, and ignores blips shorter than two seconds.
Deploy your own in three commands
Clone the example, log in, and run the deploy script. Chalk builds the image with the model weights baked in, provisions an L4, and the replica analyzes the sample stream as it boots. You get a URL that serves this exact demo.
The replica also exposes a JSON API: post a clip, get back the per-sample signals, the fused probability and the detected segments. Point it at stream recordings, moderation queues or highlight detection.
chalk-ai/activity-detectiongit clone https://github.com/chalk-ai/activity-detection
cd activity-detection
# Or export CHALK_CLIENT_ID and CHALK_CLIENT_SECRET.
chalk login
# Builds the image, provisions an L4, analyzes the sample stream, prints the URL.
./deploy.py --gpu nvidia-l4What Chalk handles
The models are the interesting part. Everything around them is the part Chalk does for you.
A GPU for the vision models
Ask for gpu="nvidia-l4" and Chalk provisions the card, schedules the replica, and gives it an HTTPS URL.
Faster than real time
The two-minute sample stream is scored in about 13 seconds on one L4: every quarter-second gets a person score, a CLIP score and a speech score.
Weights baked into the image
The detector and CLIP weights download during the image build, so a cold replica loads them from disk and never waits on a model hub.
Clips stay private
Uploaded clips are analyzed in memory on the replica and deleted before the response returns. Nothing goes to a third-party vision API.
Explainable by design
Every verdict comes with the four signals behind it, so a moderator can see whether the call came from an empty chair, silence or a be-right-back card.
Four files, open source
The pipeline, the server, the page and the deploy script are Apache-2.0 on GitHub. Swap in your own signals or weights.
Understand video on your own GPUs
Moderation, engagement, highlights or safety. Talk to an engineer about running video models in your cloud.