Retrieve fast.
Rank it right.
Ask the Chalk docs a question. An embedding model finds 50 candidates in milliseconds, then a cross-encoder on two L4 GPUs reads each one against your question and puts the best answers first. Both models run on GPUs that Chalk deploys from one Python file.
Ask the docs
Pick an example or type your own question, then press Search. The left column is what embeddings alone return. The right column is the same candidates after reranking, with how far each one moved.
How it works
Embeddings are cheap enough to compare against everything, but they judge relevance from two separate summaries. A reranker is too slow to run on everything, but it reads the question and passage together. Use the first to narrow, and the second to decide.
Embed
When the app starts, gte-modernbert-base turns every section of the Chalk docs, about 1,800 passages, into a vector. Your question goes through the same model at search time.
Retrieve
Cosine similarity against all 1,800 vectors takes well under a millisecond and picks the 50 closest passages. Fast and broad, but it only compares two summaries.
Rerank
A cross-encoder reads your question and each candidate together, word by word, and scores how well the passage answers it. The top 10 by that score are what you see.
Deploy your own in three commands
Clone the example, log in, and run the deploy script. Chalk starts the embedder and the two-GPU reranker side by side, then the search app, and prints a URL that serves this exact demo. The whole stack is up in about five minutes.
The reranker is useful on its own: call it from a feature resolver, a RAG pipeline, or an agent tool to put the right context first.
chalk-ai/embeddings-rerankergit clone https://github.com/chalk-ai/embeddings-reranker
cd embeddings-reranker
# Or export CHALK_CLIENT_ID and CHALK_CLIENT_SECRET.
chalk login
# Deploys the embedder, a reranker on two L4s, and the search app.
./deploy.py --reranker-gpus 2What Chalk handles
The models are the interesting part. Everything around them is the part Chalk does for you.
Two GPUs, one URL
The reranker is a scaling group with two L4 replicas behind a single HTTPS endpoint, and requests are balanced across both. Change min_replicas to add more.
Measured, not promised
One search takes about 150 ms end to end. Under load the pair of L4s sustains about 24 searches a second, roughly 1,200 reranked passages.
Private model endpoints
Both model servers require a Chalk token. The search app reaches them with its own rotating workload identity, so no API keys live in code.
Any container
The model servers are the stock text-embeddings-inference image from GitHub's registry. Chalk runs it as-is, with no Dockerfile and no image of your own to push.
Runs in your cloud
Every replica runs in your Chalk environment, next to your data. Queries and documents never pass through a third-party API.
Five files, open source
The search app, page, deploy script, and a load-testing client are Apache-2.0 on GitHub. Point it at your own documents and redeploy.
Put the right context first
Search, recommendations, or retrieval for your agents. Talk to an engineer about serving embedding and reranking models on GPUs in your cloud.