Serve embeddings and rerankers on Chalk GPUs
This demo shows how to run ModernBERT embedding and reranking models on Chalk GPUs to search the Chalk docs. Ask a question to try it, then explore the code to see how it works.
Compare retrieval order before and after reranking
Both columns start from the same candidates. The reranker changes their order for the question.
Try this experiment · expected results
1Search a sample question
Choose the backfill example or another preset and select Search.
Expected: The left column shows embedding retrieval; the right shows the reranked candidates.
2Inspect the movement
Compare the same passage across both columns and read the passages near the top.
Expected: Rank movement shows the reranker’s effect. Its score uses a different scale from cosine similarity.
Run one search and compare the two result orders. Chalk serves both the embedding model that retrieves candidates and the reranker that reorders them.
Built on Chalk
Chalk deploys the embedding model, reranker, and search app as separate services. The app retrieves 50 candidate passages, then asks the reranker to score each one against the question.
Run existing model containers
Chalk runs Hugging Face’s text-embeddings-inference image for both models. The Python deployment specifies the model, GPU, and readiness probe for each service.
Replicas behind one endpoint
The reranker uses two replicas, each with an NVIDIA L4, behind one HTTPS URL. Chalk distributes requests across the replicas.
Authenticated model calls
Both model endpoints require authentication. The search app uses its Chalk workload identity to call them and reads the rotating token for each request.
A search app alongside the models
Chalk also runs the CPU web app. Its code chunks the documents, builds the vector index, and shows the retrieved and reranked results side by side.
Run it on your own
Clone the example, log in to Chalk, and run the deploy script. It starts one L4 embedder, two L4 reranker replicas, and the CPU search app, then prints the demo URL.
You will need uv, the Chalk CLI, and a Chalk environment with capacity for three L4 GPUs. Point the app at your own documents, or call the reranker endpoint from your existing search pipeline.
git clone https://github.com/chalk-ai/embeddings-reranker
cd embeddings-reranker
# Or export CHALK_CLIENT_ID and CHALK_CLIENT_SECRET.
chalk login
# Deploys the embedder, a reranker on two L4s, and the search app.
./deploy.py --reranker-gpus 2Put the right passages first
Build search and retrieval with embedding and reranking models you deploy. Talk to an engineer about running them on Chalk GPUs.