Search a parts catalog with custom embeddings on Chalk
This demo shows how to fine-tune an open embedding model on Chalk to match customer descriptions to catalog parts. Try the same search with the original and fine-tuned models, then compare where the right part appears. Both models are called directly from ChalkSQL.
Find the right part without knowing its name
Select a suggested search to run both models, or enter your own description and select Search. Look for the expected part in both lists: does fine-tuning move it closer to the top?
Base model: before fine-tuning
The original all-MiniLM-L6-v2 model, without training on this catalog's example searches.
Tuned model: after fine-tuning
The same model trained to connect customer descriptions with catalog parts. The suggested searches refer to products excluded from training.
Start with the hydraulic-cylinder example. The expected result is a check valve that holds the load when the pump stops. Compare its position in the two result lists, rather than comparing the decimal scores across models. Those numbers measure vector similarity; they are not percentages or confidence estimates.
Run one description through both models and find the expected part. ChalkSQL calls each embedding model; compare the part’s rank before and after fine-tuning.
Ironbark Supply, its 1,002 catalog items, and the example searches are fictional and generated for this demo. Each search runs two real ChalkSQL queries, one per model.
Recorded benchmark · Separate from the live search above
Tuning helps customer searches.
Customer descriptions → catalog parts
35% → 54%
Searches with the right part ranked first, before and after fine-tuning. Measured on 800 searches for 200 products excluded from training, ranked against the full catalog.
See all recorded customer-search metrics
| Metric | Before tuning | After tuning |
|---|---|---|
| Right part ranked first | 35% | 54% |
| Right part in the top 5 | 63% | 83% |
| Right part in the top 10 | 74% | 91% |
| Mean reciprocal rank (top 10) | 0.47 | 0.66 |
Mean reciprocal rank averages one divided by the correct part's rank. A part missing from the top ten contributes zero. Higher is better.
Built on Chalk
Chalk trains and serves the models, then runs the SQL that ranks the parts. The example supplies the fictional catalog, training pairs, and search interface.
Fine-tune on your examples
A Chalk dataset holds the catalog and example searches. A training function fine-tunes all-MiniLM-L6-v2 on an NVIDIA L4 and saves the weights to a Chalk volume.
Serve both models as functions
Chalk runs the original and fine-tuned models as separate CPU functions. Each takes text and returns a vector, so the application can compare them on the same search.
Search directly from SQL
ChalkSQL calls an embedding function with catalog_call, then ranks stored catalog vectors with cosine_similarity_vector. The same functions embed the catalog and each new search.
Run queries without app secrets
The search application uses its own Chalk workload identity. Chalk mounts and rotates its short-lived token, so the application needs no stored client secret to run its SQL.
Train a model, then call it from SQL
Install uv, the Chalk CLI, and the Google Cloud CLI (gcloud). You need an L4-capable Chalk environment, a writable Cloud Storage bucket that ChalkSQL can read, and your active query-engine deployment ID from the Chalk console.
Replace the three YOUR_ values in Quickstart. Keep training, functions, SQL, and the app in the same environment. The commands use the included data, create and upload the Parquet inputs, embed the catalog, and print the deployed app URL.
Wait for the function deployment and catalog refresh commands to finish before embedding the catalog. Use the same bucket path in the SQL tabs. The Python function tabs are source excerpts; Quickstart runs their complete scripts.
git clone https://github.com/chalk-ai/sql-custom-embeddings
cd sql-custom-embeddings
# Use the same environment for training, functions, SQL, and the app.
export CHALK_ENVIRONMENT_ID="YOUR_ENVIRONMENT_ID"
export DEMO_BUCKET="gs://YOUR_BUCKET/ironbark-demo"
export DEMO_DEPLOYMENT_ID="YOUR_ACTIVE_DEPLOYMENT_ID"
chalk login
chalk environment "$CHALK_ENVIRONMENT_ID"
# 1. Train on the included data, download the weights, and deploy both models.
uv run --no-project --python 3.12 --script train.py --env-id "$CHALK_ENVIRONMENT_ID"
uv run --no-project --python 3.12 --script fetch_model.py --env-id "$CHALK_ENVIRONMENT_ID"
uv run --no-project --python 3.12 --script deploy_functions.py --model-dir model/ --env-id "$CHALK_ENVIRONMENT_ID"
# 2. Refresh SQL's function catalog after both functions are ready.
chalk deployment activate --deployment "$DEMO_DEPLOYMENT_ID"
# 3. Convert the included JSONL inputs to Parquet and upload them.
uv run --no-project --python 3.12 --with pyarrow python - <<'PYARROW'
import pyarrow.json as json
import pyarrow.parquet as parquet
for output, source in [("catalog_text", "products"), ("supplier_feed", "supplier_feed")]:
table = json.read_json(f"data/{source}.jsonl")
if output == "supplier_feed":
table = table.rename_columns(["expected_sku" if name == "sku" else name for name in table.column_names])
parquet.write_table(table, f"data/{output}.parquet")
PYARROW
gcloud storage cp data/catalog_text.parquet data/supplier_feed.parquet "$DEMO_BUCKET/"
# 4. Embed the catalog with both deployed models, then upload the result.
uv run --no-project --python 3.12 --script build_catalog_vectors.py "$DEMO_BUCKET/catalog_text.parquet" data/catalog.parquet
gcloud storage cp data/catalog.parquet "$DEMO_BUCKET/catalog.parquet"
# 5. Deploy the search app; open the URL it prints.
uv run --no-project --python 3.12 --script deploy_app.py --catalog "$DEMO_BUCKET/catalog.parquet" --env-id "$CHALK_ENVIRONMENT_ID"Bring your catalog to Chalk
Train on your customers' language, serve your model as a function, and search your catalog from SQL. Talk to an engineer about your search or matching workload.