Write features.
Chalk plans.
This live demo lets you plan a lending decision as one query, on Chalk, with features that are written five different ways: SQL over BigQuery, Python calling a credit bureau API, underscore expressions, a remote function and a registered scikit-learn model.
Follow one decision through its feature dependencies
The recorded query connects SQL, Python, expressions, a function, and a model in one graph.
Try this experiment · expected results
1Inspect the final decision
Start with the selected applicant. Read the decision and click a graph node to inspect its source and value.
Expected: Each dependency shows how it contributes to the final result.
2Compare another applicant
Choose the mismatched-bureau example, then inspect the name-match node and the query timeline.
Expected: The example’s identity-review rule and the overlapping source work are visible in the recorded evidence.
One graph, five ways to write a feature
Click a node to see its source. Edges point from a feature to the features computed from it.
decision: str = F.if_then_else(
_.name_match_score < 0.6,
"review_identity",
F.if_then_else(
(_.default_probability > 0.45) | (_.debt_to_income > 0.6),
"decline",
F.if_then_else(_.default_probability > 0.25, "review", "approve"),
),
)- decision
- approve
Timing: derived: runs inside a native operator
Underscore expression, planned natively · computed for this query
What ran, when, from the query's trace
Chalk planned every feature above as one query, then ran independent branches at the same time: applicant.chalk.sql, payments.chalk.sql, bureau_report, name_match_score overlapped. Python, function and model bars are measured trace spans. BigQuery scans run as native operators with no span of their own, so their bars are inferred from the spans around them; expressions run inside native operators in well under a millisecond.
No planning span: Chalk reused the plan it compiled for an earlier query of this shape.
Cache: every value was computed for this query. The demo runs on a branch, which has no online store; in production, max_staleness serves features like the bureau report from the online store instead.
The plan Chalk built
Condensed from Chalk's explain output for this query. SQL, Python, expressions, the remote function and the model sit in one plan, and each BigQuery scan asks only for the columns the graph uses.
- ValuesPersister
- UnderscoreResolverOperator<['applicant.decision']>
- NaryMergeJoinOperator
- NaryMergeJoinOperator
- UnderscoreResolverOperator<['applicant.name_match_score']>
- NaryMergeJoinOperator
- GivensScan
- BigQueryNativeDataframeResolver<'applicant', ['BQ_CHALK_DEVELOP_TESTING']> → SELECT full_name, id, monthly_debt, requested_amount, stated_annual_income
- ScalarResolver<['lending.resolvers.bureau_report']>
- UnderscoreResolverOperator<['applicant.debt_to_income']>
- DataFrameResolver<'saltmarsh_default_risk__applicant_default_probability'>
- NaryMergeJoinOperator
- NaryMergeJoinOperator
- UnderscoreResolverOperator<['applicant.loan_to_income', 'applicant.late_payments__31536000__', 'applicant.payments_due__31536000__']>
- NaryMergeJoinOperator
- BigQueryNativeDataframeResolver<'payments', ['BQ_CHALK_DEVELOP_TESTING']> → SELECT applicant_id, days_late, due_at, id
- HasManyInputOperator
- BigQueryNativeDataframeResolver<'payments', ['BQ_CHALK_DEVELOP_TESTING']> → SELECT applicant_id, due_at, id
Built on Chalk
The parts of a decision stop being separate services that you stitch together.
Mix styles freely
A SQL feature can feed an expression that feeds a model. No style is a separate system with its own pipeline, cache or deployment.
Pushdown into the warehouse
The SQL files have no WHERE clause. Chalk adds the requested ids and each window's filters to the BigQuery scan, and selects only the columns the graph uses.
Windows at decision time
Late payments in the last 90 and 365 days are counted over the has-many history when the decision is made, not in a nightly batch.
Functions for custom code
The name matcher is plain Python with a pip dependency, served by Chalk compute and called from an expression with catalog_call.
Each style, in a few lines
Every tab is taken from the project behind this demo. Put the SQL files and Python next to your feature classes, deploy the function and register the model, then chalk apply.
In the recorded runs on this page, Chalk executes the whole graph in two to five seconds, most of it the BigQuery scans; the function and the model each take tens of milliseconds.
-- applicant.chalk.sql: a SQL file resolver over the warehouse.
-- Chalk pushes the requested ids, and only the columns the query uses, into BigQuery.
-- source: BQ_CHALK_DEVELOP_TESTING
-- resolves: Applicant
SELECT id, full_name, stated_annual_income, monthly_debt, requested_amount
FROM demo_feature_graph.applicants
-- payments.chalk.sql: repayment history, joined to Applicant with has_many.
-- source: BQ_CHALK_DEVELOP_TESTING
-- resolves: Payment
SELECT id, applicant_id, due_at, amount, days_late
FROM demo_feature_graph.paymentsImplementation details
How the example works
Write each feature in its best language
Warehouse data as SQL files, an API call in Python, aggregations and ratios as expressions, custom code as a function, a score from a registered model.
Chalk plans them as one query
A request for applicant.decision pulls in every feature beneath it. Chalk resolves the graph into a single plan, pushing filters and columns down into BigQuery.
Independent work runs in parallel
The warehouse scans and the bureau call start together. The function waits for both names, the model for all its inputs, and the decision for the model.
More capabilities
Models are features
The default-risk model is registered once and referenced by version. Chalk computes its seven inputs and the score in the same query.
Traced end to end
Every run records a trace. The demo reads it back to show which resolver ran when, which ran in parallel and where each value came from.
Bring every feature into one query
Talk to an engineer about building decisions from SQL, Python, functions and models in one feature graph.