Our CEO Marc joined The Information TV to unpack why winning at AI isn't about bigger models — it's about smarter infrastructure.
The infrastructure reality check
Host Akash Pasricha cut straight to the chase — AI costs started falling, then suddenly plateaued. What gives?
Marc explained that while chips keep getting more powerful, AI workloads are scaling even faster with larger models, more parameters, and bigger context windows. The demand is simply outpacing the hardware.
His key insight: "Costs don't fall just because models improve. Costs fall when you get more efficient at what you're doing and when you learn how to use infrastructure in the right way."
What's particularly interesting is that while everyone obsesses over training costs, the real money drain has shifted to inference — running these models again and again for actual users.
Managing inference costs
Marc shared practical approaches to managing costs, such as rightsizing your models (since not everything requires GPT-4 class reasoning) and caching smartly when you can tolerate values calculated in the past.
But here's where Chalk shines: getting fresh data to models at inference time. He shared how real-time inference transforms e-commerce — new users get relevant suggestions from their first few clicks, not generic results from the previous night's batch job.
As Marc put it: "Faster computation, on-demand computation, fresher data creates less fraud, more revenue, better user experiences."
The data ownership debate
Akash wrapped with a big question about the "corporate data wars" — who really owns the data when AI agents want to pull from every app in your stack?
Marc's stance was clear: Enterprises need to own their own data. That's why Chalk deploys directly into customer environments where their data never leaves their cloud!
Watch the full conversation
There's plenty more we didn't cover, from why Chalk competes with in-house solutions (not Databricks), to how on-demand computation changes the cost equation.
Watch the full episode on The Information TV to hear Marc explain why faster, fresher data is the difference between delighting users and losing them to competitors.
P.S. Congrats to Marc on becoming a dad! His "no screens for a few years" policy might be the most ambitious goal he shared.
