Summary
Agents are being integrated more deeply into products. They are moving from adjacent tools that people consult into the workflows that run the product, where they are asked to do things rather than describe them. For example, a concierge helps a member redeem rewards, an assistant applies an eligible offer, or a workflow checks eligibility while an applicant waits. These products are judged on whether the answer arrives fast, fresh, and correct. But when customer state is spread across services, the agent must discover schemas, retrieve records, reconcile relationships, and apply business rules before it can answer. That deterministic work consumes tokens and time on every task, and the customer waits through all of it.
A live context layer moves that work into maintained data products: live, canonical business objects that agents can discover and query directly. The model spends less effort constructing the facts and more effort using them.
This benchmark compares two read architectures against the same interactive questions. One agent assembles business state from source-service APIs and documented rules. The other discovers and queries Materialize live data products over MCP. Both answer the same questions about balances, eligibility, budgets, and capacity. The evaluation section documents the full design: 720 episodes spanning six models, six operational tasks, and ten synthetic worlds. Every task has a defined correct answer or accepted outcome, and an independent deterministic oracle determines whether the agent succeeds.
All 360 attempts per architecture are included, including failures. Costs are model-inference estimates at fixed benchmark prices, not total-system costs. Input tokens include cached input across model turns.
The gains hold across every configuration tested: each model uses fewer input tokens and incurs lower aggregate model cost. Time compounds the effect in interactive settings, where the 33.8% reduction saves about 13.6 seconds per attempt on average: less time a customer waits for an account answer or a merchant waits for an eligibility explanation.
Success improved in step with efficiency, which is the unusual part: the live-context agents were cheaper and faster without trading away correctness.
Build a live context layer. Model the business as semantic objects and the relationships between them, maintain them outside the model, and let every agent read the same trusted state. Work then scales with writes plus agents rather than writes multiplied by agents: ten thousand agents add no additional derivation, because none of them rebuilds the context.
The problem: agents have to reconstruct reality
Consider an agent helping a member redeem an offer. Completing that action requires first establishing whether the member is eligible, which may depend on the member's identity and tier, consent, reward balance, active holds, the offer's conditions, and the campaign's remaining budget. Those facts may belong to several services, each with its own interface and view of the customer.
Reading the records is only the beginning. The agent must decide which identities refer to the same member, which holds are still active, which transactions affect the balance, and which business rules determine eligibility. A backdated correction or consent withdrawal can change the answer while the agent is working.
The interaction follows a familiar loop: observe, reason, act, then observe the consequences of that action. If each observation requires another round of discovery and reconciliation, the agent repeatedly spends model calls, tokens, and time establishing the state of the world.
These are not all reasoning problems. Joining records, resolving known relationships, filtering expired holds, and computing a balance are deterministic operations. Asking a model to perform them on each request makes them part of the cost and failure surface of every agent interaction.
This reframes the buying question. Model selection sets a ceiling on what an agent can reason about; architecture determines how much of that capacity is spent reconstructing facts the business already knows.
Hypothesis: move context assembly out of the model
The central hypothesis is that agents need less model work when they can read business-ready state instead of assembling it from application-native records. The stakes are highest in interactive products, where every avoidable round of discovery and reconciliation is latency the user experiences directly.
We examine five related questions:
These questions do not all have the same answer. Lower model cost is a broad finding in this evaluation. Replacing a larger model with a smaller one, or guaranteeing a response to a live event, requires a workload-specific quality test.
What we mean by a context layer
A live context layer is a shared set of continuously maintained data products that represent the business objects, attributes, and relationships agents and applications need. These contextual building blocks can be discovered, joined by shared keys, and queried for a specific decision.
The important properties are the shape of the data, its derived business meaning, the relationships between objects, and its freshness. It is represented through relational data products and SQL, exposed to the agent over MCP.
These roles can overlap. A live context layer complements retrieval and memory rather than replacing them. It also leaves actions such as placing an order or changing a reservation with the systems responsible for authorizing and committing those actions.
The evaluation
Task environment
The benchmark models operational product features in a membership and rewards business. Six independent services own membership, payments, rewards, offers, reservations, and support records. Each service has its own database and HTTP API. The synthetic worlds are configured with 200 members, 2,000 historical events, and a focused cohort of ten members.
Each of six tasks is evaluated across ten worlds and six model configurations. Every combination runs once with each architecture, producing 360 matched comparisons and 720 episodes. Both agents receive identical task requests and begin from matched business state.
Two tasks are static controls: nothing changes while the agent turns records into an answer. Four introduce a change immediately after the agent's first relevant observation. A correction alters a reward balance, a member withdraws consent, or another reservation takes the last seat. Fast and slow agents therefore encounter the change at the same logical point in their work, not at the same elapsed time. All 480 change-bearing episodes record the intended observation-triggered change.
Every task has a defined correct answer against the simulated business state, rather than an open-ended quality score. An independent, deterministic oracle reconstructs that state from the source event history and computes the expected facts using an implementation separate from the agents and Materialize's SQL. It checks the same task requirements for both architectures; success is not self-reported by the agent or judged by another language model. Passing requires correct facts, an accepted outcome, and the required evidence, observations, and actions. Tasks that permit escalation have explicitly defined acceptable outcomes, and the oracle checks that any required action actually occurred. Where an answer must cite when its facts were true, the oracle verifies that the agent observed that state. A plausible answer, or a correct number without the required observation, does not pass.
The two agents
The source-service agent assembles the answer. It discovers services with list_services, inspects their API specifications with get_openapi_spec, and reads the relevant collections.
Importantly, this agent is not denied the rules. It is offered seven loadable documents: global rules and six concept-specific guides, including reward state, offer eligibility, reservations, merchant activity, operations, and audience counts. The agent can discover them with list_skills and read them with load_skill. They include the correct hold-expiry and campaign-budget rules. Reading them consumes context like any other tool result; discovering and applying them remains the agent's responsibility.
The live-context agent reads the maintained answer. It uses Materialize's agent MCP server to discover data products with get_data_products, inspect their descriptions and columns with get_data_product_details, and run SQL with query. This is the discovery-and-query workflow available in the product, not an abstract stand-in for it. The benchmark records product discovery in every one of the 360 live-context episodes. [2]
The products are semantic objects: the things a business actually talks about — a member's reward position, an offer's eligibility, a venue's remaining capacity — modeled as people understand them rather than as any one service happens to store them. member_reward_state publishes balances and spendable points; member_offer_state publishes eligibility and reasons; reservation_state publishes available capacity. The relationships between those objects are modeled once rather than rediscovered per query: relationships identify join keys, and derived state carries source-position and entity-version evidence. Materialize maintains these products from changes to the same source databases. The agent does not receive the separate rule library because the products already publish the derived values.

Figure 1. The two agents answer the same business question. One assembles state from source APIs and documented rules; the other discovers and queries maintained data products over MCP.
Both agents retain the same ability to act. Both have a Python calculator, a wait tool, a structured-answer tool, and the same service write commands with the same authorization and concurrency checks. Neither receives unrestricted access to source databases. The live context layer changes the read interface; writes still commit through the operational services.
Representation, discovery, query capabilities, and computation placement change together. This comparison evaluates the complete read architecture, not continuous maintenance in isolation. That bundling is deliberate: in production these properties arrive together, and the section on why the context layer must be live sets out what the combination buys.
Model configurations and metrics
The models are presented by provider, from the most capable tier in the tested lineup to the smaller tier: Anthropic Opus, Sonnet, Haiku; OpenAI Sol, Terra, Luna. Each model has 60 task attempts per architecture.
Within each model, both architectures use the same configuration and limits: a 16,384-token output cap, up to 30 model calls and 100 charged tool calls, and enabled prompt caching. Opus and Sonnet use high effort; the OpenAI models use reasoning effort none. These are the evaluated settings, not a claim that different providers' configurations are equivalent.
We measure task success, input and visible output tokens, model and tool calls, elapsed time, and estimated model cost. Model cost per successful task divides total cost, including failed attempts, by the number of successful tasks. This prevents inexpensive failures from appearing to be inexpensive completed work.
What solving the tasks requires
Point-balance lookup: a balance is a business definition. An apparently active hold may already have expired and should no longer reduce spendable points. The source-service agent must reconcile the ledger and holds using that rule; the live-context agent can read the maintained balance. Even with no mid-task change, input use falls 41.8%.
Campaign-budget report: simple rules still matter. Remaining budget must deduct both outstanding reservations and reservations already consumed. Subtracting only outstanding reservations overstates what the merchant can spend. In this static control, the live-context agent succeeds in 59 of 60 attempts versus 41 of 60 for source-service access.
Spendable after correction: the answer changes after the first look. A backdated credit correction changes the reward history the agent just observed. It must reconcile the updated account and support its answer with an observed state, rather than answer from its first read. Live-context agents succeed in all 60 attempts, using 84.7% fewer input tokens.
Large-history reconciliation: report or escalate. The agent reconciles a longer ledger while a correction changes it. It may report the correct balance or open a support case for an unresolved discrepancy. Escalation requires actually creating the case, not merely suggesting one. This tests judgment as well as arithmetic.
Audience eligibility: count people, not rows. The agent must count each canonical member once, apply consent and offer rules, and explain why a named member is ineligible after consent is withdrawn. The task combines population-level calculation with a customer-facing explanation. Mean task time falls from 84.1 to 47.6 seconds.
Capacity monitoring: observing is the task. A correct first read is not enough. The agent must observe the slot reach zero availability and report when it saw that state. Live-context agents pass 52 of 60 attempts versus 58 for source-service access. A maintained answer cannot replace the decision to keep watching.
Each row includes six models across ten worlds, or 60 attempts per architecture. These are operational use cases with defined outcomes, not open-ended research prompts.
Results
Agents with live context use 76.5% fewer input tokens
The live context layer reduces average accounted input from 557,567 to 130,862 tokens per attempt, a 76.5% reduction. The context layer uses fewer input tokens in 348 of the 360 matched comparisons. The benefit is broad across the tested models, rather than concentrated in one provider.
Results use all 60 attempts per model and architecture. Reductions compare aggregate resource use within the same model; they are not averages of individual percentage savings. Token and cost reductions differ because input tokens are counted, not priced: cached reads cost roughly a tenth of uncached input, output costs several times more than input, and the two architectures differ in that mix. Where cost reduction trails token reduction — most visibly for Luna, at 68.3% against 78.8% — the source-service agent's larger but more repetitive context was served more cheaply from cache.
The reduction remains 69.9% when audience eligibility, the most token-intensive task for source-service access, is excluded. Maintained context also reduces visible output by 49.7%. The agent is not only receiving less information; it is generating less text and tool-argument output while completing the work.
Agents with live context make 45.7% fewer tool calls
Average charged tool calls fall from 15.05 to 8.17 per attempt, a 45.7% reduction. Average model calls fall from 8.74 to 7.01, a 19.9% reduction. Explicit Python compute calls, including calls that return errors, fall from 650 to 16 over the complete evaluation. When derived state is already maintained, whatever dynamic logic remains expresses as a SQL predicate against a data product rather than procedural code generated inside the agent loop. Models write SQL well, and because SQL is declarative the agent states the business logic it wants instead of the execution steps to get there which takes a class of errors off the critical path.
The amount of discovery and rule information returned to the model falls by 91.4%, and model-visible business-read payload falls by 85.9%. These measures are consistent with the agent doing less discovery, retrieval, and reconstruction inside its own loop.
The improvement does not mean every task takes fewer steps. On large-history reconciliation, the context layer agents make more model calls in aggregate, but process fewer input tokens. The benefit is a reduction in the work carried through the model, not necessarily in every individual counter.
Agents with live context incur 75.8% lower model costs
At one million agent task attempts per month, the difference becomes material. Using the rounded benchmark averages of $0.429 and $0.104 per attempt, monthly model inference would cost approximately $429,000 versus $104,000 — a difference of about $325,000. Over twelve months at the same volume, that becomes approximately $5.15 million versus $1.25 million, or $3.90 million less in annual model inference costs.
Model selection and context architecture should be evaluated together. With the live context layer, Haiku's success rises from 28 to 56 of 60 attempts, while its estimated model cost per successful task falls from about $0.375 to $0.051. Luna rises from 48 to 56 successes, with cost per successful task falling from about $0.037 to $0.010.
These results show that smaller configurations can benefit substantially from better context. For some workloads a smaller model does become sufficient once context is maintained, and in no tested case did the context layer require a stronger model than the source-service path — but sufficiency is workload-specific and has to be tested. Haiku, Terra, and Luna each achieve 93.3% success with the context layer, which would not meet a 95% acceptance target. Select a model against the required business outcome, not cost alone.
Input-token reduction and task success by model, grouped by provider and ordered by tier. All 60 attempts per model and architecture are included.
Agents with live context succeed on 96.1% of tasks
The live context layer completes 346 of 360 tasks successfully, compared with 301 of 360 for source-service access. That is an increase from 83.6% to 96.1%, or 12.5 percentage points.
Errors in source-service episodes include counting expired reward holds as active, omitting consumed reservations from budget deductions, and miscomputing eligible audiences. These are examples of business-state construction errors that a maintained data product is intended to reduce.
The improvement is not universal. Opus completes one fewer task with the context layer. The context layer also cannot prevent an agent from overriding a correct published value with its own incorrect interpretation. Data quality and model behavior remain separate responsibilities.
Agents with live context complete tasks in 33.8% less time
The live context layer reduces average end-to-end task time by 33.8%, from 40.16 to 26.57 seconds. Every tested model finishes faster on average. For customers waiting on an account answer or merchants waiting on an eligibility explanation, responsiveness is part of product quality, not just a resource metric.
The workload examples make that difference concrete. A point-balance lookup averages 12.6 seconds instead of 21.4; a corrected-balance task averages 15.9 instead of 35.6; an audience-eligibility task averages 47.6 instead of 84.1. These are observed task averages, not response-time guarantees. Headroom is the second-order benefit: when a task budget is fixed by what a user will wait for, a faster attempt leaves room to retry a failed tool call, re-read state after a write, or escalate to a stronger model — recovery paths that a slower architecture has to spend its budget to avoid.
Serving latency and agent task time are different measurements. Materialize's single-digit-millisecond serving claim concerns the data layer's response to a query. [5] The 33.8% reduction here measures the agent's complete task, including model calls and tool interactions; it does not benchmark query serving in isolation. Fewer calls and less material to process are consistent with the shorter task times, although the experiment does not isolate their individual contributions.
Why it works
The measured reductions point to three ways context architecture can change the agent's work.
Less discovery. A business-facing data product gives the agent an entry point such as member reward state or campaign budget, rather than requiring it to navigate several application interfaces. The 91.4% reduction in discovery and rule payload reflects this difference in the tested architectures.
Less repeated rule application. Joins, filtering, aggregation, and eligibility rules can be expressed in software outside the model. The agent then consumes their result rather than generating another implementation during the task. The reduction in explicit compute calls is consistent with this shift.
Less information carried through the conversation. Source records, schemas, and intermediate calculations accumulate across model turns. Returning the relevant derived state reduces the material the model must process again on later calls, even when prompt caching is available.
These are interpretations supported by the observed resource counters. The experiment does not assign a separate share of the savings to semantic modeling, SQL query capabilities, or incremental maintenance.
The architectural principle: a live context layer of semantic objects
The architectural principle is a live context layer: the business modeled as semantic objects and the relationships between them, maintained outside the model, and read by every agent and service that needs it. The objects are the things the business already talks about — a member's reward position, an offer's eligibility, a venue's remaining capacity. The relationships are what let an agent move between them without a bespoke join for each new feature, so new use cases compose existing objects instead of starting again at the source records.
This implies a division of labor. Agent architecture should separate deterministic state construction from probabilistic reasoning.
Compute once, reason many times is how the layer is maintained, not the principle itself. Each object is defined once in SQL and kept current as its inputs change, so consumers read a maintained answer rather than re-deriving one. This does not mean changing data is computed only once forever; it means the definition is written once and the work of keeping it true is done incrementally, as changes arrive.
Without that layer, each request may reconstruct the same member, reward balance, or eligibility decision. With it, agents and applications can reuse a common maintained definition. New features compose existing contextual building blocks rather than starting again with every service's records. The benchmark measures the resulting reduction in agent-side work, but that reduction is only the most visible of three distinct benefits.
First, work scales additively rather than multiplicatively. When each agent assembles its own context, the derivation work is repeated per request, so total work grows with writes multiplied by agents. When the context layer maintains the objects, derivation happens once per change and every agent reads the result, so total work grows with writes plus agents. Ten thousand agents require no additional derivation — reads remain, but they are lookups against a maintained result rather than re-derivations against the systems of record. Capacity planning decouples from agent adoption.
Second, correctness improves. Deterministic logic — identity resolution, hold expiry, balance arithmetic, rule evaluation — runs in a system built to execute it, where it can be tested, reviewed, and versioned like any other production code. The model is never asked to re-derive a rule it might get wrong, and two agents asking the same question cannot arrive at different answers.
Third, the model has less to reason over. A smaller, business-shaped context leaves more of the model's attention for the part only it can do: interpreting what the user wants and deciding what to do about it.
Why the context layer must be live
Preassembled context is useful only if its age is appropriate for the decision. A historical report can use a historical snapshot. An agent deciding whether to reserve capacity or apply an offer needs to account for changes that affect whether the action is still valid.
Three intervals matter: how long a source change takes to reach the data product, how long the query takes to return it, and how long the agent takes to observe and act. A fast query over old state does not solve the first problem. A fresh result does not solve the last.
The intended feedback loop is source change, context update, agent observation, action, updated context. Each step should have an explicit contract: how fresh the data must be, what happens if that requirement cannot be met, and which checks the action service performs before committing a write.
Continuous maintenance addresses the update step. It does not eliminate source-system delays, replace transaction checks in an operational service, or guarantee that an agent's action remains valid after its last read.
What liveness does buy is a closed loop, and three things follow from closing it. Resources scale better, because context is maintained once as data changes rather than recomputed once per agent per request. The useful measure of freshness becomes time to trusted action rather than time to query, because a fresh answer is only worth having if it is still true when the agent acts on it. And an agent can observe the consequences of its own writes on the next read instead of the next refresh — which is what makes multi-step workflows tractable at all: reserve the capacity, confirm the reservation landed, then tell the customer.
Building a live context layer
Materialize is the live context layer for agents and apps. Developers use SQL to turn siloed operational records into reusable data products, maintained as their sources change. This is the foundation for grounding agents in a trustworthy, up-to-the-second view of the business, with freshness measured from source change through the served result. [5]
Ingest operational changes. Connect the relevant databases and event streams. Establish freshness requirements and measure the source-to-query path. Source CDC behavior and available ingestion capacity remain part of the system's performance, as the companion ingestion paper explains.
Maintain business definitions. Express relationships, filters, and derived values in SQL. Materialize incrementally updates indexed views and materialized views as data arrives, rather than requiring every consumer to recompute the underlying result. Indexes make maintained results available in memory for serving. [1]
Make context discoverable over MCP. Publish documented data products with scoped access. Materialize's built-in agent MCP server exposes their descriptions, fields, and query interface: the same discover, inspect, and query workflow exercised in this benchmark. The MCP interface is currently documented as public preview. [2]
Define consistency and freshness together. Materialize supports strict-serializable transactions, but transaction consistency is not the same as a promise that every latest upstream change is already reflected. Choose the appropriate read policy and monitor ingestion lag rather than treating “consistent” as synonymous with “zero delay.” [3]
Propagate changes where needed. Downstream consumers can use SUBSCRIBE to receive changes to query results. Applications can use those updates in event-processing or index-update pipelines; an external search or vector index still needs its own integration and freshness contract. [4]
Example context layer
A rewards question illustrates the distinction between exposing data and exposing business meaning.
Suppose a member asks, “How many points can I spend after the recent correction?” Source records may include posted credits, pending credits, reversals, and holds. A hold can still have an active status while its expiry means it should no longer reduce the spendable amount. The agent must apply the full rule, not just sum a column.
A semantic product can expose the relevant state through a member-centered interface:

Figure 2. Example: related operational records contribute to a maintained member-reward state. This is an illustrative data-product design, not an additional benchmark result.
These are not simply copied fields from one service. They encode relationships and business definitions that the agent would otherwise need to reconstruct.
On the benchmark's spendable-after-correction task, the context layer agents pass all 60 attempts versus 52 for source-service access, while using 84.7% fewer input tokens. Efficiency is the pleasant side effect; auditability is the substantive gain. Because the rule executes in the data layer, the figure the agent reports traces back to a maintained definition and the records that produced it, rather than to a model's intermediate reasoning. A reviewer can ask why the spendable amount is what it is, and get the same answer twice. The model still needs to understand the request and explain the result; it no longer needs to implement the balance calculation as part of answering it.
Implications for agent architecture
Evaluate context before paying for more model capability. A stronger model may handle a difficult source interface better, but model choice is not the only way to improve the workflow. Test whether the required state can be represented more directly.
Design tools around business questions. Start with the objects and derived facts needed for a decision. A concise eligibility or budget product may be a better read interface than a collection of low-level endpoints, while the existing application API remains the right interface for writes.
Treat context as a maintained data product. Give the definitions owners, tests, access rules, and freshness objectives. A smaller prompt is useful; a reusable and governed definition of the underlying business state has a broader role.
Measure cost per correct outcome. Compare quality, elapsed time, and model inference on the same workload, including failed attempts. Set the required success rate before choosing a model, then evaluate the model and context interface together.
For a proof of concept, choose a bounded workflow with explicit rules and measurable outcomes. Run the current architecture and a maintained-context version against the same tasks, include state changes and adverse cases, and test the model tiers you would actually deploy. This makes the evaluation a purchasing decision about your workload, not a prediction based on a benchmark average.
Limitations
This benchmark is a model for operational product features: an account assistant explaining a balance, a merchant tool reporting budget, an eligibility workflow explaining consent, or an agent monitoring availability. These tasks have explicit business rules, identifiable entities, and outcomes that can be checked against operational state.
Open-ended research and tasks dominated by unstructured discovery may benefit differently. Their work is less about deriving a defined business fact from known records and more about finding, weighing, and synthesizing new information. The measured gains should be understood in the context of the operational features tested here.
A live context layer cannot prevent all model mistakes. The capacity-monitoring result makes the boundary concrete: maintaining current state and choosing to observe it at the right moment are separate responsibilities.
Conclusion
Across the tested models and tasks, maintained semantic context changes the economics of agent execution. It reduces accounted input tokens by 76.5%, estimated model cost per attempt by 75.8%, and average elapsed time by 33.8%, while increasing overall task success from 83.6% to 96.1%.
The results suggest a practical division of labor. Let the data system maintain defined relationships and derived business state. Let the model interpret requests, plan, and reason over that state. Keep freshness, observation policy, and action validation explicit.
A live context layer is the organizing principle: the business modeled as semantic objects and the relationships between them, maintained outside the model, and read by every agent that needs it. Compute once, reason many times is what makes it scale. Work grows with writes plus agents rather than writes multiplied by agents, so ten thousand agents add no additional derivation. The opportunity is not only to make models better at reconstructing context, but to remove the need to reconstruct it at all.
Companies like Bilt Rewards already run this architecture in production. Bilt runs its AI concierge and real-time audiences on a live context layer, and its VP of Data Analytics describes the result this way: "We serve most concierge traffic on fast, efficient models and get frontier-quality answers because the context does the heavy lifting, not the model." Others report the same shape of outcome from the same architecture: Neo Financial cut infrastructure spend on its fraud system by roughly 80%, and Vontive brought loan-eligibility rule evaluation from 27 seconds to half a second. This is customer-reported experience, separate from the synthetic benchmark; Bilt's production traffic is not the dataset evaluated here. [6]
Run the comparison on your own workload. Pick one interactive feature — a balance, eligibility, budget, or availability workflow — and define its context as data products in Materialize. Expose them over MCP, point the same model at both read paths, and measure success rate, end-to-end task time, and model cost per successful task. Include a source change after the first read; that is where the two architectures separate.
Start free at materialize.com, or Book a Materialize demo and bring the workflow with you.
Appendix: results and measurement notes
Model economics
Model cost per successful task includes the cost of all attempts. Each row covers 60 attempts per architecture.
Model labels are shortened in the main text; Haiku's recorded identifier is claude-haiku-4-5-20251001.
Benchmark design
The evaluation uses ten synthetic worlds, six fixed task templates, and one trajectory per model-task-world-architecture combination. Model and architecture execution order were not randomized. The comparison changes semantic representation, discovery, query capabilities, and computation placement together; it does not isolate incremental maintenance as the cause of the measured gains.
How the figures are calculated
Input tokens are the sum of uncached, cache-read, and cache-write tokens across model turns. They are not unique context size or a price-weighted unit. Output figures refer to visible output; hidden reasoning is not combined with it because reporting differs by configuration. Cost estimates use the fixed September 8, 2026 USD price schedule associated with the benchmark.
Percentage reductions compare the sum for the context-layer architecture with the sum for source-service access on the same population. Every attempt remains in the resource totals. There is no enforced episode wall-clock limit; the reference of 120 seconds is not itself a pass/fail condition.
The ten world seeds are 1, 3, 4, 5, 6, 7, 8, 9, 10, and 11. Exploratory uncertainty estimates resample worlds, retaining all models, tasks, and both architectures together. With 100,000 paired world-cluster bootstrap resamples, the 95% interval is 74.0%-78.5% for input reduction, 73.8%-77.3% for model-cost reduction, 28.9%-37.6% for mean-time reduction, and +7.8 to +17.2 percentage points for the success difference. These intervals are conditional on the tested models and templates, not forecasts for an arbitrary customer workload.
Product and customer references
[1] Materialize documentation: Views.
[2] Materialize documentation: MCP Server for Agents.
[3] Materialize documentation: Isolation levels.
[4] Materialize documentation: SUBSCRIBE.
[5] Materialize: The live context layer for agents and apps.
[6] Materialize customer story: Inside Bilt's live context layer.
