Budgets & configuration
JAZ separates two concerns:
- Pool budgets — total LLM cost and call count, shared across the whole invoke/recursion
tree. Tracked and enforced only while a
BudgetPoolis active. - Per-level execution limits — recursion depth and REPL iterations per level. Like
ReturnTypeandBudgetPool, these are hooks — the extension objects you pass toinvoketo observe or constrain a run.
Pool budgets
Section titled “Pool budgets”from jaz import invokefrom jaz.hooks import ReturnType, BudgetPool
# Tracked + enforced only while the hook is active. USD cost / call-count.with BudgetPool(cost_budget=0.50, calls_budget=100): result = invoke(ReturnType(str), task="Complex task")Per-level execution limits
Section titled “Per-level execution limits”RecursionLimit caps how deep nested invoke calls may nest; IterationLimit caps the
REPL turns at each level. Both are hooks, activated as context managers:
from jaz.hooks import RecursionLimit, IterationLimit
with RecursionLimit(max_depth=2), IterationLimit(max_iterations=20): result = invoke(ReturnType(str), task="Complex task")Configuration
Section titled “Configuration”Configuration is grouped by the three components of an agent:
llm— how JAZ talks to the model (the backend, the model id, and per-request settings liketemperature).repl— the REPL that runs the agent’s code (e.g. its execution timeout).protocol— the codec that sits between the LLM and the REPL: it turns the model’s reply into code to run, and turns each result back into the next message the model sees. The default,CodeOnlyProtocol, is fine for most use — you rarely set it.
You configure a group by passing the component object itself (e.g. a LiteLLM(...)
instance), not a dict of options — its constructor is the list of settings. Setting a group
replaces it wholesale: you restate the whole component rather than tweaking one field, so
there is no partial/merge update.
jaz.ConfigOverride changes config temporarily, and — exactly like a hook — it can be passed
two ways (see the two ways to pass a hook):
- As a
withblock, it propagates to every nestedinvokeinside the block. - Passed positionally to a single
invoke, it is local to that one call — nested invokes the agent spawns fall back to the ambient config.
import jazfrom jaz.llm import LiteLLMfrom jaz.repl import PythonREPL
# `with` block — propagates to nested invokes:with jaz.ConfigOverride(llm=LiteLLM(model="openai/gpt-5-mini", temperature=0.7, max_tokens=2000)): result = invoke(ReturnType(str), task="Creative task")
# Positional — local to this one invoke only:result = invoke( jaz.ConfigOverride(llm=LiteLLM(model="openai/gpt-5-mini", temperature=0.7)), task="Creative task",)
# Global configuration (applies to every invoke, no block).jaz.configure( llm=LiteLLM(model="openai/gpt-5-mini"), repl=PythonREPL(exec_timeout=60),)LiteLLM is the default (and only built-in) backend — one backend routing to every provider
LiteLLM supports; its model is a LiteLLM route like openai/gpt-5-mini. See the
configure and jaz.llm reference
for every field.