Skip to content

Budgets & configuration

JAZ separates two concerns:

  • Pool budgets — total LLM cost and call count, shared across the whole invoke/recursion tree. Tracked and enforced only while a BudgetPool is active.
  • Per-level execution limits — recursion depth and REPL iterations per level. Like ReturnType and BudgetPool, these are hooks — the extension objects you pass to invoke to observe or constrain a run.
from jaz import invoke
from jaz.hooks import ReturnType, BudgetPool
# Tracked + enforced only while the hook is active. USD cost / call-count.
with BudgetPool(cost_budget=0.50, calls_budget=100):
result = invoke(ReturnType(str), task="Complex task")

RecursionLimit caps how deep nested invoke calls may nest; IterationLimit caps the REPL turns at each level. Both are hooks, activated as context managers:

from jaz.hooks import RecursionLimit, IterationLimit
with RecursionLimit(max_depth=2), IterationLimit(max_iterations=20):
result = invoke(ReturnType(str), task="Complex task")

Configuration is grouped by the three components of an agent:

  • llm — how JAZ talks to the model (the backend, the model id, and per-request settings like temperature).
  • repl — the REPL that runs the agent’s code (e.g. its execution timeout).
  • protocol — the codec that sits between the LLM and the REPL: it turns the model’s reply into code to run, and turns each result back into the next message the model sees. The default, CodeOnlyProtocol, is fine for most use — you rarely set it.

You configure a group by passing the component object itself (e.g. a LiteLLM(...) instance), not a dict of options — its constructor is the list of settings. Setting a group replaces it wholesale: you restate the whole component rather than tweaking one field, so there is no partial/merge update.

jaz.ConfigOverride changes config temporarily, and — exactly like a hook — it can be passed two ways (see the two ways to pass a hook):

  • As a with block, it propagates to every nested invoke inside the block.
  • Passed positionally to a single invoke, it is local to that one call — nested invokes the agent spawns fall back to the ambient config.
import jaz
from jaz.llm import LiteLLM
from jaz.repl import PythonREPL
# `with` block — propagates to nested invokes:
with jaz.ConfigOverride(llm=LiteLLM(model="openai/gpt-5-mini", temperature=0.7, max_tokens=2000)):
result = invoke(ReturnType(str), task="Creative task")
# Positional — local to this one invoke only:
result = invoke(
jaz.ConfigOverride(llm=LiteLLM(model="openai/gpt-5-mini", temperature=0.7)),
task="Creative task",
)
# Global configuration (applies to every invoke, no block).
jaz.configure(
llm=LiteLLM(model="openai/gpt-5-mini"),
repl=PythonREPL(exec_timeout=60),
)

LiteLLM is the default (and only built-in) backend — one backend routing to every provider LiteLLM supports; its model is a LiteLLM route like openai/gpt-5-mini. See the configure and jaz.llm reference for every field.