Orchestrator

techrevati.runtime.orchestrator

Orchestrator — single execution loop wiring the runtime primitives together.

Pairs an agent session with lifecycle tracking, usage accounting, circuit-breaker protection, automatic failure classification, permission gating, and policy evaluation. Use the primitives standalone or use AgentSession.session() (sync) / AgentSession.asession() (async) to get all of them wired in.

Example (sync): from techrevati.runtime import AgentSession, UsageSnapshot

agent = AgentSession(role="writer", phase="draft", project_id=1)
with agent.session() as session:
    text, usage = session.run_turn(
        lambda: call_model(prompt),
        model="model-a",
        usage=UsageSnapshot(input_tokens=5000, output_tokens=1200),
    )
print(session.summary())

Example (async): async with agent.asession() as session: text, usage = await session.arun_turn( lambda: acall_model(prompt), model="model-a", timeout=30.0, )

DEFAULT_FAILOVER_SCENARIOS module-attribute

DEFAULT_FAILOVER_SCENARIOS = frozenset(
    (
        FailureScenario.PROVIDER_FAILURE,
        FailureScenario.LLM_TIMEOUT,
    )
)

OrchestrationSession dataclass

OrchestrationSession(
    worker,
    tracker,
    recovery,
    registry,
    phase,
    role,
    project_id,
    permissions=None,
    policy_engine=None,
    quality_gate=None,
    budget_usd=None,
    enforce_budget=False,
    max_iterations=25,
    guardrails=list(),
    event_sink=NoopEventSink(),
    usage_sink=NoopUsageSink(),
    events=list(),
    thread_id=None,
    saver=None,
    provider_router=None,
    usage_limits=None,
    governance=None,
    hooks=list(),
    audit_health=_AuditHealth(),
    circuit_breaker=None,
    rate_limiter=None,
)

Bases: _SessionBase

Single-agent sync execution context. Created by AgentSession.session().

run_tool

run_tool(
    tool_name,
    fn,
    *,
    hook_ctx=None,
    args=None,
    timeout=None,
    idempotency_key=None,
)

Execute a tool with permission + guardrail + governance + hook checks.

Order: permission → governance → replay lookup → before_tool hooks → pre-guardrails → fn() → post-guardrails → after_tool hooks.

hook_ctx is a mutable context shared with the hook chain. When omitted the runtime synthesizes a fresh HookContext so hooks still fire — but any mutation hooks attempt on ctx.args is lost because the caller's closure does not see it. Pass a context (and close over ctx.args inside fn) when you want hooks to redact or rewrite tool inputs.

args describes the call to hooks and to MaxRepeatedToolCallsLimit; fn still takes no arguments.

timeout raises :class:ToolTimeoutError. The worker thread cannot be killed, so a timed-out fn finishes in the background.

With a saver and thread_id configured, idempotency_key records the result once the call passes its checks and replays it on a re-run instead of repeating the side effect. A crash between fn returning and the record being written still re-executes it, so the guarantee is at-least-once: pair it with an idempotent tool or pass the same key to the downstream API.

run_turn

run_turn(
    fn,
    model="",
    usage=None,
    estimate_usage=None,
    timeout=None,
    idempotency_key=None,
    *,
    hook_ctx=None,
)

Execute one model turn with circuit-breaker + recovery wiring.

When timeout is set, fn is dispatched to a single-worker ThreadPoolExecutor and waited on with that deadline. The executor is created per-turn (cheap) so there is no pool to manage across the session.

On exception: classifies the failure, attempts recovery once, and re-raises so the caller decides whether to retry.

If a saver + thread_id are configured and idempotency_key is supplied, this method first looks for a prior checkpoint with the same key on the same thread; on a hit it returns the cached (result, usage) without calling fn, and counts that usage towards this session's limits. After a successful execution, a new checkpoint is written so a restart can replay through it. A turn that crosses a budget is checkpointed before the budget error is raised, because it was already paid for.

run_turn_with_fallback

run_turn_with_fallback(
    fn,
    *,
    model,
    usage=None,
    estimate_usage=None,
    timeout=None,
    idempotency_key=None,
    hook_ctx=None,
    failover_on=DEFAULT_FAILOVER_SCENARIOS,
)

Run a turn, asking provider_router for another model on failure.

fn receives the model to call. Returns (result, usage, model), where model is the one that answered. Each attempt is a full turn: it counts towards max_iterations and governance, so those still bound the total. Only failures in failover_on switch provider. A governance breach, budget or usage limit, permission denial, or open circuit always propagates, as does the last failure once the router has no candidate.

A circuit breaker on the session is shared by every model, so it should guard something they have in common, such as a gateway.

AsyncOrchestrationSession dataclass

AsyncOrchestrationSession(
    worker,
    tracker,
    recovery,
    registry,
    phase,
    role,
    project_id,
    permissions=None,
    policy_engine=None,
    quality_gate=None,
    budget_usd=None,
    enforce_budget=False,
    max_iterations=25,
    guardrails=list(),
    event_sink=NoopEventSink(),
    usage_sink=NoopUsageSink(),
    events=list(),
    thread_id=None,
    saver=None,
    provider_router=None,
    usage_limits=None,
    governance=None,
    hooks=list(),
    audit_health=_AuditHealth(),
    circuit_breaker=None,
    async_rate_limiter=None,
)

Bases: _SessionBase

Single-agent async execution context. Created by AgentSession.asession().

Sibling of OrchestrationSession. Sync helpers (authorize, evaluate_policy, evaluate_gate, summary, lifecycle methods) are inherited; only the execution path (arun_turn / arun_tool) and the human-in-the-loop pause are async.

ahandoff_to async

ahandoff_to(
    target_role, reason, context=None, *, hook_ctx=None
)

Async handoff helper that awaits async before_handoff hooks.

arun_tool async

arun_tool(
    tool_name,
    coro_factory,
    *,
    hook_ctx=None,
    args=None,
    timeout=None,
    idempotency_key=None,
)

Async sibling of run_tool.

Permission + governance are sync; guardrails are awaited when they implement AsyncGuardrail and called inline when they are sync Guardrail instances. Hooks are dispatched both sync and async — see hooks.py.

args, timeout and idempotency_key work as in run_tool, except that a timed-out tool is cancelled rather than left running.

arun_turn async

arun_turn(
    coro_factory,
    model="",
    usage=None,
    estimate_usage=None,
    timeout=None,
    idempotency_key=None,
    *,
    hook_ctx=None,
)

Execute one model turn with async circuit-breaker + recovery wiring.

timeout is enforced with asyncio.timeout. Cancellation from outside (parent task) is propagated as CancelledError; an internal timeout becomes TurnTimeoutError.

idempotency_key behaves the same as in run_turn: when the session has a saver + thread_id configured, a prior checkpoint with the same key short-circuits the call and counts its usage.

arun_turn_with_fallback async

arun_turn_with_fallback(
    coro_factory,
    *,
    model,
    usage=None,
    estimate_usage=None,
    timeout=None,
    idempotency_key=None,
    hook_ctx=None,
    failover_on=DEFAULT_FAILOVER_SCENARIOS,
)

Async sibling of run_turn_with_fallback.

arun_turn_stream async

arun_turn_stream(
    chunk_factory,
    *,
    model="",
    usage=None,
    estimate_usage=None,
    timeout=None,
    idempotency_key=None,
    hook_ctx=None,
)

Stream a model turn as a sequence of StreamEvent values.

chunk_factory returns an async iterator over text chunks (e.g. SSE deltas from a provider client). Each chunk is reemitted as StreamEvent.text(delta). After upstream exhausts, the generator yields a terminal StreamEvent.final("completed", usage=...) carrying the resolved usage snapshot.

Hook chain runs once before iteration begins (before_model) and once after the aggregated text is computed (after_model). Per-chunk hooks are intentionally out of scope — they would invert the cost model of streaming.

timeout is a deadline for the whole stream, checked only while waiting on upstream, so it can never fire inside the consumer's own code. A consumer that is slow between chunks hits it on the next pull.

idempotency_key replays a recorded turn as one text event plus the terminal event, without calling upstream.

Cancellation: wrap the generator in contextlib.aclosing. When the consumer leaves the block early the upstream iterator's aclose() is awaited and session._last_stream_cancelled flips to True. A plain break without aclosing defers all of that to garbage collection. No final("cancelled") event is emitted because the consumer is no longer listening; check the flag instead.

A stream that ends early books its usage when usage or estimate_usage can provide it (the latter sees the partial text), because the provider charges for what it already sent.

On failure, including a limit crossed once the stream has finished, error + final("failed") events are yielded and the exception is re-raised so callers see it.

arun_parallel_tools async

arun_parallel_tools(coro_factories, *, timeout=None)

Run several tool calls concurrently with structured concurrency.

Uses asyncio.TaskGroup so any child failure cancels its siblings and surfaces as ExceptionGroup to the caller — no orphan tasks, no swallowed exceptions. timeout (if given) applies to the whole group via asyncio.timeout.

Returns results in input order. Each coro_factory is a zero-arg callable that returns an awaitable; this matches the contract used elsewhere in the session API and means callers can build the coroutine lazily inside the group.

This method only provides the concurrency. It applies no permission, guardrail, governance or hook checks, so have each factory go through arun_tool: lambda: session.arun_tool("search", search_factory).

pause_for_input async

pause_for_input(prompt)

Mark the worker WAITING_FOR_INPUT and await an external response.

Returns a future the caller resolves via session.provide_input(value) from elsewhere in the program. Use this to wire human-in-the-loop or out-of-band approvals without leaving the session machinery.

provide_input

provide_input(value)

Resolve the most recent pause_for_input future with value.

AgentSession dataclass

AgentSession(
    role,
    phase,
    project_id=None,
    registry=AgentRegistry(),
    permissions=None,
    circuit_breaker=None,
    async_circuit_breaker=None,
    policy_engine=None,
    quality_gate=None,
    budget_usd=None,
    enforce_budget=False,
    max_iterations=25,
    guardrails=list(),
    event_sink=NoopEventSink(),
    usage_sink=NoopUsageSink(),
    saver=None,
    rate_limiter=None,
    async_rate_limiter=None,
    provider_router=None,
    usage_limits=None,
    governance=None,
    hooks=list(),
    audit_log=None,
    compliance=None,
    audit_fail_closed=False,
)

Factory for sessions. Holds shared, long-lived components.

Components are optional; the simplest invocation is AgentSession(role=..., phase=...). Provide circuit_breaker for sync sessions, async_circuit_breaker for async sessions, or both — they are independent.

To make sessions restart-resumable, pass saver (any object that satisfies the CheckpointSaver protocol) and a thread_id at session() / asession() time. The thread id is the durable handle a future process uses to pick up where this one left off.

session

session(*, thread_id=None, handoff=None)

Open a single-agent sync session.

On clean exit: worker → COMPLETED if still running. On exception: worker → FAILED with the error classified into an AgentFailureClass.

If thread_id is supplied and the orchestrator has a saver, the session writes a checkpoint after each turn and an idempotency_key on run_turn makes that turn replay-safe.

Pass the Handoff returned by handoff_to as handoff to run on the worker it reserved; otherwise that worker stays INITIALIZING.

asession async

asession(*, thread_id=None, handoff=None)

Open a single-agent async session.

Mirrors session() but uses async primitives. CancelledError from anywhere inside the async with body transitions the worker to CANCELLED instead of FAILED, and is re-raised.

The same thread_id / saver contract from session() applies; pair with arun_turn(..., idempotency_key=...) for replay-safe async turns, and pass handoff to run on the worker a Handoff reserved.

PermissionDeniedError

PermissionDeniedError(outcome)

Bases: Exception

Raised when a tool is blocked by the configured PermissionEnforcer.

AuditWriteError

AuditWriteError(error_type)

Bases: Exception

Raised by audit_fail_closed sessions once the audit log cannot be written.

The session refuses further actions, because they could not be recorded. It is permanent for that session: open a new one once the log is healthy again.

ToolTimeoutError

ToolTimeoutError(tool_name, timeout_seconds)

Bases: TurnTimeoutError

Raised when run_tool / arun_tool exceeds its timeout.

Subclasses :class:TurnTimeoutError so a single except covers both. An async tool is cancelled. A sync tool cannot be killed, so its worker thread finishes in the background: a tool whose side effects must not outlive the deadline has to enforce its own timeout too.

TurnTimeoutError

TurnTimeoutError(timeout_seconds)

Bases: Exception

Raised when a sync or async turn exceeds the configured timeout.

For sync callers, this wraps the underlying concurrent.futures.TimeoutError; for async callers, the original asyncio.TimeoutError is re-raised as this type to give a single error class across both code paths.

MaxIterationsExceededError

MaxIterationsExceededError(max_iterations)

Bases: Exception

Raised when a session attempts more turns than max_iterations allows.

Default cap of 25 prevents runaway agent loops; stopping conditions are a production-readiness requirement.