Runtime

Session Memory

Conversation history with compaction strategies. See the memory pattern for usage.

techrevati.runtime.memory

Session memory — conversation history with compaction.

ConversationMemory is a small protocol for accumulating a turn-by-turn message history that a caller feeds back into the model. Long sessions outgrow the model's context window, so memory applies a compaction strategy after each append:

  • :class:NoCompaction — keep everything (default).
  • :class:WindowCompaction — keep the last N messages (optionally always retaining system messages).
  • :class:TokenBudgetCompaction — drop oldest messages until an estimated token budget is met (system messages retained first).

The runtime does not own the model call, so memory is caller-driven: append what you send and receive, then read messages() to build the next prompt.

memory = InMemoryConversationMemory(
    compaction=TokenBudgetCompaction(max_tokens=8000)
)
memory.add(MemoryMessage("system", "You are a loan assistant."))
memory.add(MemoryMessage("user", question))
reply, _ = session.run_turn(lambda: call_model(memory.messages()))
memory.add(MemoryMessage("assistant", reply))

MemoryMessage dataclass 

MemoryMessage(role, content, metadata=None)

One conversation message.

CompactionStrategy

Bases: Protocol

Reduce a message list to fit a budget. Pure; returns a new list.

ConversationMemory

Bases: Protocol

Accumulates conversation messages with compaction.

NoCompaction dataclass 

NoCompaction()

Keep every message.

WindowCompaction dataclass 

WindowCompaction(max_messages, keep_system=True)

Keep the most recent max_messages (system messages retained first).

TokenBudgetCompaction dataclass 

TokenBudgetCompaction(
    max_tokens,
    estimator=_default_estimator,
    keep_system=True,
)

Drop oldest messages until the estimated token budget is met.

System messages are retained first; if they alone exceed the budget they are all kept (correctness over budget — never silently drop instructions).

InMemoryConversationMemory 

InMemoryConversationMemory(*, compaction=None, initial=())

Process-local conversation memory; compacts after each append.