Session Memory

Conversation history with compaction strategies. See the memory pattern for usage.

techrevati.runtime.memory

Session memory — conversation history with compaction.

ConversationMemory is a small protocol for accumulating a turn-by-turn message history that a caller feeds back into the model. Long sessions outgrow the model's context window, so memory applies a compaction strategy after each append:

  • :class:NoCompaction — keep everything (default).
  • :class:WindowCompaction — keep the last N messages (optionally always retaining system messages).
  • :class:TokenBudgetCompaction — drop oldest messages until an estimated token budget is met (system messages retained first).

Both limiting strategies share two guarantees, even when that means exceeding the limit: the newest message always survives (it is usually the question being asked), and the kept window never starts on a role="tool" message, whose originating call would otherwise be gone. Messages are selected by position, so appending the same MemoryMessage instance repeatedly is safe.

The runtime does not own the model call, so memory is caller-driven: append what you send and receive, then read messages() to build the next prompt.

memory = InMemoryConversationMemory(
    compaction=TokenBudgetCompaction(max_tokens=8000)
)
memory.add(MemoryMessage("system", "You are a loan assistant."))
memory.add(MemoryMessage("user", question))
reply, _ = session.run_turn(lambda: call_model(memory.messages()))
memory.add(MemoryMessage("assistant", reply))

MemoryMessage dataclass

MemoryMessage(role, content, metadata=None)

One conversation message.

CompactionStrategy

Bases: Protocol

Reduce a message list to fit a budget. Pure; returns a new list.

ConversationMemory

Bases: Protocol

Accumulates conversation messages with compaction.

NoCompaction dataclass

NoCompaction()

Keep every message.

WindowCompaction dataclass

WindowCompaction(max_messages, keep_system=True)

Keep the most recent max_messages (system messages retained first).

The newest message survives even if system messages alone fill the window, so the result can exceed max_messages; see the module docstring.

TokenBudgetCompaction dataclass

TokenBudgetCompaction(
    max_tokens,
    estimator=_default_estimator,
    keep_system=True,
)

Drop oldest messages until the estimated token budget is met.

System messages are retained first; if they alone exceed the budget they are all kept (correctness over budget — never silently drop instructions). The newest message is kept on the same principle, even when it alone exceeds the budget; see the module docstring.

InMemoryConversationMemory

InMemoryConversationMemory(*, compaction=None, initial=())

Process-local conversation memory; compacts after each append.