Session Memory¶
Conversation history with compaction strategies. See the memory pattern for usage.
techrevati.runtime.memory ¶
Session memory — conversation history with compaction.
ConversationMemory is a small protocol for accumulating a turn-by-turn message
history that a caller feeds back into the model. Long sessions outgrow the model's
context window, so memory applies a compaction strategy after each append:
- :class:
NoCompaction— keep everything (default). - :class:
WindowCompaction— keep the last N messages (optionally always retainingsystemmessages). - :class:
TokenBudgetCompaction— drop oldest messages until an estimated token budget is met (system messages retained first).
Both limiting strategies share two guarantees, even when that means exceeding the
limit: the newest message always survives (it is usually the question being
asked), and the kept window never starts on a role="tool" message, whose
originating call would otherwise be gone. Messages are selected by position, so
appending the same MemoryMessage instance repeatedly is safe.
The runtime does not own the model call, so memory is caller-driven: append what
you send and receive, then read messages() to build the next prompt.
memory = InMemoryConversationMemory(
compaction=TokenBudgetCompaction(max_tokens=8000)
)
memory.add(MemoryMessage("system", "You are a loan assistant."))
memory.add(MemoryMessage("user", question))
reply, _ = session.run_turn(lambda: call_model(memory.messages()))
memory.add(MemoryMessage("assistant", reply))
CompactionStrategy ¶
Bases: Protocol
Reduce a message list to fit a budget. Pure; returns a new list.
ConversationMemory ¶
Bases: Protocol
Accumulates conversation messages with compaction.
WindowCompaction
dataclass
¶
WindowCompaction(max_messages, keep_system=True)
Keep the most recent max_messages (system messages retained first).
The newest message survives even if system messages alone fill the window,
so the result can exceed max_messages; see the module docstring.
TokenBudgetCompaction
dataclass
¶
TokenBudgetCompaction(
max_tokens,
estimator=_default_estimator,
keep_system=True,
)
Drop oldest messages until the estimated token budget is met.
System messages are retained first; if they alone exceed the budget they are all kept (correctness over budget — never silently drop instructions). The newest message is kept on the same principle, even when it alone exceeds the budget; see the module docstring.
InMemoryConversationMemory ¶
InMemoryConversationMemory(*, compaction=None, initial=())
Process-local conversation memory; compacts after each append.