## The lorebook agent is a transformation function Every prompt engineer in the LLM orchestration business eventually hits the same wall. They want the agent to remember things across turns. They want it to update its own memory as it learns new facts. They reach for a vector store. They reach for an external service. They build a "memory layer." The agent becomes a client of its own memory. Ten Trillion Triangles TPipe does not work that way. In Ten Trillion Triangles TPipe, the agent updates its own lorebook in the same pipe that just produced the output. The mechanism is `setTransformationFunction { ... }`. The hook fires after the LLM runs. The hook reads the bank, merges new entries, writes the bank back. The player's turn continues. The lorebook grows. This is a transformation step on the same pipe that just emitted the agent's output. It runs in the same coroutine, the same execution context, the same lifecycle as the agent's main turn. Autogenesis runs this way. The lorebook update lives at `lorebookAgent.kt:166-283` in the Ten Trillion Triangles TPipe codebase. Every turn, the gameplay orchestrator fires the lorebook writer in a detached coroutine and continues serving the player. The lorebook grows in the background. By turn 120, the agent has remembered every character, every location, every faction, every relationship the player has touched. Here is the actual production code. ## Autogenesis: the canonical example ```kotlin setTransformationFunction { val extraction = extractJson(it.text) ?: LorebookExtraction() val storyContext = ContextBank.getContextFromBank("story") var mergedCount = 0 var newCount = 0 // Merge characters for (char in extraction.characters) { val existing = storyContext.findLoreBookEntry(char.name) val merged = if (existing != null) { mergedCount++ mergeCharacterEntry(deserialize(existing.value), char) } else { newCount++ char } storyContext.addLoreBookEntry( key = char.name, value = serialize(merged), aliasKeys = merged.aliases ) // Fuzzy match lorebook keys to players and update their history in the world state. WorldManager.updatePlayerHistoryDITL(merged.name, merged.aliases, merged.description) } // ...events, locations, items, factions, relationships all follow the same shape... ContextBank.emplaceWithMutex("story", storyContext) Logger.info(LogCategory.SYSTEM, "[LOREBOOK] Updated story lorebook: $newCount new, $mergedCount merged") return@setTransformationFunction it } ``` Read this carefully. The transformation function gets the LLM output as `it.text`. It extracts a typed `LorebookExtraction` object from the output. It reads the `"story"` page key from `ContextBank`. For each character the LLM extracted, it looks up the existing lorebook entry by name. If an entry exists, it calls `mergeCharacterEntry` to merge the new facts onto the old description. If not, it adds a new entry. Six entity types — characters, events, locations, items, factions, relationships — each get their own merge function that knows how to combine the old value with the new value without losing data. Then `ContextBank.emplaceWithMutex("story", storyContext)` writes the merged window back. One line. The mutex is what makes this safe in a coroutine context where multiple writers could race. The merge functions are the part the docs skip. Six of them, all in the same file, all doing the same shape: preserve the old `description`, concatenate new facts onto it, de-dupe aliases with `.distinct()`, prefer non-blank values over blanks. The relationship merge uses a composite key: `"${rel.entity1}-${rel.entity2}"`. There is no natural canonical name for "Alice knows Bob." Every merge function is the same ten lines. Boring production code that actually works. ## Fire-and-forget means the player's turn does not wait The lorebook update runs detached from the turn loop. The invocation is at `gameplayOrchestrator.kt:1794-1821`: ```kotlin // Update lorebook with current turn's narrative (fire and forget) if(narrative.isNotBlank()) { AgentCoroutineScope.scope.launch { try { val lorebookAgent = agent.builders.lorebook.buildLorebookUpdateAgent().apply { pipelineName = "lorebook" enableTracing(TraceConfig(enabled = true, detailLevel = TraceDetailLevel.DEBUG)) enablePipeTimeout( applyRecursively = true, duration = 180000, autoRetry = true, retryLimit = 5 ) init(true) val broadcastIds = getAllConnectedClientIds() streamPipelineOutputToAgentWorkBuffer(broadcastIds, this) attachProgressHooks(this, 3) } lorebookAgent.execute(MultimodalContent(narrative)) saveSystemTrace("LorebookUpdate", lorebookAgent) } catch(e: Exception) { Logger.error(LogCategory.SYSTEM, "Failed to background update lorebook: ${e.message}") } } } ``` Three properties of this call matter. The lorebook update runs in `AgentCoroutineScope.scope.launch`. It is detached from the calling code. The gameplay orchestrator returns to the player the moment it has launched the coroutine. The player does not wait for the lorebook writer to finish its 180-second timeout. The lorebook agent has `enablePipeTimeout(applyRecursively = true, duration = 180000, autoRetry = true, retryLimit = 5)`. The lorebook writer is allowed to fail and retry up to five times. The player's turn does not have access to retry logic. The lorebook writer does. The asymmetry is deliberate. The whole `try/catch` block exists to guarantee the lorebook writer can never fail the player's turn. If the lorebook extraction pipeline throws, the exception is logged and the player's turn completes anyway. The lorebook is allowed to be stale. The player's experience is not. ## TPipeWriter: the honest version TPipeWriter is the multi-agent creative writing pipeline the team at Ten Trillion Triangles ships as a proof-of-concept for long-horizon agent work. It uses the same pattern: let the LLM write to a lorebook-shaped JSON, then merge and persist. The production code keeps the workarounds visible. Here is the actual transformation function from `Env.kt:729-768`: ```kotlin suspend fun recordLoreBook(content: MultimodalContent) : MultimodalContent { /** * bug: There's quite a few issues here: * * 1. The llm keeps adding _ instead of spacing * 2. We aren't using the add lorebook function so it's not addressing with casing issues. This means * we'll almost never have a hit. */ //Create new context window to prepare to merge it with our global context. var newLoreBookEntries = extractJson(content.text) newLoreBookEntries?.contextElements?.clear() //Stop deepseek from writing to this for some reason. if(newLoreBookEntries == null) { newLoreBookEntries = repairAndDeserialize(content.text) if(newLoreBookEntries == null) { throw Exception("Cannot deserialize deepseek jank ass json: ${content.text}") } } //Fix nonsense like _ being here and missing casing issues. newLoreBookEntries.cleanLorebook("_", " ") //Merge in new keys that do not exist yet. var bankedContext = ContextBank.getContextFromBank("main") bankedContext.merge(newLoreBookEntries, content.currentPipe?.getLorebookScheme()!!.second, content.currentPipe?.getLorebookScheme()!!.first) //Update the banked context. content.context = bankedContext ContextBank.emplaceWithMutex("main", content.context) return content } ``` The comments in this function are the production bug log. The LLM keeps emitting underscores where it should emit spaces. The casing is wrong, so substring lookups miss. The `cleanLorebook("_", " ")` call is the workaround. It is a real production codebase being honest about what breaks at the boundary between probabilistic output and deterministic storage. There is a cleaner version. `recordLoreBookPlus` at `PlusWriterUtil.kt:246-266` does the same merge without the workaround hack. Both versions live in the tree. The team kept the janky one because deleting it would delete the bug notes. The history is the documentation. New readers see the workaround and learn the failure mode. Old readers see the cleaner version and learn what the workaround fixed. ## TStep: when the writer is a code indexer The same shape shows up in TStep, the debugging agent the Ten Trillion Triangles team uses to drive LLDB. The motivation is different. The lorebook is populated by a code indexer writing symbol locations into the banked context, not by an LLM extracting entities from a turn's narrative. `LorebookPublisher.kt:7-56`: ```kotlin class LorebookPublisher( private val repository: CodeIndexRepository ) { fun publish(request: LorebookPublishRequest): LorebookPublishResult { val index = repository.get(request.indexId) ?: throw IllegalArgumentException("Code index '${request.indexId}' not found") val contextWindow = ContextBank.copyBankedContextWindow() ?: throw IllegalStateException("Context window is not initialised in ContextBank") val updatedLorebook = contextWindow.loreBookKeys.toMutableMap() val createdKeys = mutableListOf() val symbolLocations = request.symbolIdentifiers.flatMap { identifier -> resolveSymbolLocations(index, identifier) } symbolLocations.forEach { location -> val fileSummary = index.files[location.filePath] val key = buildLorebookKey(location, request) val existing = updatedLorebook[key] val entry = existing ?: LoreBook().apply { this.key = key } entry.value = buildLorebookValue(location, fileSummary, index, request) entry.weight = request.weight if (request.includeAliases) { entry.aliasKeys = (entry.aliasKeys + buildAliasCandidates(location)).distinct().toMutableList() } if (request.includeLinkedKeys) { val linked = gatherLinkedKeys(index, location) entry.linkedKeys = (entry.linkedKeys + linked).distinct().toMutableList() } updatedLorebook[key] = entry createdKeys.add(key) } contextWindow.loreBookKeys = updatedLorebook runBlocking { ContextBank.updateBankedContextWithMutex(contextWindow) } return LorebookPublishResult( indexId = index.id, keysAdded = createdKeys ) } ``` The pattern is identical to Autogenesis. The code index is the writer. The bank is the destination. The shape of the operation is the same: read the bank, merge new entries keyed by some stable identifier, write the bank back. This is the property that makes the architecture general. The agent's memory is coupled to the bank's API, not to the mechanism that populates it. The same `ContextBank` works whether the writer is an LLM extracting narrative entities, a code indexer publishing symbol locations, or a hand-coded routine that pushes game state after every turn. The bank does not know or care who is writing. The bank knows how to merge, how to lock, and how to persist. ## 120+ turns is what this buys you The Autogenesis turn-completion handler is where the continuity actually shows up. Every turn, the system reads the `"story"` page key, appends the new turn's narrative, and writes the bank back. `ActionHistoryRpcHandlers.kt:47-63`: ```kotlin suspend fun processTurnComplete(ctx: RpcCallContext, batch: ActionHistoryBatch): GameHistory { val turnPlayer = batch.turnPlayer.ifBlank { batch.events.firstOrNull()?.event?.player.orEmpty() } val gameHistory = processor.processActionHistoryToGameHistory(batch.events.map { it.event }, turnPlayer) WorldManager.history.add(gameHistory) updateWorldFromGameHistory(gameHistory) val scoringResult = parser.parseHistoryForScoring(gameHistory) // Add story content to ContextBank val storyContext = ContextBank.getContextFromBank("story") ?: ContextWindow() storyContext.contextElements.add(gameHistory.turnStory) ContextBank.emplace("story", storyContext) broadcastTurnComplete(gameHistory) return gameHistory } ``` Three lines. Read the bank. Append the new turn. Write the bank back. The Ten Trillion Triangles TPipe Autogenesis game master has survived 25 rounds and 120+ turns on this exact pattern. No vector store. No external memory service. The lorebook grows. The agent remembers. The story accumulates. The player does not wait. ## Patterns that emerged across all three projects Three projects, three motivations, one architecture. The patterns that fell out: **`aliasKeys` is the production workaround for substring matching.** Autogenesis aliases `"history"` to `["story", "event", "turn", "previous", "prior", "past", "round"]` at `answerAgent.kt:168` so the lorebook fires when the player mentions any of those words. TStep aliases symbols to `["name", "stem.name", "stem"]` at `LorebookPublisher.kt:34` so the same entry fires whether the LLM says the symbol by its qualified name, its short name, or its file path. Substring matching is exact. The user is not. Aliases bridge the gap. **Per-page mutexes are how concurrent writers avoid losing updates.** `ContextBank` has a coarse `bankMutex` and a `pageMutexes: ConcurrentHashMap` table. `emplaceWithMutex` acquires the global lock. Unrelated page keys can be written in parallel without contention. Autogenesis's lorebook writer holds the global lock while it merges six entity types. TStep's `LorebookPublisher` uses `updateBankedContextWithMutex` to mutate the banked context's lorebook keys in place, the only place in the codebase that does. **`StorageMode.DISK_ONLY` keeps the hot bank small while preserving persistence.** Autogenesis's per-NPC chat history lives under the page key `"chat-$connectionId-${npc.name}"` at `chatAgent.kt:94`. Every turn, the system writes the chat history to disk with `emplaceSuspend(chatContextKey, chatHistory, StorageMode.DISK_ONLY)`. The hot in-memory bank stays small. The on-disk page is loaded when the next conversation opens. Long-term memory without in-memory bloat. **No project prunes lorebook entries.** The production policy across all three projects is: never throw away lore. The only pruning anywhere in the codebase is the `takeLast(6000)` overflow handler in `tpipe-manifold-validation`'s `ManifoldOrchestrator.kt:81-88`: ```kotlin // CRITICAL: Add context management to prevent overflow manifold.autoTruncateContext() manifold.setContextTruncationFunction { context -> // Simple truncation - keep most recent content within reasonable limits if (context.text.length > 8000) { context.text = context.text.takeLast(6000) } } ``` Eight lines. No summarization. No lorebook pruning. No semantic compression. Just `takeLast(6000)` when the context exceeds 8000 characters. It is the simplest possible overflow safety net, and it is in production because it is the smallest thing that works. The contrast with TPipeWriter's 13K lorebook budget planner in `chapterPreValidate` is the real lesson. Two valid answers to the same problem. TPipeWriter budgets the lorebook explicitly because chapter continuity is the product. The manifold validator does not budget at all because context freshness is the product. Ten Trillion Triangles TPipe lets each project pick its own overflow policy. ## The architectural claim The lorebook agent is a transformation function. The transformation function is a hook on the pipe. The pipe runs in the same coroutine that produced the agent's output. The hook fires after the output. The hook reads the bank. The hook writes the bank. The hook is what makes Ten Trillion Triangles TPipe agents update their own memory in real time. The LLM is the mouth. The substrate is the brain. The pipe is the loop. The transformation function is where the brain writes its own memory while the mouth is still forming the next sentence. The agent's memory is the agent's. The pipe gives the hook. The bank is the persistence. What the writer does with the hook is up to the writer. Autogenesis extracts narrative entities. TPipeWriter builds character bibles. TStep indexes codebases. Same hook, three writers, one architecture.