MultimodalContent Class API

Table of Contents

• Overview
• Public Properties
  • Core Content
  • Pipeline Control
  • Context Management
  • Metadata & State
• Public Functions
  • Content Management
  • Pipeline Control
  • Content Queries
  • Pipeline Navigation
  • Utilities

Overview

The MultimodalContent class represents content that can contain both text and binary data, along with pipeline control flags and context information.

@Serializable
class MultimodalContent(
    var text: String = "",
    var binaryContent: MutableList<BinaryContent> = mutableListOf(),
    var terminatePipeline: Boolean = false,
    var context: ContextWindow = ContextWindow(),
    var miniBankContext: MiniBank = MiniBank(),
    var tools: PcPRequest = PcPRequest(),
    @Transient var modelReasoning: String = "",
    var useSnapshot: Boolean = false
)

Public Properties

Core Content

text

var text: String = ""

Primary text content passed to LLM prompts.

binaryContent

var binaryContent: MutableList<BinaryContent> = mutableListOf()

List of binary content (images, documents, etc.) that the LLM can process. Transformed based on model support.

modelReasoning

@Transient var modelReasoning: String = ""

Model reasoning/thinking content. Transient - not serialized but preserved during pipeline execution.

Pipeline Control

terminatePipeline

var terminatePipeline: Boolean = false

Signals critical failure forcing pipeline termination. Set by validation checks and functions.

repeatPipe

@Transient var repeatPipe: Boolean = false

If true, current pipe will be called again with the same content until set to false. Enables multi-step tasks and reasoning modes.

passPipeline

@Transient var passPipeline: Boolean = false

Allows pipeline to exit early without error. Useful when additional steps are unnecessary.

skipReasoningPipe

@Transient var skipReasoningPipe: Boolean = false

When true, the reasoning pipe’s output is discarded before injection. The parent pipe’s main LLM call proceeds as if no reasoning pipe was attached. Useful for conditional reasoning where runtime state determines whether reasoning output should be used.

Context Management

context

var context: ContextWindow = ContextWindow()

Context window from TPipe context system. Allows manipulation in validation functions and determines context propagation.

miniBankContext

var miniBankContext: MiniBank = MiniBank()

Multi-page context object for complex scenarios requiring multiple context domains.

tools

var tools: PcPRequest = PcPRequest()

PCP requests generated by the LLM. Preserved when constructing converse data objects.

Metadata & State

useSnapshot

var useSnapshot: Boolean = false

If true, creates a copy in metadata before LLM processing for state restoration. Doubles memory usage.

pipeError

@Transient var pipeError: PipeError? = null

Error information from a failed pipe. Automatically populated when a pipe fails during execution. Contains exception, event type, phase, pipe name, pipe ID, and timestamp.

metadata

@Transient var metadata: MutableMap<Any, Any> = mutableMapOf()

Generic metadata storage for validation, transformation, and branch functions. Acts as workspace for complex logic.

currentPipe

@Transient var currentPipe: Pipe? = null

Reference to pipe currently processing this content. Useful for pipe templating in branch functions.

Public Functions

Content Management

addText(additionalText: String)

Adds text to existing content with proper formatting.

Behavior: If existing text is empty, sets text directly. Otherwise, appends with double newline separator for readability.

addBinary(content: ByteArray, mimeType: String, filename: String? = null)

Adds binary content from byte array.

Behavior: Wraps byte array in BinaryContent.Bytes and adds to binaryContent list.

addBinary(content: BinaryContent)

Adds pre-constructed binary content.

hasError(): Boolean

Extension function to check if content has an error.

Returns: true if pipeError is not null, false otherwise.

Example:

val result = pipe.execute(content)
if (result.hasError()) {
    println("Pipe failed: ${result.pipeError?.message}")
}

merge(other: MultimodalContent)

Merges another MultimodalContent object into this one.

Behavior: Complex merge operation:

• Text: Appends other’s text using addText() formatting
• Binary Content: Replaces with other’s content if other has any
• Model Reasoning: Replaces with other’s reasoning if other has any
• Contexts: Merges both context and miniBankContext using their merge methods
• Metadata: Copies all key-value pairs from other, overwriting existing keys

---

Pipeline Control

terminate()

Marks content to terminate the pipeline.

Behavior: Sets terminatePipeline = true, causing pipeline to stop execution immediately.

repeat()

Marks current pipe to repeat execution.

Behavior: Sets repeatPipe = true, causing the current pipe to be called again with the same content until this flag is cleared.

skipReasoning()

Marks content to skip reasoning pipe injection.

Behavior: Sets skipReasoningPipe = true. When active, injectTPipeReasoning returns immediately and no reasoning content is added to the parent pipe’s prompt. The reasoning pipe still executes — only injection is suppressed.

Example:

pipe.setTransformationFunction { content ->
    if (content.metadata["semanticCompressionApplied"] != true) {
        content.skipReasoning()
    }
    content
}

See Also: Skipping Reasoning Pipes

---

Content Queries

isEmpty(): Boolean

Checks if content is empty.

Behavior: Returns true if both text is empty and binaryContent list is empty.

hasBinaryContent(): Boolean

Checks if content contains binary data.

shouldTerminate(): Boolean

Checks if pipeline should terminate.

Behavior: Returns true if terminatePipeline is true OR content is empty (via isEmpty()).

getImages(): List<BinaryContent>

Extracts all image content from binary data.

Behavior: Filters binaryContent for items with MIME types starting with “image/”.

getDocuments(): List<BinaryContent>

Extracts all document content from binary data.

Behavior: Filters binaryContent for items with MIME types starting with “application/”, “text/plain”, or “text/html”.

---

Pipeline Navigation

skipToNextPipe()

Marks content to skip to the next pipe in sequence.

Behavior: Sets internal jumpToPipe to reserved value “skip-to-next-pipe” for pipeline navigation.

jumpToPipe(pipeName: String)

Marks content to jump to a specific named pipe.

Behavior: Sets target pipe name for pipeline jumping. Pipeline continues forward from jump destination even if jumping backwards. Throws IllegalArgumentException if pipeName is “skip-to-next-pipe” (reserved).

getJumpToPipe(): String

Returns the current jump target pipe name.

clearJumpToPipe()

Clears the jump target.

Behavior: Called by Pipeline class after jump execution to reset navigation state.

---

Utilities

copyMultimodal(): MultimodalContent?

Creates a deep copy of this content object.

Behavior: Uses serialization/deserialization for deep copying. Returns null if serialization fails. Useful for snapshots and state preservation.

getSnapshot(): MultimodalContent?

Retrieves snapshot from metadata if present.

Behavior: Helper function that extracts snapshot stored in metadata["snapshot"] by the pipe execution system when useSnapshot = true.

Next Steps

• ContextWindow API - Continue into memory and context operations.