how-to ~18 min
Reasoning Pipes

Reasoning Pipes are specialized processing plants. They isolate the LLM's "thinking" (like `<think>` tags) from its final output, sending clean water down the line.

Reasoning Pipes

💡 Tip: Reasoning Pipes are specialized processing plants. They isolate the LLM’s “thinking” (like <think> tags) from its final output, sending clean water down the line.

Table of Contents

Reasoning pipes provide chain-of-thought reasoning capabilities to any AI model, enabling complex problem-solving and step-by-step analysis even on models that don’t natively support reasoning modes. TPipe-Defaults provides proven reasoning builders that create optimized reasoning pipes for different problem types.

What are Reasoning Pipes?

Reasoning pipes are specialized AI models that execute before your main pipe to provide structured thinking and analysis:

  • Chain-of-thought reasoning for step-by-step problem solving
  • Multi-round reasoning for complex analysis
  • Structured reasoning strategies for different problem types
  • Automatic reasoning injection into main pipe processing

The reasoning pipe generates detailed analysis that enhances the main pipe’s decision-making capabilities.

Core Concepts

Reasoning Execution Flow

  1. Input received by main pipe
  2. Reasoning pipe executes first with the input
  3. Reasoning content generated through structured analysis
  4. Reasoning injected into main pipe processing
  5. Main pipe processes with enhanced reasoning context
  6. Final result combines reasoning and main processing

Reasoning Budget and Rounds

import com.TTT.Pipe.TokenBudgetSettings

val mainPipe = BedrockPipe()
    .setSystemPrompt("Solve problems using provided reasoning.")
    .setReasoningPipe(reasoningPipe)
    .setTokenBudget(TokenBudgetSettings(
        reasoningBudget = 2000  // Token budget for reasoning
        // By default, reasoningBudget subtracts from maxTokens. Use subtractReasoningFromInput = true to subtract from the context window instead.
    ))

Note: Reasoning rounds are configured in the ReasoningSettings when building the reasoning pipe, not on the main pipe.

Building Reasoning Pipes with TPipe-Defaults

TPipe-Defaults provides the ReasoningBuilder with proven reasoning strategies:

Structured Chain-of-Thought Reasoning

import Defaults.reasoning.ReasoningBuilder.reasonWithBedrock
import Defaults.reasoning.*
import Defaults.BedrockConfiguration
import com.TTT.Structs.PipeSettings

val bedrockConfig = BedrockConfiguration(
    region = "us-east-1",
    model = "anthropic.claude-3-sonnet-20240229-v1:0"
)

val pipeSettings = PipeSettings(
    temperature = 0.7,
    topP = 0.8,
    maxTokens = 2000
)

val reasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.StructuredCot,
    depth = ReasoningDepth.Med,
    duration = ReasoningDuration.Med,
    reasoningInjector = ReasoningInjector.SystemPrompt,
    numberOfRounds = 2
)

val structuredReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    reasoningSettings,
    pipeSettings
)

val problemSolvingPipe = BedrockPipe()
    .setSystemPrompt("Solve problems systematically using step-by-step analysis.")
    .setReasoningPipe(structuredReasoningPipe)
    .setTokenBudget(TokenBudgetSettings(reasoningBudget = 1500))

Managing the Reasoning Budget

When allocating a reasoningBudget inside TokenBudgetSettings, TPipe by default subtracts this amount from maxTokens (the output budget).

If you are injecting reasoning into the system prompt (ReasoningInjector.SystemPrompt), user prompt (ReasoningInjector.BeforeUserPrompt), or context (ReasoningInjector.AsContext), you might prefer this space to be subtracted from the available context window rather than the final output capability.

To achieve this, set subtractReasoningFromInput = true in your TokenBudgetSettings. This guarantees the allocated reasoning block has reserved space during prompt truncation, without decreasing your maxTokens capability.

Explicit Chain-of-Thought Reasoning

val explicitReasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.ExplicitCot,
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Long,
    reasoningInjector = ReasoningInjector.BeforeUserPrompt,
    numberOfRounds = 1
)

val explicitReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    explicitReasoningSettings,
    pipeSettings
)

val analyticalPipe = BedrockPipe()
    .setSystemPrompt("Analyze problems with explicit step-by-step reasoning.")
    .setReasoningPipe(explicitReasoningPipe)

Process-Focused Chain-of-Thought

val processReasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.processFocusedCot,
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Med,
    reasoningInjector = ReasoningInjector.AfterUserPrompt,
    numberOfRounds = 3
)

val processReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    processReasoningSettings,
    pipeSettings
)

val methodicalPipe = BedrockPipe()
    .setSystemPrompt("Apply methodical reasoning with process focus.")
    .setReasoningPipe(processReasoningPipe)

Reasoning Methods

Structured Chain-of-Thought (StructuredCot)

val structuredSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.StructuredCot,
    depth = ReasoningDepth.Med,
    duration = ReasoningDuration.Med
)

What it does: Uses a formal phase-based framework (analyze→plan→execute→validate) for systematic problem-solving. Provides structured, step-by-step reasoning with clear transitions between logical phases.

Best for: General problem solving, systematic analysis, structured decision making

Explicit Chain-of-Thought (ExplicitCot)

val explicitSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.ExplicitCot,
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Long
)

What it does: Shows detailed step-by-step reasoning with clear transitions between each logical step. Makes the thinking process completely transparent and traceable.

Best for: Complex logical problems, mathematical reasoning, detailed analysis where transparency is crucial

Process-Focused Chain-of-Thought (processFocusedCot)

val processSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.processFocusedCot,
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Med
)

What it does: Focuses on methodological justification and adaptive thinking strategies. Emphasizes the reasoning process itself and how to adapt approaches based on the problem type.

Best for: Methodological problems, process optimization, adaptive problem-solving scenarios

Best Idea Strategy

val bestIdeaSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.BestIdea,
    depth = ReasoningDepth.Med,
    duration = ReasoningDuration.Short,
    reasoningInjector = ReasoningInjector.SystemPrompt
)

val bestIdeaReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    bestIdeaSettings,
    pipeSettings
)

What it does: Asks the AI to come up with what it thinks is the single best idea to solve the problem. Focuses on generating one high-quality solution rather than exploring multiple options.

Use cases: Quick decision making, brainstorming, creative problem solving, time-sensitive decisions

Comprehensive Plan Strategy

val comprehensiveSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.ComprehensivePlan,
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Long,
    reasoningInjector = ReasoningInjector.BeforeUserPrompt
)

val planningReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    comprehensiveSettings,
    pipeSettings
)

What it does: Asks the AI to develop a substantial, comprehensive plan on how it would solve the problem. Creates detailed, multi-step strategies with thorough planning.

Use cases: Strategic planning, project management, complex problem decomposition, long-term planning

Role-Play Reasoning

val rolePlaySettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.RolePlay,
    roleCharacter = "You are an experienced business consultant with 20 years of strategic planning experience.",
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Med,
    reasoningInjector = ReasoningInjector.SystemPrompt
)

val consultantReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    rolePlaySettings,
    pipeSettings
)

What it does: Asks the AI to play as a specific character and act as that character trying to reason through the given task. Applies domain expertise and perspective from the specified role.

Use cases: Domain expertise simulation, specialized knowledge application, perspective-based analysis

Chain of Draft Reasoning

val chainOfDraftSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.ChainOfDraft,
    depth = ReasoningDepth.Med,
    duration = ReasoningDuration.Short,
    reasoningInjector = ReasoningInjector.SystemPrompt
)

val chainOfDraftReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    chainOfDraftSettings,
    pipeSettings
)

What it does: Chain of Draft (CoD) is an innovative prompting technique that revolutionizes model reasoning by using concise, high-signal thinking steps rather than verbose explanations. Each reasoning step is limited to 5 words or less, forcing the model to focus on essential logical components while minimizing unnecessary verbosity. This mirrors how humans solve problems with brief mental notes rather than detailed explanations.

Key Innovation: CoD recognizes that most reasoning chains contain high redundancy. By distilling steps to their semantic core, it helps models focus on logical structure rather than language fluency, achieving the same reasoning quality with dramatically fewer tokens.

Performance Benefits:

  • Up to 75% reduction in token usage compared to Chain-of-Thought
  • Over 78% decrease in latency while maintaining accuracy
  • Cleaner output for downstream parsing and automation
  • Significant cost savings in production environments

Example Comparison:

  • Chain-of-Thought: “Jason had 20 lollipops. He gave some to Denny and now has 12 left. So he gave away 8.”
  • Chain-of-Draft: “Start: 20, End: 12, 20 – 12 = 8.”

Best for: Mathematical calculations, logical puzzles, structured reasoning tasks, cost-sensitive applications, real-time systems where latency matters, scenarios requiring minimal token usage

When to avoid: Zero-shot scenarios (works best with few-shot examples), tasks requiring high interpretability (legal/medical), small language models (<3B parameters), creative or open-ended tasks where elaboration adds value

Semantic Decompression Reasoning

val decompressionReasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.SemanticDecompression,
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Long,
    reasoningInjector = ReasoningInjector.BeforeUserPrompt
)

val decompressionReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    decompressionReasoningSettings,
    pipeSettings
)

What it does: Uses the official semantic-decompression reasoning schema to inspect legend-backed compressed prompts, expand the short codes, identify the parent task, and produce structured decompression reasoning with quote, sentence, and paragraph reconstruction fields that the parent pipe can unwrap into plain thought-stream text.

Best for: Semantic-compression round trips, legend expansion, prompt reconstruction, and live regressions that need to prove the decompression reasoning path is wired end-to-end.

Reasoning Injection Methods

Configure how reasoning content is injected into the main pipe:

System Prompt Injection

val systemPromptSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.StructuredCot,
    reasoningInjector = ReasoningInjector.SystemPrompt
)

How it works: Reasoning is injected at the end of the system prompt

Before User Prompt Injection

val beforeUserSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.ExplicitCot,
    reasoningInjector = ReasoningInjector.BeforeUserPrompt
)

How it works: Reasoning appears before the user’s input

After User Prompt Injection

val afterUserSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.processFocusedCot,
    reasoningInjector = ReasoningInjector.AfterUserPrompt
)

How it works: Reasoning appears after the user’s input

Converse History Injection

val converseSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.StructuredCot,
    reasoningInjector = ReasoningInjector.BeforeUserPromptWithConverse
)

How it works: Reasoning is injected into a ConverseHistory block

Context Injection

val contextSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.ComprehensivePlan,
    reasoningInjector = ReasoningInjector.AsContext
)

How it works: Reasoning is injected as context to a designated page key

Multi-Round Reasoning with Focus Points

What are Focus Points?

Focus points are specific instructions that guide the reasoning pipe’s attention during each round of multi-round reasoning. In the modern round-directive flow, each round can also declare a mode:

  • Blind: the harness withholds prior round content and gives the model only the original user prompt plus that round’s focus point.
  • Merge: the harness supplies the accumulated flattened thought stream and asks the model to synthesize it into one conclusion.

How focus points work:

  • Each round gets a specific focus area to concentrate on
  • Blind rounds stay isolated from earlier round content in the harness
  • Merge rounds receive prior round output as a single flattened stream for synthesis
  • Enables systematic exploration of complex problems
  • Ensures important considerations aren’t overlooked

Progressive Multi-Round Analysis

import com.TTT.Structs.ReasoningRoundDirective
import com.TTT.Structs.ReasoningRoundMode

val focusPoints = mutableMapOf<Int, String>()
focusPoints[1] = "risk assessment and potential challenges"
focusPoints[2] = "cost analysis and resource requirements"  
focusPoints[3] = "timeline and implementation strategy"

val roundDirectives = mutableMapOf(
    1 to ReasoningRoundDirective(
        focusPoint = "risk assessment and potential challenges",
        mode = ReasoningRoundMode.Blind
    ),
    2 to ReasoningRoundDirective(
        focusPoint = "cost analysis and resource requirements",
        mode = ReasoningRoundMode.Blind
    ),
    3 to ReasoningRoundDirective(
        focusPoint = "timeline and implementation strategy",
        mode = ReasoningRoundMode.Merge
    )
)

val multiRoundSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.StructuredCot,
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Long,
    reasoningInjector = ReasoningInjector.SystemPrompt,
    numberOfRounds = 3,
    focusPoints = focusPoints,          // legacy compatibility
    roundDirectives = roundDirectives   // canonical blind/merge config
)

What happens in each round:

  • Round 1: Blind mode sees only the original user prompt plus “risk assessment and potential challenges”
  • Round 2: Blind mode sees only the original user prompt plus “cost analysis and resource requirements”
  • Round 3: Merge mode sees the accumulated flattened round stream plus “timeline and implementation strategy”

After each round completes, TPipe normalizes that round through the same unravel() path used by single-round reasoning, then appends the flattened block to one cumulative thought stream. The reasoning pipe still sees the round-by-round separation scaffold, but the parent pipe receives the resolved stream, not a serialized history blob. When roundDirectives is empty, the legacy focusPoints path remains available for older callers.

Single-round nested reasoning does not add the ROUND N / FOCUS: wrapper at runtime. Those labels are reserved for true multi-round flows so the injected reasoning stays plain text when a pipe is only doing one pass.

Focus Point Examples by Domain

Business Strategy Focus Points

val businessFocusPoints = mutableMapOf<Int, String>()
businessFocusPoints[1] = "market analysis and competitive landscape"
businessFocusPoints[2] = "financial impact and ROI projections"
businessFocusPoints[3] = "implementation risks and mitigation strategies"
businessFocusPoints[4] = "organizational change management requirements"

Technical Architecture Focus Points

val technicalFocusPoints = mutableMapOf<Int, String>()
technicalFocusPoints[1] = "scalability and performance requirements"
technicalFocusPoints[2] = "security considerations and compliance"
technicalFocusPoints[3] = "integration challenges and dependencies"
technicalFocusPoints[4] = "maintenance and operational complexity"

Creative Innovation Focus Points

val creativeFocusPoints = mutableMapOf<Int, String>()
creativeFocusPoints[1] = "user experience and interface design"
creativeFocusPoints[2] = "market differentiation and unique value proposition"
creativeFocusPoints[3] = "technical feasibility and implementation approach"
creativeFocusPoints[4] = "scalability and future enhancement possibilities"

Focus Points vs Single-Round Reasoning

Without Focus Points (Single Round)

val singleRoundSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.StructuredCot,
    numberOfRounds = 1  // No focus points needed
)

Result: AI provides general reasoning covering all aspects but may not dive deep into specific areas.

With Focus Points (Multi-Round)

val focusedSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.StructuredCot,
    numberOfRounds = 3,
    focusPoints = mapOf(
        1 to "technical feasibility analysis",
        2 to "business impact assessment", 
        3 to "implementation roadmap planning"
    )
)

Result: AI provides dedicated, deep analysis of each focus area while building comprehensive understanding across rounds.

val strategicReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    multiRoundSettings,
    pipeSettings
)

val strategicPipe = BedrockPipe()
    .setSystemPrompt("Make strategic decisions with focused multi-round analysis.")
    .setReasoningPipe(strategicReasoningPipe)
    .setTokenBudget(TokenBudgetSettings(reasoningBudget = 4000))

Cross-Provider Reasoning

Ollama-Based Reasoning

import Defaults.reasoning.ReasoningBuilder.reasonWithOllama
import Defaults.OllamaConfiguration

val ollamaConfig = OllamaConfiguration(
    host = "localhost:11434",
    model = "llama3.1:70b"
)

val ollamaReasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.StructuredCot,
    depth = ReasoningDepth.Med,
    duration = ReasoningDuration.Med,
    reasoningInjector = ReasoningInjector.SystemPrompt
)

val ollamaReasoningPipe = reasonWithOllama(
    ollamaConfig,
    ollamaReasoningSettings,
    pipeSettings
)

val ollamaProblemSolver = OllamaPipe()
    .setSystemPrompt("Solve problems using systematic reasoning.")
    .setReasoningPipe(ollamaReasoningPipe)

Reasoning Depth and Duration Settings

Reasoning Depth

// Light reasoning for simple problems
ReasoningDepth.Low

// Moderate reasoning for standard problems  
ReasoningDepth.Med

// Deep reasoning for complex problems
ReasoningDepth.High

Reasoning Duration

// Quick reasoning for time-sensitive decisions
ReasoningDuration.Short

// Standard reasoning duration
ReasoningDuration.Med

// Extended reasoning for thorough analysis
ReasoningDuration.Long

Skipping Reasoning Pipes

TPipe allows you to conditionally skip reasoning injection at runtime using the skipReasoning() method on MultimodalContent. When the skip flag is active, the reasoning pipe’s output is discarded and never injected into the parent pipe’s prompt — the system behaves as if no reasoning pipe was attached.

When to Skip Reasoning

Skipping reasoning is useful when:

  • Semantic compression didn’t fire — a reasoning pipe configured for decompression has nothing to decompress if the token budget didn’t need to compress the prompt
  • Conditional reasoning — a developer-in-the-loop function determines at runtime that reasoning is unnecessary for a particular input
  • Cost control — avoiding injection overhead for simple inputs that don’t benefit from chain-of-thought analysis

Basic Usage

// In a validation function, transformation function, or branch function
pipe.setTransformationFunction { content ->
    if (inputIsSimpleEnough(content.text)) {
        content.skipReasoning()  // Reasoning output will be discarded
    }
    content
}

Direct Property Access

The skipReasoning() convenience method sets the skipReasoningPipe flag on MultimodalContent:

// Equivalent to content.skipReasoning()
content.skipReasoningPipe = true

Semantic Compression Example

The most common use case is pairing with semantic compression. When a pipe uses SemanticDecompression reasoning to reconstruct compressed prompts, but the token budget didn’t need to compress, the decompression reasoning step should be skipped:

val mainPipe = BedrockPipe()
    .setSystemPrompt("Process the user request.")
    .setReasoningPipe(decompressionReasoningPipe)
    .enableSemanticCompression()
    .setTransformationFunction { content ->
        // If compression never fired, skip the decompression reasoning
        if (content.metadata["semanticCompressionApplied"] != true) {
            content.skipReasoning()
        }
        content
    }

How It Works

  1. The reasoning pipe executes normally and generates its output
  2. Before injection, TPipe checks the skipReasoningPipe flag
  3. If true, injectTPipeReasoning returns immediately — no reasoning content is added to the system prompt, user prompt, or context
  4. The parent pipe’s main LLM call proceeds without any reasoning context

⚠️ Note: The reasoning pipe still executes when skipReasoningPipe is set. The flag prevents injection of the reasoning output, not execution. To skip execution entirely, use setPreInvokeFunction to bypass the whole pipe before it runs.

Practical Examples

Business Decision Making

val businessFocusPoints = mutableMapOf<Int, String>()
businessFocusPoints[1] = "market analysis and competitive landscape"
businessFocusPoints[2] = "financial impact and ROI projections"
businessFocusPoints[3] = "implementation risks and mitigation strategies"

val businessReasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.ComprehensivePlan,
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Long,
    reasoningInjector = ReasoningInjector.BeforeUserPrompt,
    numberOfRounds = 3,
    focusPoints = businessFocusPoints
)

val businessReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    businessReasoningSettings,
    pipeSettings
)

val businessDecisionPipe = BedrockPipe()
    .setSystemPrompt("Make informed business decisions based on comprehensive analysis.")
    .setReasoningPipe(businessReasoningPipe)
    .pullPipelineContext()

// Usage
val decision = runBlocking { businessDecisionPipe.execute("Should we expand into the European market?") }

Technical Problem Solving

val engineerReasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.RolePlay,
    roleCharacter = "You are a senior software architect with expertise in scalable system design.",
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Med,
    reasoningInjector = ReasoningInjector.SystemPrompt,
    numberOfRounds = 2
)

val engineerReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    engineerReasoningSettings,
    pipeSettings
)

val systemDesignPipe = BedrockPipe()
    .setSystemPrompt("Design robust technical systems with expert analysis.")
    .setReasoningPipe(engineerReasoningPipe)

// Usage
val design = runBlocking { systemDesignPipe.execute("Design a microservices architecture for our e-commerce platform") }

Creative Innovation

val creativeFocusPoints = mutableMapOf<Int, String>()
creativeFocusPoints[1] = "user experience and interface design"
creativeFocusPoints[2] = "market differentiation and unique value proposition"
creativeFocusPoints[3] = "technical feasibility and implementation approach"

val creativeReasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.BestIdea,
    depth = ReasoningDepth.Med,
    duration = ReasoningDuration.Med,
    reasoningInjector = ReasoningInjector.AfterUserPrompt,
    numberOfRounds = 3,
    focusPoints = creativeFocusPoints
)

val creativeReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    creativeReasoningSettings,
    pipeSettings
)

val innovationPipe = BedrockPipe()
    .setSystemPrompt("Develop innovative solutions with creative analysis.")
    .setReasoningPipe(creativeReasoningPipe)

// Usage
val innovation = runBlocking { innovationPipe.execute("Create a new mobile app concept for remote team collaboration") }

Mathematical Problem Solving with Chain of Draft

val mathReasoningSettings = ReasoningSettings(
    reasoningMethod = ReasoningMethod.ChainOfDraft,
    depth = ReasoningDepth.Med,
    duration = ReasoningDuration.Short,
    reasoningInjector = ReasoningInjector.SystemPrompt,
    numberOfRounds = 1
)

val mathReasoningPipe = reasonWithBedrock(
    bedrockConfig,
    mathReasoningSettings,
    pipeSettings
)

val mathSolverPipe = BedrockPipe()
    .setSystemPrompt("Solve mathematical problems with concise, step-by-step reasoning using minimal drafts.")
    .setReasoningPipe(mathReasoningPipe)
    .setTokenBudget(TokenBudgetSettings(reasoningBudget = 500)) // Lower budget due to CoD efficiency

// Usage - demonstrates the efficiency gains
val solution = runBlocking { 
    mathSolverPipe.execute("Jason had 20 lollipops and gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops did Jason give to Denny?") 
}

// Expected CoD reasoning output:
// "Start: 20 lollipops
//  End: 12 lollipops  
//  20 - 12 = 8
//  Answer: 8 lollipops"

Why Chain of Draft excels here: Mathematical problems benefit greatly from CoD’s concise approach. Instead of verbose explanations, the model focuses on essential operations and calculations, reducing token usage by up to 75% while maintaining accuracy. This makes it ideal for applications requiring frequent mathematical reasoning at scale.

Nested Reasoning Pipes

TPipe supports unlimited nesting of reasoning pipes, enabling sophisticated multi-layered reasoning architectures for complex problem-solving scenarios.

Nested single-round reasoning follows the same plain-text unravel path as any other single-round reasoning pipe. It does not emit multi-round ROUND N headers unless the nested layer itself is configured for multiple rounds.

Creating Nested Reasoning Structures

// Level 3: Deep analysis reasoning
val deepAnalysis = ReasoningBuilder()
    .setReasoningMethod(ReasoningMethod.StructuredCot)
    .setReasoningRounds(2)
    .build()

// Level 2: Strategic reasoning with nested deep analysis
val strategicReasoning = ReasoningBuilder()
    .setReasoningMethod(ReasoningMethod.ComprehensivePlan)
    .setReasoningRounds(3)
    .build()
    .setReasoningPipe(deepAnalysis)  // Nested reasoning

// Level 1: Main reasoning with nested strategic layer
val mainReasoning = ReasoningBuilder()
    .setReasoningMethod(ReasoningMethod.BestIdea)
    .setReasoningRounds(2)
    .build()
    .setReasoningPipe(strategicReasoning)  // Nested reasoning

// Root pipe with multi-layered reasoning
val rootPipe = BedrockPipe()
    .setSystemPrompt("Solve complex problems using layered reasoning.")
    .setReasoningPipe(mainReasoning)
    .enableTracing()  // Traces all nested levels automatically

Nested Reasoning Execution Flow

  1. Root pipe receives input
  2. Main reasoning executes first
  3. Strategic reasoning executes within main reasoning
  4. Deep analysis executes within strategic reasoning
  5. Results bubble up through each layer
  6. Final output incorporates all reasoning levels

Use Cases for Nested Reasoning

Complex Problem Decomposition

// Research analysis with nested reasoning
val detailAnalysis = ReasoningBuilder()
    .setReasoningMethod(ReasoningMethod.StructuredCot)
    .setReasoningRounds(4)
    .build()

val researchReasoning = ReasoningBuilder()
    .setReasoningMethod(ReasoningMethod.ComprehensivePlan)
    .setReasoningRounds(2)
    .build()
    .setReasoningPipe(detailAnalysis)

val researchPipe = BedrockPipe()
    .setSystemPrompt("Conduct thorough research analysis.")
    .setReasoningPipe(researchReasoning)

Multi-Perspective Analysis

// Different reasoning approaches at each level
val criticalAnalysis = ReasoningBuilder()
    .setReasoningMethod(ReasoningMethod.StructuredCot)
    .setReasoningRounds(3)
    .build()

val creativeThinking = ReasoningBuilder()
    .setReasoningMethod(ReasoningMethod.BestIdea)
    .setReasoningRounds(2)
    .build()
    .setReasoningPipe(criticalAnalysis)

val balancedReasoning = BedrockPipe()
    .setSystemPrompt("Provide balanced analysis combining creativity and critical thinking.")
    .setReasoningPipe(creativeThinking)

Tracing Nested Reasoning

TPipe automatically propagates tracing through all nested reasoning levels:

val pipeline = Pipeline()
    .enableTracing()  // Traces all nested reasoning automatically
    .add(pipeWithNestedReasoning)

val traceReport = pipeline.getTraceReport(TraceFormat.HTML)
// Contains chronological events from all reasoning levels

Performance Considerations

  • Token budgets apply to each reasoning level independently
  • Nested depth should be balanced against performance needs
  • Reasoning rounds multiply across nesting levels
  • Tracing overhead increases with nesting depth

Best Practices for Nested Reasoning

  1. Limit nesting depth to 3-4 levels for optimal performance
  2. Use different reasoning methods at each level for diverse perspectives
  3. Configure appropriate token budgets for each reasoning layer
  4. Enable tracing to monitor nested execution flow
  5. Test thoroughly with representative problem complexity

Best Practices

1. Choose Appropriate Reasoning Method

// Use StructuredCot for systematic problem solving
ReasoningMethod.StructuredCot

// Use BestIdea for quick creative solutions
ReasoningMethod.BestIdea

// Use ComprehensivePlan for strategic planning
ReasoningMethod.ComprehensivePlan

// Use RolePlay for domain expertise
ReasoningMethod.RolePlay

// Use ChainOfDraft for concise mathematical reasoning
ReasoningMethod.ChainOfDraft

2. Match Depth and Duration to Problem Complexity

// Simple problems
ReasoningSettings(
    depth = ReasoningDepth.Low,
    duration = ReasoningDuration.Short
)

// Complex problems
ReasoningSettings(
    depth = ReasoningDepth.High,
    duration = ReasoningDuration.Long
)

3. Use Focus Points for Multi-Round Reasoning

val focusPoints = mutableMapOf<Int, String>()
focusPoints[1] = "problem analysis"
focusPoints[2] = "solution generation"
focusPoints[3] = "implementation planning"

ReasoningSettings(
    numberOfRounds = 3,
    focusPoints = focusPoints
)

4. Select Appropriate Injection Method

// SystemPrompt: For instruction-like reasoning
ReasoningInjector.SystemPrompt

// BeforeUserPrompt: For context-like reasoning
ReasoningInjector.BeforeUserPrompt

// AsContext: For structured context integration
ReasoningInjector.AsContext

Reasoning pipes built with TPipe-Defaults ReasoningBuilder provide proven, structured approaches to complex problem-solving that enhance any AI model’s analytical and decision-making capabilities through systematic chain-of-thought processing.

Next Steps

Now that you understand chain-of-thought reasoning, learn about dynamic pipeline control:

→ Streaming Callbacks - Real-time token streaming with multiple callbacks