Skip to main content
The Pipeline Pattern is a powerful orchestration approach that organizes agents into a linear sequence where each agent performs a specific transformation on data before passing it to the next agent in the chain. This pattern mirrors traditional assembly lines or data processing pipelines where information flows through a series of specialized stages, with each stage adding value or transforming the data in some way. Like an assembly line in manufacturing, the Pipeline Pattern maximizes efficiency by allowing each agent to focus exclusively on its specialized task.

Key Characteristics

Pipeline Pattern The Pipeline Pattern creates a streamlined, sequential processing flow where data moves through a series of specialized transformations. Each agent in the pipeline receives input from its predecessor, performs its specific processing, and passes the result to its successor. This approach simplifies complex workflows by breaking them into discrete, manageable stages.
  • Specialized Stages: Each agent in the pipeline specializes in a specific type of data transformation or processing task, focusing exclusively on its area of expertise.
  • Unidirectional Flow: Information typically flows in one direction from the beginning to the end of the pipeline, with clear handoffs between stages.
  • Progressive Refinement: Data is progressively refined, enhanced, and transformed as it moves through each stage, with each agent building upon the work of previous agents.
  • Well-Defined Interfaces: Each agent has clear expectations about the format and content of both its inputs and outputs, ensuring smooth transitions between stages.

Information Flow

Pipeline Flow Pattern In the Pipeline Pattern, information flows along a predetermined path, creating a predictable sequence of transformations. The pattern enforces a disciplined approach to data processing, where each stage contributes its specific expertise to the evolving data. This linear progression ensures that data undergoes all necessary transformations in the correct order, with each agent focusing solely on its specialized processing. The workflow begins with raw input data entering the first stage of the pipeline. Each agent applies its specific transformation before passing the enriched data to the next agent. This continues until the data reaches the final stage, which produces the fully processed output. If an error occurs at any point, the pipeline can terminate early, with error information flowing back to the user rather than proceeding to subsequent stages.
  • Sequential Progression: Data moves through a predetermined sequence of processing stages, with each agent receiving, transforming, and passing data forward.
  • Transformation Focus: Each stage applies its specialized expertise to the data, progressively refining it toward the final output.
  • Accumulative Value: Each transformation builds upon previous ones, with the data becoming more refined and complete as it progresses through the pipeline.

Implementation

Our implementation using AG2’s Swarm demonstrates the Pipeline Pattern in an e-commerce order processing workflow. We’ve created a linear sequence of specialized agents that process an order from initial receipt through notification. The implementation showcases how complex workflows can be broken into discrete, manageable stages with clear responsibilities and interfaces. The pipeline begins with an Entry Agent that receives the order data and initiates processing. The order then flows sequentially through specialized stages: Validation checks the order’s correctness, Inventory verifies item availability, Payment processes the transaction, Fulfillment prepares shipping instructions, and finally, Notification sends confirmation to the customer. Each agent focuses exclusively on its stage of the process, applying its specialized transformation before passing the enhanced data to the next agent.
  • Structured Data Models: Pydantic models define the input/output for each stage, creating clear contracts between pipeline stages and ensuring type safety.
  • Stage-Specific Functions: Each processing stage has dedicated functions that transform the input, update the shared context, and determine whether to proceed or terminate with an error.
  • Strategic Handoffs: AfterWork handlers and context variables manage transitions between agents, with SwarmResult directing the flow based on each stage’s outcome.
The Pipeline Pattern excels at tasks requiring sequential processing with progressive refinement, particularly when the output of one stage naturally serves as the input to the next. This makes it ideal for workflows like document processing, approval sequences, and e-commerce transactions where data needs to undergo a series of distinct transformations in a specific order.

Agent Flow

Code

In this code example we use OpenAI’s GPT-4o mini with structured outputs.We also set the LLM parameter parallel_tool_calls to False so that our agents don’t recommend more than one tool call at a time. This parameter may not be available with all model providers.

Output