Back to Systems
Multi-Agent Framework

SwarmFlow

An open-source framework for orchestrating multi-agent AI workflows — leader–worker architecture, real-time monitoring, and provider-agnostic LLM support.

Problem

Complex tasks — research synthesis, multi-step reasoning, parallel data processing — exceed what a single LLM call can reliably handle. Multi-agent systems are the answer, but most existing frameworks are either too opinionated, poorly observable, or tightly coupled to a single LLM provider.

Building a production-ready multi-agent system requires: a clear agent topology, task decomposition logic, state management across agents, visibility into what each agent is doing, and the ability to swap LLM providers without rewriting the system.

Solution

SwarmFlow is a framework built on LangGraph that implements a leader–worker agent topology. A Leader agent receives the high-level task, decomposes it into subtasks, and delegates to specialized Worker agents. A FastAPI + WebSockets monitoring dashboard provides real-time visibility into agent state, task progress, and inter-agent messages. LLM provider is a pluggable configuration — works with OpenAI, any OpenAI-compatible API, or Ollama for fully local execution.

Agent Architecture

Topology: Leader–Worker
Leader Agent
  • Receives the high-level task from the user
  • Decomposes into atomic subtasks
  • Assigns subtasks to worker agents
  • Aggregates results and synthesizes final output
  • Handles retry and fallback on worker failure
Worker Agents
  • Receive a single scoped subtask from the Leader
  • Execute independently with their own tools
  • Return structured output with status and result
  • Stateless — each invocation is isolated
  • Configurable per worker: model, tools, system prompt
LangGraph State Machine

The agent pipeline is implemented as a LangGraph StateGraph. Each node is an agent function; edges represent conditional routing based on agent output. State is passed as a typed dict across all nodes — full audit trail of every agent decision.

Task InputLeaderWorker[n]AggregatorOutput

Real-Time Monitoring

A FastAPI + WebSockets server broadcasts agent state events to a live dashboard. This is one of the key differentiators from other frameworks — you can watch agent execution unfold in real time.

Agent state stream

Every agent transition — start, tool call, completion, error — emitted as a WebSocket event.

Task progress tracker

Overall task progress calculated from subtask completion ratio. Live percentage shown in dashboard.

Inter-agent messages

Full message history between Leader and Workers stored in state and visible in the monitoring UI.

LLM Provider Abstraction

Provider configuration is a single environment variable. No code changes needed to switch between cloud and local execution.

OpenAI / Compatible

Any model via OpenAI-compatible API endpoint — OpenAI, Groq, DeepSeek, OpenRouter.

Ollama (Local)

Run fully local with Llama 3, Mistral, or any Ollama-supported model. No data leaves the machine.

Per-agent config

Each Worker agent can use a different model — e.g., fast model for triage, stronger model for synthesis.

Challenges

Challenge

State consistency across agents

Solution

LangGraph's typed StateDict passed as immutable snapshot to each node — workers read their input, return output, Leader merges. No shared mutable state.

Challenge

Task decomposition quality

Solution

Leader agent uses structured output (JSON) for subtask definitions — each subtask has a schema-validated format: id, description, required_tools, expected_output_type.

Challenge

WebSocket scalability

Solution

Monitoring dashboard is a separate lightweight service — runs independently and subscribes to the event bus. Doesn't block or slow the agent execution path.

Challenge

Worker failure handling

Solution

Each Worker invocation wrapped in retry logic with exponential backoff. Leader receives a structured error result and can reassign or skip the subtask.

Tech Stack

Orchestration
LangGraphLangChain
Backend
PythonFastAPIWebSockets
LLM Providers
OpenAIOllamaGroqOpenRouter
Infrastructure
DockerDocker Compose

Future Improvements

  • Persistent task history with SQLite/Postgres — replay and audit past runs
  • Dynamic worker pool — spawn worker agents on demand based on Leader's task decomposition
  • Tool registry — centralized tool definitions shared across all workers
  • Human-in-the-loop step — pause for approval before executing high-stakes subtasks
  • Web-based visual graph editor for defining agent topologies without code