--- title: "How a Council of AI Agents Orchestrates 100 Concurrent Workers Without Chaos" id: "12418" type: "post" slug: "how-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos" published_at: "2026-08-04T18:01:59+00:00" modified_at: "2026-08-04T18:01:59+00:00" url: "https://lucidhive.com/how-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos/12418/" markdown_url: "https://lucidhive.com/how-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos/12418.md" excerpt: "How a Council of AI Agents Orchestrates 100 Concurrent Workers Without Chaos Every month, a new “multi-agent framework” launches. Every month, the same screenshot: a chat window where one model talks to another model. That is not multi-agent orchestration. That..." taxonomy_category: - "AI & Automation" taxonomy_post_tag: - "ai agents" - "api first" - "autonomous operations" - "Digital Architecture" - "web40" --- SHARE [https://www.facebook.com/sharer.php?u=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F](https://www.facebook.com/sharer.php?u=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F) [https://twitter.com/intent/tweet?text=How+a+Council+of+AI+Agents+Orchestrates+100+Concurrent+Workers+Without+Chaos&url=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F&via=](https://twitter.com/intent/tweet?text=How+a+Council+of+AI+Agents+Orchestrates+100+Concurrent+Workers+Without+Chaos&url=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F&via=) [#](#) [javascript:if(window.print)window.print()](javascript:if(window.print)window.print()) [#](#) # How a Council of AI Agents Orchestrates 100 Concurrent Workers Without Chaos Every month, a new “multi-agent framework” launches. Every month, the same Contents [The claim everyone makes](#the-claim-everyone-makes) [Why BEAM, not threads](#why-beam-not-threads) [The wire protocol that doesn’t lose bytes](#the-wire-protocol-that-doesn8217t-lose-bytes) [The benchmark](#the-benchmark) [What breaks at 1000](#what-breaks-at-1000) [What this means for your stack](#what-this-means-for-your-stack) - Advertisement - screenshot: a chat window where one model talks to another model. That is not multi-agent orchestration. That is a conversation with extra steps. - Advertisement - Real orchestration is a **systems problem**. It asks: how many workers can you actually run *concurrently*, with what latency, and what happens when fifty of them decide they need an answer at the same instant? We built a council of AI - Advertisement - agents — a hierarchy of specialist cores coordinated by a single orchestrator — and we stress-tested the orchestration layer itself. This is what the numbers look like when you stop talking about agents and start measuring them. - Advertisement - ## The claim everyone makes Most teams describe “scale” as: we ran a prompt, it produced output, we called it an agent. Under load, those systems fall apart in three predictable ways: 1. **Process explosion** — one thread per agent, and the OS scheduler becomes - Advertisement - the bottleneck long before the model does. 2. **Port/framing corruption** — subprocesses that talk to each other with naive line-based protocols drop data under concurrency, and the corruption - Advertisement - is silent. 3. **Supervision collapse** — when a worker dies, nothing restarts it, and the parent waits forever on a child that no longer exists. - Advertisement - We designed around all three from the start, using the Erlang/BEAM runtime as the orchestration substrate and a length-prefixed binary framing protocol for inter-process communication. Then we measured it. - Advertisement - ## Why BEAM, not threads The orchestration layer runs on Erlang/OTP, which is not a fashionable choice for AI work — and that is precisely why it works. The BEAM virtual machine is built for exactly this shape of problem: - Advertisement - - **Processes, not threads.** Erlang processes are lightweight (a few KB), isolated, and garbage-collected independently. A single node supports over one million of them. Your “thousand agents” question isn’t a question on BEAM; it’s a Tuesday. - Advertisement - - **Schedulers match cores.** On an M-series Mac, OTP runs one scheduler per core (16 on the Max). The runtime does preemptive scheduling — no worker can starve the others, no matter how long its reasoning loop runs. - **Supervision trees.** Every worker is a child of a supervisor with a restart policy. `simple_one_for_one` means a dying worker is respawned - Advertisement - automatically, with intensity limits that prevent crash-loop cascades. The “who restarts it?” question has a structural answer, not a hope. Each profile actor is a `gen_server` — a stateful process that owns an - Advertisement - identity (id, name, model), a status (`idle | reasoning | responding | error`), and a dedicated subprocess port for its work. Dispatch is by PID, not by name: the orchestrator holds the handles and talks to workers directly. - Advertisement - ## The wire protocol that doesn’t lose bytes The workers are Python subprocesses — one per actor — because that is where the actual model calls and tool work happen. Erlang and Python do not share memory, so they talk over pipes. The naive approach is line-delimited text; - Advertisement - it fails the moment a payload contains a newline, and under concurrency it fails silently. We use **`{packet,4}` framing**: every message is prefixed with a 4-byte - Advertisement - big-endian length. The reader reads exactly four bytes, retries on short reads (POSIX pipes deliver partial data; this is not an edge case, it is the norm), unpacks the length, reads that many bytes, and only then processes. - Advertisement - No delimiters to collide with. No ambiguous boundaries. The cost is negligible; the guarantee is that a 364-byte response arrives as a 364-byte response, always. - Advertisement - ## The benchmark Phase 1 (simulated reasoning, no subprocesses): 1 profile → ~344ms lifecycle; 10 profiles → ~2.7s; **100 profiles → ~29.6s**, zero contention. Phase 2 (real Python subprocess IPC, `{packet,4}` framing): 1 profile → - Advertisement - ~654ms; 10 profiles → ~2.2s; **100 profiles → ~16.9s**, ~150ms average per call, **100/100 success, zero data loss**. Read that second result again: one hundred BEAM processes, each holding an - Advertisement - independent Python subprocess, all reasoning concurrently, all answered, nothing corrupted. The OS scheduler handles the subprocess load; the BEAM handles the orchestration; the framing protocol handles the data integrity. - Advertisement - That is what “the mesh works” looks like in numbers instead of adjectives. ## What breaks at 1000 We are honest about the limits, because the whole point is to design around them: - Advertisement - - **Memory.** Each Python subprocess costs roughly 30MB RSS. At 1000 actors that is ~30GB before the model calls. The fix is a shared port dispatcher — one Python process serving many BEAM actors — instead of per-actor subprocesses. - Advertisement - - **Blocking calls.** The current `gen_server` blocks waiting for the port response. A non-blocking design returns `{noreply, State}` and replies via `gen_server:reply/2` when the port delivers. For high-throughput reasoning, that is the next architectural step. - Advertisement - - **Name registration.** PID-based dispatch is perfect for a prototype; a production mesh wants a name→PID registry (ETS or gproc) so actors can be addressed by role, not by handle. ## What this means for your stack The pattern generalizes beyond our council. If you are running multiple AI - Advertisement - agents in production: - **Put an orchestration substrate under your agents, not just a prompt layer.** The supervisor tree is the difference between “one crashed worker” and “the whole fleet hangs.” - Advertisement - - **Frame your IPC.** If your agents talk to subprocesses over pipes, use length-prefixed framing. Line-delimited protocols will corrupt under load, and silent corruption is worse than failure. - **Measure concurrency, not conversations.** A benchmark that shows 100 concurrent workers completing with zero loss is worth more than a demo of - Advertisement - two models chatting. The council doctrine is simple: all tools flow through one gate, all tasks flow through one board, all agents serve one system. The architecture makes - Advertisement - that doctrine cheap to enforce — and the numbers show it holds at 100 concurrent workers. *This article is grounded in a live internal benchmark (100-profile actor - Advertisement - mesh, OTP 29, M-series). Verifiable numbers, not vibes.* - Advertisement - TAGGED:[ai agents](https://lucidhive.com/tag/ai-agents/) [api first](https://lucidhive.com/tag/api-first/) [autonomous operations](https://lucidhive.com/tag/autonomous-operations/) [Digital Architecture](https://lucidhive.com/tag/digital-architecture/) [web40](https://lucidhive.com/tag/web40/) Share This Article [Facebook](https://www.facebook.com/sharer.php?u=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F) [https://twitter.com/intent/tweet?text=How+a+Council+of+AI+Agents+Orchestrates+100+Concurrent+Workers+Without+Chaos&url=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F&via=](https://twitter.com/intent/tweet?text=How+a+Council+of+AI+Agents+Orchestrates+100+Concurrent+Workers+Without+Chaos&url=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F&via=) [Copy Link](#) [Print](javascript:if(window.print)window.print()) [#](#) Share [https://www.facebook.com/sharer.php?u=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F](https://www.facebook.com/sharer.php?u=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F) [https://twitter.com/intent/tweet?text=How+a+Council+of+AI+Agents+Orchestrates+100+Concurrent+Workers+Without+Chaos&url=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F&via=](https://twitter.com/intent/tweet?text=How+a+Council+of+AI+Agents+Orchestrates+100+Concurrent+Workers+Without+Chaos&url=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F&via=) [#](#) [javascript:if(window.print)window.print()](javascript:if(window.print)window.print()) [#](#) 00votes Article Rating Subscribe [Login](https://lucidhive.com/wp-login.php?redirect_to=https%3A%2F%2Flucidhive.com%2Fhow-a-council-of-ai-agents-orchestrates-100-concurrent-workers-without-chaos%2F12418%2F) 0 Comments OldestNewestMost Voted