Elatify
Back to Think
Systems Engineering

Leaked Claude Code Deep Dive: Streaming Tool Execution for Agentic Systems

If your agent feels slow, the bottleneck is often not the model. It’s the orchestration: waiting for a complete response before doing any work, then dumping results at the end. Streaming tool execution removes dead time by overlapping inference and I/O.

8 min read
April 2, 2026

Start tools as soon as tool blocks appear

Don’t wait for a full model response. Begin executing tools the moment the stream yields a tool request.

Bound concurrency with “safe to run together” rules

Some tools can run concurrently (reads), others must serialize (writes). Enforce a simple concurrency predicate.

Stream progress, not just final results

Expose intermediate progress so users (and orchestration layers) see movement instead of a frozen UI.

Contain failures with sibling aborts

If one tool failure makes its siblings invalid (e.g., shell pipelines), cancel siblings without killing the whole turn.

Concurrency predicate (fail-closed)
A minimal rule: you can execute concurrently only if the new tool is concurrency-safe and every running tool is also concurrency-safe.
type RunningTool = { isConcurrencySafe: boolean; status: 'executing' | 'done' };

function canExecute(nextIsSafe: boolean, running: RunningTool[]) {
  const executing = running.filter((t) => t.status === 'executing');
  if (executing.length === 0) return true;
  if (!nextIsSafe) return false;
  return executing.every((t) => t.isConcurrencySafe);
}
Error containment: abort siblings, keep the turn alive
In enterprise workflows, partial failure handling matters. A common pattern is “abort siblings” when a tool failure makes downstream tools meaningless, while preserving the parent turn for recovery.
async function runToolsWithSiblingAbort(tools: ToolCall[]) {
  const siblingAbort = new AbortController();

  try {
    return await Promise.all(
      tools.map((t) => executeTool(t, { signal: siblingAbort.signal }))
    );
  } catch (e) {
    // if tool A fails and tools are implicitly dependent, abort siblings
    siblingAbort.abort();
    // parent can recover or re-plan without crashing the whole session
    throw e;
  }
}

Need agent workflows that feel instant?

Elatify designs low-latency, production-ready agent architectures with observability and guardrails.

Talk to ElatifyAgentic workflow design