Reducing Context and Generation Traffic in a Local-First LLM Orchestrator: A Transferability Case Study

Abstract

This technical note reports a bounded transferability experiment in the Local Frontier Orchestrator (LFO), a local-first conversational architecture combining a small local language model with selective escalation to a stronger remote model. The optimization direction was materially informed by Daniel Correa Villa's Pushing Four Raspberry Pis to the Memory Wall. Rather than reproducing the Raspberry Pi experiment, this work evaluates whether its broader architecture-first principle — eliminating unnecessary computation and data/context movement before relying on hardware-specific tuning — transfers to a substantially different environment. Two selectively activated LFO optimizations were evaluated: a compact read schema that removes unused model output fields on protected read-only turns, and a lean read policy that removes write-extraction instructions from the corresponding input context. In a four-observation correctness-gated pilot, input decreased from 1756 to 1210 tokens and mean output from 173 to 24 tokens. All four answers were correct and persistent L1/L2 memory remained unchanged. Mean observed complete-turn time decreased from 159.629 s to 24.791 s, although unequal cache and model-load conditions prevent interpreting the exact latency difference as a general or statistically established speedup. The principal result is therefore architectural rather than numerical: eliminating model-visible work that is known in advance to be unnecessary substantially reduced prompt and generation traffic while preserving behavior within the tested boundary. The report also documents negative and deferred variants, including an aggressively shortened extraction policy, grouped extraction output, and an alternative placement of retrieved evidence. Source code, frozen implementation state, experimental records, and reproducibility materials are available in the public Local Frontier Orchestrator repository.