TomasuLLM: Out-of-Order Speculative Execution for LLM Agents
Long-running tools can dominate coding-agent latency: compilers, test suites, and repository commands take seconds to minutes while the agent idles. This observation stall presents the same tension that drove out-of-order processors -- asequential interface hides work that can be predicted and started early, but a spec...
Jiang-Nan Yu, Ce-Yu Xu, Meng-Ming Li et al.
· 0 citations