Democratizing Autonomous Deep Research: A Neuro-Symbolic Framework for Small Language Models via Grammar-Constrained Decoding
The Agentic AI model consists of systems that can plan on their own, use tools and perform multi-step tasks. These capabilities have mainly been accomplished by large models, such as GPT-4 or Claude 3, are large language models LLMs have demonstrated skill with multi-step reasoning tasks. they have high computational requirements and increased latency. require considerable data transfer to centralized servers to them unsuitable for private, edge-based use. Open-source Small Models that implement SLMs have less than 13 billion parameters, a possible option for local execution. In agentic situations, however, these predictions are unreliable. However, they do not have a strictly defined syntax for calling tools interfaces, such as JSON schemas. They can also create other functions for long-sighted tasks exist or disappear when leading to meaningful failures.We designed a Neuro-Symbolic architecture for deterministic control layer to improve the frozen, quantized SLMs. Our proach consists of two major components: [1] a recursive "Critic-Planner" feedback loop that filters out noisy retrieval results before they affect the agent’s limited working memory. [2] At inference-time, Grammar-Constrained Decoding (GBNF) focuses on conforming to JSON schema at the logit level, enforcing syntactic correctness of tool interactions.We apply our framework to the quantized Llama-3-8B. Mistral-7B and Phi-3-mini within six domains. including technical analysis to ensure legal compliance. thesis. Our system achieved a 100% syntactic accuracy baseline correctness and completed complex tasks effectively. In contrast, strong industry standards, including LangChain ReAct, failed. to format issues.