Artifacts of "Grounding Large Language Models in a Symbolic Formalization Environment for Natural-Language Planning"

Abstract

Experimental results for the paper "Grounding Large Language Models in a Symbolic Formalization Environment for Natural-Language Planning". The archive contains the results of 5,100 formalization sessions. Qwen 3.5 (122B) and GPT-OSS (120B) each formalized the 2,550 Blocksworld, Logistics and Rovers problems of the benchmark corpus (DOI 10.5281/zenodo.20094410). Each model was evaluated with the proposed procedural interface to a Unified Planning environment and with a one-shot text-to-PDDL baseline. Contents: results_proposed_method/: one folder per domain and model. Each instance folder holds the complete chat history, with tool calls, environment replies, planner outputs and plans, and the final formalization as PDDL. A results.csv file per domain and model reports formalization accuracy, plan validity, tokens, runtime and tool calls. results_baseline_method/: the PDDL problems generated in one shot, with a results.csv file per domain and model. json_schemas/: the JSON contracts used to communicate the domain and the problem state to the model. prompt_system.md and prompt_verification.md: the system prompt and the self-verification prompt. formalization_errors.md: the 21 inaccurate formalizations of the proposed method. rejected_tool_call_examples.md: examples of tool calls the environment rejected, with the returned feedback. figures/: the JSON representations and two example conversations.