MineCEraft: Evaluating Language Models as Construction Engineers in the World of Minecraft
The MineCEraft benchmark comprises 723 domain-expert hand-crafted natural-language instructions with programmatically verifiable evaluation, providing a safe and controllable experimental environment for assessing LLMs'ability to perform realistic construction engineering tasks.