Evaluating large language models on text-based engineering design tasks: performance gaps and failure modes
The study examines Large Language Model-generated answers to 86 text-based questions covering three task categories chosen for their direct correspondence to typical engineering design activities to characterize the specific capabilities that AI-enabled design systems must develop to be reliable in practice and establi...