Multidimensional Evaluation of LLM-Generated MPC Programs

Abstract

Large language models can generate programs that compile and pass functional tests, but these outcomes are insufficient for evaluating code in secure multi-party computation (MPC). MPC programs must also follow the target interface's programming constraints, implement the requested algorithm, and satisfy the stated secure-operation efficiency criteria. Using NLMPC, a staged code-generation framework, we conduct an empirical study of 3,600 C++ programs generated by three LLMs for three algorithms under four generation settings that vary API instructions, explicit programming constraints, and iterative repair. We evaluate each program along four dimensions: functional correctness, MPC conformance, algorithmic adherence, and MPC efficiency. Although 59.2% of the generated programs passed the functional correctness tests, only 31.0% satisfied all four criteria. Functional testing alone did not establish conformance, adherence to the requested algorithm, or compliance with the stated efficiency criteria. Adding MPC-specific constraints revealed trade-offs: aggregate functional correctness and MPC conformance increased, while algorithmic adherence decreased. Validation-guided repair had a higher observed overall pass rate than single-pass generation, but this improvement was not statistically significant after correction. Its recorded token use, including LLM-validator calls, was 1.77×-3.19× that of single-pass generation. These findings show that passing functional tests does not establish the overall quality of LLM-generated MPC code, and that additional constraints and repair do not necessarily improve all evaluation dimensions. They motivate separate checks for functional behavior, MPC programming constraints, algorithmic invariants, and source-level efficiency, rather than relying on compilation or functional testing alone.