Evolutionary Optimization of Instruction-Tuned Large Language Model Inference Parameters for Spanish Text Simplification

Abstract

Large Language Models (LLM) have become valuable tools for automatic text simplification, yet their output quality is highly sensitive to inference-time parameters such as temperature, repetition penalty, top-p, and top-k. These parameters are typically set heuristically rather than systematically optimized. In this work, we apply Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to identify high-performing inference configurations for an instruction-tuned Large Language Model (LLM) fine-tuned for Spanish Text Simplification (TS) on the Financial Education Corpus IN SpAnish (FEINA) benchmark. Using System output Against References and against the Input sentence (SARI) as the optimization objective and Sentence Bidirectional Encoder Representations from Transformers (SBERT) as a monitoring metric, CMA-ES converges in 9 of 25 generations, improving test-set SARI from 36.57 (default) to 38.87 (+6.3%) while increasing SBERT from 0.85 to 0.88. Grid search analysis reveals that repetition penalty and temperature are the dominant factors influencing simplification quality, while top-k and maximum tokens have negligible effects. The strong positive correlation between SARI and SBERT across all 2,500 evaluated configurations provides empirical support for the single-objective approach within the studied setting. Our framework offers a replicable methodology for inference parameter optimization applicable to Spanish and other under-resourced languages in Latin American Natural Language Processing (NLP) contexts.