Large Language Models (LLM) have become valuable tools for automatic text simplification, yet their output quality is highly sensitive to inference-time parameters such as temperature, repetition penalty, top-p, and top-k. These parameters are typically set heuristically rather than systematically optimized. In this work, we apply Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to identify high-performing inference configurations for an instruction-tuned Large Language Model (LLM) fine-tuned for Spanish Text Simplification (TS) on the Financial Education Corpus IN SpAnish (FEINA) benchmark. Using System output Against References and against the Input sentence (SARI) as the optimization objective and Sentence Bidirectional Encoder Representations from Transformers (SBERT) as a monitoring metric, CMA-ES converges in 9 of 25 generations, improving test-set SARI from 36.57 (default) to 38.87 (+6.3%) while increasing SBERT from 0.85 to 0.88. Grid search analysis reveals that repetition penalty and temperature are the dominant factors influencing simplification quality, while top-k and maximum tokens have negligible effects. The strong positive correlation between SARI and SBERT across all 2,500 evaluated configurations provides empirical support for the single-objective approach within the studied setting. Our framework offers a replicable methodology for inference parameter optimization applicable to Spanish and other under-resourced languages in Latin American Natural Language Processing (NLP) contexts.
Nelson Pérez-Rojas, Danny Xie-Li, M. Solís et al.· Proceedings of the Genetic a...· 0 citations
We address the challenging problem of multiobjective optimization via stochastic simulation over a discrete design space. We consider the setting where objective functions are expensive black-box simulations corrupted by heteroscedastic noise, and the decision space is usually too large for exhaustive enumeration. Existing methods often struggle to balance three competing needs: scalable surrogate modeling on discrete domains, principled handling of simulation noise (specifically regarding the uncertainty of the current best solution), and efficient navigation of the multiobjective landscape. Our proposed framework extends the single-objective Complete Expected Improvement acquisition function to the bi-objective case. Our contribution is threefold: (1) we employ Gaussian Markov Random Field surrogates to exploit the integer lattice structure; (2) we use ParEGO-style scalarizations but restrict them to linear to preserve the Gaussianity of the posterior, allowing us to derive a closed-form scalarized acquisition function that explicitly accounts for the covariance between the candidate solution and the noisy incumbent. (3) To maximize this acquisition function, we integrate a discrete Genetic Algorithm with specialized local and jump mutation operators as the inner optimizer. We benchmark our approach against an adaptation of state-of-the-art methods on noisy variants of standard test functions, showing faster early convergence while retaining computational tractability.
Sebastián Rojas Gónzalez, I. Couckuyt, Joshua Knowles· Annual Conference on Genetic...· 0 citations