InstructSum: A Benchmark to Evaluate Instruction-Following Capability of Large Language Models in Summarization

Generation of Instruction and Preference Dataset for Improving Japanese Instruction Following in LLMs

Beyond Scores: Understanding LLM-as-a-Judge Mechanisms in Summarization Evaluation

This work investigates LLM-based evaluators of natural language generation quality mechanistically through an eight-attack perturbation taxonomy across the Readability and Adequacy dimensions of NLG quality, a generation pipeline that produces paired clean and corrupt summaries with controlled error intensity and expli...

Synthetic Instruction Generation for Low-Resource Nordic Languages: Viability and Limitations in LLM Instruction-Tuning

DKL: Decoupled Knowledge Learning for Instruction-Tuned Language Models

RAG has become the de facto method for incorporating new, corpus-specific knowledge into an instruction following LLM (Instruct LLM). Although RAG-based prompting improves factual grounding, it fails when retrieval is incorrect or incomplete, leading to hallucinations. Finetuning methods such as RAFT and PA-RAG enhance...

Assessing Readability with LLMs: The Role of Reasoning and Few-Shot Prompting

A systematic benchmark of diverse open-source LLMs for multilingual readability assessment is conducted, focusing on the prediction of discrete readability levels required by educational frameworks, to establish the viability of out-of-the-box LLMs as robust, cross-lingual readability assessors.

STAR: instruction tuning for Arabic across tasks, datasets, and models

An in-depth evaluation of instruction tuning for Arabic NLP tasks using three prominent LLMs: LLaMA 3.1-8B, AceGPT-v2-8B, and Qwen3-8B shows that instruction tuning consistently improves performance across most tasks, with notable variations in effectiveness across different tasks and prompts.