The Maskability Index: Predicting Task-Objective Alignment in Pretrained Language Models

Ahmad Pouramini Mahsa Afsharzadeh

Jul 2026 · arXiv.org · Vol abs/2607.20265 · 0 citations · 30 references

Computer Science

TL;DR

The Maskability Index (MI) is introduced, a quantitative metric that estimates whether a knowledge relation is better suited to masked-style prompting or prefix-style prompting in few-shot generation, providing a principled measure of objective-template alignment.

Abstract

Large-scale pretrained language models such as T5 and BERT have demonstrated strong capabilities for generating structured knowledge. However, their performance depends on how closely the prompting strategy matches the objectives used during pretraining. We introduce the Maskability Index (MI), a quantitative metric that estimates whether a knowledge relation is better suited to masked-style prompting or prefix-style prompting in few-shot generation. MI is computed from differences in DepthRank scores between masked and unmasked templates, providing a principled measure of objective-template alignment. We evaluate MI on a diverse set of relations from the ATOMIC2020 knowledge base completion benchmark and show that it is positively correlated with downstream generation performance. These results indicate that MI can help select appropriate prompting templates and adaptation strategies for extracting relational knowledge from pretrained language models, especially in low-resource settings.

DOI arXiv Semantic Scholar

View source

Similar papers

#artificial intelligence Preprint Sep 2026

Prompt-Robust Language Models: Which Training Strategies Work?

The current robustness fine-tuning methods improve over standard fine-tuning and in-context learning, but the best-to-worst prompt gap remains as high as 40-57% of performance, and recent robustness-enhancing methods often fail to outperform the simplest data construction strategy: training on one template per batch.

Frederic Sadrieh, Michal Štefánik · 0 citations

The Pitfalls of Maximum Likelihood Training of TPMs for Controllable Language Generation

This work finds that likelihood-trained TPMs can result in failed generations due to overly large corrections to the LM’s logits, and trains TPMs with LM-aligned objectives that better align with the LM token-probability space.

Hanzhang Liu, William Zhao, Zi-Lei Shao et al. · 0 citations

Preprint Aug 2026

BengaliMCQ: Automatic Generation and Answer Prediction of Academic Multiple-Choice Questions in a Low-Resource Language

Traditional retrieval-augmented generation (RAG) frameworks process documents without attending to their hierarchical structure, leading to poor performance, especially in low-resource languages such as Bengali. To address this, we propose a structure-aware RAG framework that models Bengali textbooks as hierarchical gr...

Abu Tarabin Surzo, A. K. M. Nihalul Kabir, Sm Azmain Faysal et al. · 0 citations

Open access Aug 2026

Ensemble-Based Approach for Amazigh POS Tagging: Leveraging Multiple Models for Enhanced Performance in Low-Resource Language Processing

The results show that hybrid ensemble methods can improve token-level accuracy in low-resource POS tagging, while also revealing a trade-off between frequent-tag accuracy and rare-tag robustness.

Abdelouahed Moussaoui, Nor-Eddine Azalmad, Said Bahassine et al. · Information · 0 citations

Preprint Aug 2026

SAM3-LoRA: Parameter-Efficient Adaptation of a Concept-Promptable Foundation Model for Multi-Class Structural Defect Segmentation

Promptable segmentation foundation models such as SAM3 accept an open-vocabulary text concept and return every instance matching it, but adapting them to a specialized domain by full fine-tuning is computationally prohibitive for the organizations that would benefit most. This study applies Low-Rank Adaptation (LoRA) t...

P. Malaisree, S. Youwai, S. Janrungautai et al. · 0 citations

Jul 2026

Beyond Scale and Generation: Understanding Language Model-based Entity Matching

The factors underlying performance differences across matcher architectures are clarified and motivate future research and benchmark designs that better disentangle architectural choices from model-level factors while explicitly evaluating distribution shift and cross-dataset transferability.

Zeyu Zhang, Xue Li, Iacer Calixto et al. · arXiv.org · 0 citations