Recognizing out-of-distribution (OOD) samples is essential for deploying robust machine learning systems in open-world environments. While conventional OOD detection approaches rely on feature representations from the penultimate layer of neural networks, they often overlook informative signals embedded in intermediate layers. In this paper, we present a straightforward feature mixing approach for pre-trained Transformers, which combines multi-layer representations via calculated importance weights, and identifies OOD samples using Mahalanobis distance in the blended feature space. When in-distribution samples are accessible, we show that parameter-efficient fine-tuning strategies effectively balance classification accuracy and OOD detection performance. We conduct extensive empirical analyses to validate the superiority of our proposed method under zero-shot, and fine-tuning settings using both class-balanced and long-tailed datasets. The source code is available at https://github.com/SEUML/X-Maha .
Tong Wei, Bolin Wang, Jiang-Xin Shi et al.· Neural Information Processin...· 8 citations· ⚡2
Recent advances in multimodal large reasoning models (MLRMs) have demonstrated impressive capabilities on complex multimodal tasks, yet their reliance on long Chain-of-Thoughts (CoTs) often leads to redundant reasoning and high computational cost. Existing chain-based distillation and refinement approaches alleviate redundancy to some extent but remain limited when handling ultra-long reasoning chains, suffering from weakened dependency modeling, accumulated redundancy, and the omission of critical reasoning steps. To this end, in this paper, we propose GRACE, a Graph-based Rationalized CoT Compression framework that transforms long CoTs into reasoning graphs to enable efficient CoT compression, where reasoning steps are represented as nodes and semantic dependencies as edges. Specifically, after constructing the reasoning graph, GRACE applies a graph rationale learning approach with a selector–predictor architecture. Among them, the selector estimates node-level rationale scores to sample a compact rationale subgraph, while the predictor leverages this subgraph for final prediction. The resulting subgraph serves as the distilled core rationale, retaining essential reasoning while removing irrelevant steps. Finally, the compressed CoTs are then used to build supervised fine-tuning datasets for further MLRMs tuning. Experiments on multiple multimodal benchmarks show that GRACE substantially improves inference efficiency and interpretability without sacrificing accuracy.
Yizhi Wang, Linan Yue, Deng-Bao Wang et al.· Proceedings of the 32nd ACM...· 0 citations