Skip to content

AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models

Sep 2026 · arXiv.org · Vol abs/2609.36368 · 0 citations · 35 references
Computer Science

TL;DR

AdaKerNet is introduced, a novel learnable task-adaptive neural kernel decoder that operates solely on its (frozen) rich representations obtained from the diverse available modalities and established as an effective approach for prediction from frozen multimodal representations in the scarce label regime.

Abstract

Large foundation models have been introduced with the promise of efficient adaptation to downstream tasks. Yet, under limited supervision, MLLMs, an important class of large foundation models, remain challenging to adapt to various downstream tasks. Adaptation typically relies either on MLLM parameter fine-tuning or on training neural-based decoders. Both approaches struggle under limited supervision, while fine-tuning additionally requires access to model parameters, which is often unavailable for closed-source models. We introduce AdaKerNet, a novel learnable task-adaptive neural kernel decoder. AdaKerNet is fully agnostic to the parameters of the underlying MLLM and operates solely on its (frozen) rich representations obtained from the diverse available modalities. AdaKerNet relies on (i) a set of learnable, Lipschitz-controlled multimodal features derived from these MLLM representations; (ii) a reference kernel that provides a soft structural prior on those features; and (iii) a lightweight nonlinear neural predictor that adaptively deforms that structure. Learning the kernel representation and the neural predictor jointly within a unified optimization framework allows AdaKerNet to capture features and geometric relationships relevant to the downstream task. Numerical tests across four MLLMs: BLIP-2, LLaVA-1.5, Qwen2.5-VL, and Gemini Embedding 2, and multimodal inputs spanning text, audio, images, and tabular measurements demonstrate significant and consistent improvements over direct MLP, attention-, autoencoder- and kernel-based decoders, across a range of scarce-label budgets, with average error reduction of up to 41% across baselines. These results establish AdaKerNet as an effective approach for prediction from frozen multimodal representations in the scarce label regime. Additional structural ablations highlight the complementary contributions of AdaKerNet's components.

View source

Similar papers

#machine learning Preprint Oct 2026

Compact set-valued deep ensembling in multi-class classification

This paper tackles visible challenges in deep ensemble learning, where deep neural networks serve as ensemble members: training and storage burdens, and robustness of cautious (set-valued) predictions targeting multiple utilities, which may involve reward-sensitivity. To mitigate the training and storage burdens, we pr...

K. Tran, D. Nguyen, Vu-Linh Nguyen et al. · 0 citations
#artificial intelligence Preprint Oct 2026

Local Support Learning

This work proposes Local Support Learning (LSL), a general-purpose framework that augments gradient-based training for retention of prior capabilities without access to prior data, and presents a gate based on a Gaussian Mixture Model, whose likelihood decays rapidly away from its training data, giving it a natural ten...

Assaf Ben-Kish, Akarsh Kumar, James R. Glass et al. · 0 citations
Open access 2026

BiReg: Bilateral Regularized Kernel Adaptation for Training-Free Few-Shot Adaptation of Vision–Language Models

Bilateral Regularized Kernel Adaptation (BiReg), a closed-form, training-free framework that formulates few-shot VLM adaptation as multi-output kernel ridge regression, is proposed that achieves superior accuracy and robustness while retaining the efficiency of a closed-form, training-free solution.

Omar Arif, Aizah Arif · 0 citations
#artificial intelligence Preprint Oct 2026

Measure Less, Know More: Self-Supervised Test-Time Feature Acquisition

ECHO-$k$ is introduced, a task-agnostic and self-supervised learning principle for modality acquisition that uses a deep model's internal pretrained representations as proxy targets that summarize cross-modal information and consistently improves budgeted downstream performance across diverse foundation-model backends.

Eeshaan Jain, L. Bleistein, B. Deplancke et al. · 0 citations
#machine learning Preprint Oct 2026

MotherTree: Meta-learning on synthetic data improves decision tree training

Conventional decision tree algorithms produce effective, transparent models that can be audited, communicated, and deployed independently of the training data, but require learning every new task from scratch. In contrast, tabular foundation models demonstrate that meta-learning from a synthetic prior distribution enab...

Zi-Yu Wang, Fredrik D. Johansson · 0 citations

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.