Aug 2026· IEEE Transactions on Neural Networks and Learning Systems· Vol PP, pp. 1-12· 0 citations
Medicine
TL;DR
SpectraMancer, which learns kernels directly in the Fourier spectral domain induced by multilevel circulant matrices, thereby enabling generalizable kernel learning for complex data and improves spectrum-aware kernel selection and predictive performance across diverse benchmarks.
Abstract
Kernel methods, which embed data distributions into a reproducing kernel hilbert space (RKHS) via positive-definite similarity measures, continue to play an important role. However, learning a good, generalizable kernel for high-dimensional and heterogeneous data under temporal or regional distribution shift remains challenging. To address these issues, we propose SpectraMancer, which learns kernels directly in the Fourier spectral domain induced by multilevel circulant matrices, thereby enabling generalizable kernel learning for complex data. SpectraMancer embeds all shift-invariant candidates into a common multilevel order via randomized multilevel circulant matrices, which yields a fixed Fourier diagonalization and turns inverses, products, and gradients into elementwise fast Fourier transform (FFT) operations. To the best of our knowledge, this is the first kernel-learning approach that exploits randomized multilevel circulant matrices for joint diagonalization across kernels. SpectraMancer further enforces scale invariance via kernel double centering and Frobenius normalization, reduces spectral variance through antithetic phase pairing with quasi-Monte Carlo draws, and optimizes a solver-free spectral risk proxy (SRP) for bandwidth weighting without repeated inner solves. Experimental results show that SpectraMancer improves spectrum-aware kernel selection and predictive performance across diverse benchmarks.
Kernel spectral clustering with a single bandwidth can be inadequate for data exhibiting multiple characteristic pairwise-distance scales, a problem particularly prevalent in the high-dimensional regime. We address this issue through a multi-kernel formulation that aggregates kernels with different bandwidths. The bandwidths are selected as prescribed empirical quantiles of the pairwise squared distances, thereby capturing the relevant distance scales without requiring prior population-scale information. We develop a rigorous theoretical analysis of the resulting method under a general high-dimensional, multi-scale mixture model with heterogeneous cluster centers and covariance geometries. We construct a blockwise constant, low-rank informative approximation to the empirical multi-kernel matrix and establish row-wise $\ell_{2,\infty}$ perturbation bounds for its leading spectral components, as well as for the associated normalized Laplacian matrix. These bounds yield observation-level control of the spectral embedding, which is more informative than conventional global eigenspace perturbation estimates. Under suitable eigen-gap and cluster-separation conditions, we show that approximate $K$-means applied to the multi-kernel spectral embedding achieves exact recovery with high probability.
Ze-Qin Lin, Guangming Pan, Zhixiang Zhang et al.· 0 citations
This work extends the orthogonal polynomial kernel paradigm by introducing a novel family based on discrete Hermite I polynomials, a class of $q$-orthogonal polynomials that generalize classical Hermite polynomials through a deformation parameter $q$.
Álvaro Sánchez-Paniagua Ríos, J. P. Llerena, Alberto Lastra et al.· 0 citations
We propose a family of random feature maps for scalable kernel machines on low-dimensional subspaces, ie on the Grassmannian manifold. Such representations are useful when data classes or clusters are well described by the span of a few samples. Classical Grassmannian kernels, including the projection and Binet-Cauchy kernels, require full Gram matrices, which leads to prohibitive computational and memory costs for large high-dimensional subspace datasets. We address this limitation using random features based on rank-one projections of subspace projection matrices followed by bounded non-linear transforms, either periodic or binary, to control the resulting distributions. We show that inner products in the random feature space approximate well-defined rotation-invariant Grassmannian kernels that depend only on the principal angles between subspaces. When the number of features is sufficiently large relative to the intrinsic subspace dimension, the approximation holds uniformly over all fixed-dimensional subspaces with high probability. For periodic transforms, the approximated kernel has a closed-form expression with tunable behaviour between inverse Binet-Cauchy and Gaussian-type regimes. Binary transforms yield compact one-bit subspace features, although no closed-form kernel is known. Structured rank-one projections based on randomised fast Fourier transforms further reduce computation without sacrificing practical accuracy. Experiments on synthetic data and ETH-80 classification tasks show that these features accurately preserve Grassmannian geometry while reducing computation, memory, and storage. Rank-one embeddings therefore provide a practical and scalable alternative to classical Grassmannian kernels.
Rémi Delogne, L. Jacques· Trans. Mach. Learn. Res.· 0 citations
We introduce an intrinsic spectral sparsity model for nonparametric density estimation on compact connected Riemannian manifolds. Instead of penalizing coefficients in an arbitrarily chosen Laplace--Beltrami eigenbasis, we group each complete eigenspace and measure the Hilbert norm of its spectral component. The resulting block-variation space is basis independent and isometry invariant. We establish its structural, atomic, and nonlinear approximation properties and clarify its relation to Sobolev, Besov, and coefficientwise spectral $\ell^1$ classes. We then construct a coordinate-free block-shrinkage estimator and prove a nonasymptotic signal-dependent $L^2$-oracle inequality that adapts to the unknown set of detectable eigenspaces. Under polynomial spectral growth, the risk theory separates the number of spectral blocks from their multiplicities and exhibits two regimes: one driven by a single high-dimensional eigenspace and the other by cumulative spectral complexity. Under matching spectral-growth and nondegeneracy assumptions, corresponding minimax lower bounds show that this multiplicity dependence is intrinsic, with sharp consequences for spheres and the rotation group $SO(3)$. Finally, we develop a positive, normalized, block-penalized exponential spectral sieve for log-densities and derive likelihood oracle inequalities together with expected Kullback--Leibler, Hellinger, and $L^2$ risk bounds. The resulting framework provides a geometry-respecting theory of sparse density estimation that remains invariant under changes of eigenbasis.
We present a samplet-based framework for the efficient numerical solution of saddle-point systems arising from conditionally positive definite (CPD) kernel approximation in general and universal Kriging in particular. The vanishing moment property of samplets as well as the particular structure of the associated scaling distributions, which correspond to discrete orthogonal polynomials, allow for a numerically favorable representation of these saddle-point systems. Concretely, they enable a natural null-space reduction of the indefinite saddle-point system to a (large) linear system for the detail coefficients and a small triangular system for the polynomial coefficients. We derive error bounds for the approximation by polyharmonic splines in Beppo-Levi spaces and show that the detail coefficients span precisely the subspace on which the CPD kernel is positive definite, rendering the reduced block symmetric positive definite. In view of the quasi-sparsity of the samplet-transformed kernel matrix for asymptotically smooth kernels, the resulting method achieves O(N log N) cost for the assembly and the storage of the saddle-point system. The reduced system can efficiently be solved by a sparse Cholesky factorization. We illustrate the framework with three applications, namely Gaussian process regression with generalized covariances, landmark-based image registration via samplet-compressed thin plate splines, and three-dimensional mesh deformation.
Sara Avesani, Rüdiger Kempf, M. Multerer et al.· 0 citations
Kernelized graph methods - spectral clustering, diffusion maps, and sparse kernel -regression graphs - that use Gaussian kernels depend on the choice of Gaussian bandwidth sigma, which governs the spectral character of the local kernel operator. When sigma is too small, the kernel overestimates local complexity and treats each sample as an independent direction; when sigma is too large, the kernel collapses multiple directions together, the condition number diverges, and all geometric discrimination is lost. We propose a choice of scale to make the spectral complexity of the kernel consistent with the intrinsic complexity of the underlying manifold. We propose a per-node bandwidth criterion that operationalizes this principle by jointly matching the kernel's effective rank to the local intrinsic dimension estimated via minimum spanning tree, anchoring the search in the manifold-consistent log-log scaling regime. We evaluate SSL embeddings from six encoders on CIFAR-100, showing that adaptive bandwidth consistently improves leave-one-out (LOO) classification and label propagation (LP) accuracy over fixed-bandwidth methods and competing adaptive methods.
Ecem Bozkurt, Antonio Ortega· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
