Skip to content
Log in

Author

Takahiro Ogawa

View on Semantic Scholar

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Preprint Aug 2026

Self-Supervised Representation-Guided Generative Dataset Distillation

Dataset distillation compresses a large training set into a compact synthetic set while retaining its downstream utility. Most existing methods target randomly initialized networks, whereas modern vision systems often adapt frozen pretrained encoders with lightweight modules. Distilled samples should therefore preserve the discriminative geometry of the pretrained representation space, which existing generative objectives do not explicitly consider. We propose self-supervised representation-guided generative dataset distillation (SRG), a framework that translates the SSL geometry into diffusion guidance. Specifically, SRG constructs class-wise prototypes from real-image SSL representations and performs guidance through three SSL-space objectives for prototype alignment, inter-class discrimination, and intra-class assignment. During diffusion sampling, it adopts a stage-wise guidance strategy: early denoising is anchored to the latent of the real image whose SSL representation is nearest to the assigned prototype, whereas later denoising is guided by the SSL-space objectives. This division preserves the visual realism provided by the generative prior while progressively steering samples toward representative and class-discriminative regions of the SSL representation space. SRG consistently outperforms the evaluated generative baselines across multiple datasets and IPC settings. A cross-encoder evaluation further indicates transfer across pretrained representation spaces. These results demonstrate the effectiveness of representation-guided generation for dataset distillation with pretrained SSL models.

Mingzhuo Li, Guangcheng Li, Linfeng Ye et al. · 0 citations
Book Open access Jul 2026

Revisiting the Role of Learned Attention Weighting in SASRec

Causal self-attention models such as SASRec are widely used in sequential recommendation, where learned attention weights are often assumed to provide crucial importance weighting over past interactions. Yet it is unclear when predictive performance truly depends on such non-uniform weighting. We study a controlled SASRec variant that replaces learned attention weights with uniform aggregation and is trained under an otherwise identical block structure and training recipe. Across fourteen benchmark datasets, this modification often yields performance comparable to the original model, with clear dataset-dependent exceptions. To explain this heterogeneity, we introduce a stage-wise norm-based decomposition that quantifies self-preserving vs. cross-position mixing within attention blocks. Across datasets, we find distinct regimes: low mixing yields robustness to uniformization; higher mixing tends to coincide with sensitivity, while some datasets exhibit substantial mixing without dependence on learned weighting. Our results provide a practical diagnostic for identifying when attention weighting is functionally utilized in sequential recommendation. The code is available at: https://github.com/keito0329/revisiting-sasrec.

K.Ozaki K.Ozaki, Keigo Sakurai, Ren Togo et al. · 0 citations