Large language models (LLMs) show increasingly advanced emergent capabilities and are being incorporated across various societal domains. Understanding their behavior and reasoning abilities is therefore important. We argue that a fruitful direction for research is engaging LLMs in experiments inspired by psychology th...
When you ask a vision-language model (VLM) a question with an image, it answers your question based on the image. But to act means to choose: saying one thing means choosing to respond not say another. This becomes an issue when the image contains multiple features that equally relate to the question at hand: how would...
Nurbüke Teker, Rui Xiao, Zeynep Akata et al.· 0 citations
A central challenge in continual learning is to acquire new knowledge without forgetting what the model has already learned. This challenge appears in language model training when training data comes from various domain-, user-, or task-specific distributions that are encountered unevenly over time. In such settings, s...
Anton Baumann, Jonas Hübotter, Zeynep Akata et al.· 0 citations
Text-to-image (T2I) models are judged by benchmarks that measure whether requested content appears, but these benchmarks largely overlook the complementary ability to satisfy negated constraints, for example, generating"a non-red cup."Measuring negation raises challenges not faced by affirmation-based benchmarks and re...
Omar Elfatairy, M. A. Bravo, Jessica Bader et al.· 0 citations
Agent evaluations should be evaluated as configurable systems themselves, and that some desired behaviors are more effectively implemented in the system than requested through prompting, according to a new benchmark of four scientific tasks.
Luis Wiedmann, Leander Girrbach, C. Schmid et al.· 1 citation· ⚡1
Constantly evolving real-world knowledge necessitates models to be updated continuously. Especially in medicine, as clinical evidence changes over time, outdated knowledge can pose safety risks. Existing evaluations of knowledge integration focus on factual recall, offering limited insight into whether newly integrated...
Lukas Thede, Yash Kumar Atri, David Chen et al.· 0 citations
Modern transformers pair impressive capabilities with substantial memory and compute demands. Low-rank weight factorization reduces both while keeping the matrices dense, and thus efficient on standard hardware. Existing methods, however, choose the subspace to remove from each weight matrix with local closed-form crit...
Massimo Bini, Anders Christensen, Stephan Alaniz et al.· arXiv.org· 0 citations
Large language models (LLMs) implicitly infer attributes of their users and adapt their behavior accordingly, yet these beliefs remain difficult to inspect and causally manipulate. We introduce Belief Self-Distillation (BSD), a unified read-write framework that bridges linear and causal probing by learning a compact us...
Ali Holmov, Yi-Ran Huang, Kirill Bykov et al.· 0 citations
This work introduces a semi-supervised setting in which pretrained unimodal encoders are aligned using a small number of image-text pairs together with large amounts of unpaired data and proposes SOTAlign, a two-stage framework that first recovers a coarse shared geometry from limited paired data using a linear teacher...
Simon Roschmann, Paul Krzakala, Sonia Mazelet et al.· arXiv.org· 3 citations· ⚡1
This work proposes Time Vision Transformer (TiViT), the first framework that successfully unlocks the representational power of frozen Vision Transformers pretrained on large-scale image datasets for time series classification.
Simon Roschmann, Quentin Bouniot, Vasilii Feofanov et al.· 7 citations· ⚡2
VisLens (Visual Focus via Logit Lens), a Visual Search method built on the logit lens, which decodes the semantics held in a hidden state by projecting it through the LLM head, and which matches or exceeds prior baselines while delivering a substantial latency advantage.
Jingfeng He, Sanghwan Kim, Zeynep Akata· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.