Existing activation steering methods often assume that a high-level concept can be mediated by a single steering direction. To support this, two complementary interventions should be achieved: additive steering should induce the target behavior, while directional ablation should suppress it. Yet behaviors may occupy ri...
Han-Gong Wang, Erik Miehling, Dennis Wei et al.· 1 citation
Chain-of-thought (CoT) can sound plausible yet be unfaithful to the model's underlying reasoning. Most prior work probes CoT faithfulness through input--output behavior or input attributions, leaving internal computation largely underexplored. We instead cast faithfulness as internal concept grounding: Does a large lan...
Qianli Wang, Yi-Long Wang, Dennis Wei et al.· arXiv.org· 1 citation
The architecture family implementing this function class is named CoFrGeNets - Continued Fraction Generative Networks, and novel architectural components based on this function class that can replace Multi-head Attention and Feed-Forward Networks in Transformer blocks while requiring much fewer parameters are designed.
Amit Dhurandhar, Vijil Chenthamarakshan, Dennis Wei et al.· arXiv.org· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.