Overall, the results show that SAE- and probe-based techniques often outperform basic prompt-based methods for steering LLM agents, although this advantage depends on the specific prompting strategy involved.
Jia-Run Fan, Arul Murugan, Shreyas R. Krishnan et al.· 0 citations
We present ORQA, a method for testing occupation-level knowledge in large language models. Prior methods either map abstract LLM skills to occupations via task definitions or utilize expert knowledge which is difficult to obtain at scale and expensive. ORQA complements both of these methods by connecting O*NET occupati...
Shreyas R. Krishnan, Serina Chang, Abhishek Nagaraj· 0 citations
This article introduces a framework for designing and running simulated experiments with LLM‐powered agents and applies the framework to the exploration–exploitation dilemma and shows that LLM‐based experiments reproduce patterns observed among human participants.
A unified framework that evaluates the capability of models to automate and augment another agent's performance, motivating benchmarks that evaluate models according to the roles they play in human-AI and multi-agent systems.
Pattaraphon Kenny Wongchamcharoen, K. Gulati, Min-Min Fong et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.