LLMs as Annotators: Evaluating Model–Human Alignment in Detecting Contentious Language in Historical Corpora

Ya-Hui Zhao Clemencia Siro L. Hollink OpenAI Josh Steven Achiam Sandhini Adler Lama Agarwal Ilge Ahmad Florencia Akkaya Diogo Aleman Janko Almeida Sam Altenschmidt Shyamal Alt-man Anadkat Red Igor Avila Suchir Babuschkin Va-lerie Balaji Paul Balcom Haim-ing Baltescu Bao-Lin Mo Jeff Bavarian Ir-wan Belgum J. Bello Gabriel Berdine Christo-pher Bernadett-Shapiro Lenny Berner Oleg Bogdonoff Boiko Made Laine Anna-Luisa Boyd Greg Brakman Tim Brock-man Miles Brooks Kevin Brundage T. Button Rosie Cai Andrew Campbell Brittany Cann Chelsea Carey Rory Carlson B. Carmichael Che Chan Fotis Chang Derek Chantzis Sully Chen Ruby Chen C. Jason Mark Chen Benjamin L. Chen Chess Elie Georges C. Gibson Vik Goel Tarun Gogineni Gabriel Goh Raphael Gontijo-Lopes J. Gordon Morgan Grafstein Scott Gray Ryan Greene Shixiang Joshua Gross Shane Gu Yu-Fei Guo Chris Hallacy Jesse Han J. Harris Yu-Chen He Mike Heaton Johannes Heidecke Chris Hesse Alan Hickey W. Hickey P. Hoeschele Brandon Houghton Kenny Hsu Sheng-Li Hu Xin Hu Joost Huizinga Shantanu Jain Shawn Jain Joanne Jang Angela Jiang R. Jiang Hao-Zhun Jin Denny Jin Shino Jomoto Billie Jonn Hee-woo Jun T. Kaftan Lukasz Kaiser A. Kamali Ingmar Kanitscheider Nitish Shirish Tabarak Keskar Khan Logan Jong Wook Kilpatrick Christina Kim Yongjik Kim Hendrik Kim Jamie Kirch-ner Matt Kiros Daniel Knight Łukasz Kokotajlo A. Kondraciuk Aris Kondrich Kyle Kon-stantinidis G. Kosic V. Krueger Michael Kuo I. Lampe Teddy Lan Jan Lee J. Leike Daniel Leung C. Levy L. Rachel Molly Lim Stephanie L. Lin Mateusz Lin Theresa Teusz Litwin Ryan Lopez Patricia Lowe Lue Anna Kim Makanju Sam Malfacini Manning Todor Yaniv Markov B. Markovski Katie Mayer Andrew Mayne Bob McGrew S. McKinney Christine McLeavey Paul McMillan Jake McNeil David Medina Aalok Mehta Jacob Menick Luke Metz Andrey Mishchenko Pamela Mishkin Vinnie Monaco

Semantic Scholar