"You"Is Not Toxic: How LLM Labelers Rewrite the Web's Conversation Laws
It is proved that a fixed Cohen's kappa leaves a law nearly unconstrained, and that Threadsafe uses the labeler's scores to choose which comments human raters label and to predict the others, and corrects the predictions with the human labels; linear laws are estimated without bias whatever the labeler does.
Liang Yue, Eric A. Vance
· 0 citations