How Much Junk Do Rules Catch? Evaluating and Tuning pygarble, a Dependency-Free Gibberish Screener, on Survey Responses and Social-Media Text

Abstract

Text that reaches a language model, a survey database or a training corpus is not always text. Keyboard mashing, random strings and placeholder answers are common, and the usual detectors for them are either trained classifiers with model downloads or hand-written rules that are rarely measured. pygarble is a Python library with no runtime dependencies that screens text with 32 rule-based and statistical strategies combined into named profiles. We evaluate it on two public English short-text datasets, a Kaggle collection of 3,513 free-text survey responses labelled as junk and a Hugging Face collection of 9,971 English rows that pairs 4,000 random strings with 5,971 tweets, against a character Markov-chain gibberish detector and a language identifier, every method at its own default decision. Out of the box, the english_fusion profile captures 89.5% of the survey junk and recalls 99.1% of the random strings at a precision of 0.906, from a 1.05 MB install that processes a thousand short texts in 62.0 ms; no baseline captures more survey junk while also flagging fewer tweets. A small labelled sample shifts the trade-off: a weight for every strategy and a threshold learned with Optuna from 20% of the rows, in about 17 seconds of search, cuts the false-positive rate on tweets from 6.9% to 4.9% and raises precision from 0.906 to 0.930 on the held-out 80%, with capture about level (89.7% against 89.6%) and recall lower (97.6% against 99.1%; means over five draws). The learned weights are interpretable at the extremes: word lookup gets full weight in every draw, and the two encoding checks, which fire on tweets this dataset labels clean, get almost none. A dependency-free rule ensemble is therefore competitive on real junk as shipped and adapts to a domain in seconds, with every decision explainable strategy by strategy. Code, per-row predictions and a verification script are released so that every number can be recomputed.