From Blind Edits to Verified Repair: Building Trustworthy User-Side LLM Agents for Web Accessibility
TL;DR
Three building blocks toward a complete, privacy-preserving browser agent, a verified repair instrument pairing a trilingual seeded-violation benchmark with an audit-inject-verify loop, and a dual-condition protocol that measures harm as carefully as benefit are contributed.
Abstract
Assistive agents that adapt web pages on the user's side, at the moment of browsing, could reach the accessibility failures that site authors leave unfixed, and large language models make such agents newly plausible. We contribute three building blocks toward that goal. The first is a complete, privacy-preserving browser agent: a Chrome extension that extracts a page's style sheets, condenses them to fit a local model's context window, asks the model for additive CSS addressing 18 metrics from WCAG and the W3C cognitive accessibility guidance, and injects the result reversibly into the live page. The second is a dual-condition protocol that measures harm as carefully as benefit, applied to six small open-weight models (7B to 14B) on ten violation-rich and ten highly accessible live sites. The diagnosis is sobering but precise: unverified generation improved and regressed pages at similar rates (24 improvements against 20 regressions across the 100 trials of the five models that produced injectable CSS), fixing typography while breaking perception-dependent properties. The third answers the diagnosis: a verified repair instrument pairing a trilingual seeded-violation benchmark with an audit-inject-verify loop that accepts a change only if violations strictly decrease, so regression on the automated checks is impossible by construction. In a real browser the instrument detects 57 of 57 seeded violations with no false positives and rejects 126 of 126 adversarially harmful candidates. All code, prompts, benchmark materials, aggregate data, and validation logs are released.