Supporting Information-Need Expression with Icons and Text: An Exploratory Study with Adults with Learning Disabilities

Abstract

Access to digital information depends on a user’s ability to express an information need before a retrieval or answer system can respond. Text-centred query interfaces can make this entry point difficult for adults with learning disabilities when interaction depends on spelling, lexical recall, and abstract keyword construction. We combined community fieldwork with an exploratory study of an icon-supported multimodal query interface paired with a generative answer backend. Fieldwork observations informed design requirements for recognition-based input, visual composition, guided suggestions, and simplified interaction. Seven adults completed two structured activities under instruction-defined icon and text conditions within the same multimodal system, followed by a self-directed free-choice task. Screen and session recordings were analysed to trace query trajectories, assistance, input-resource crossovers, response outcomes, and observable evidence that participants followed the responses. Retrospective ratings of the two structured conditions and synchronised physiological traces provided secondary descriptive context. The observed patterns did not indicate a single best input resource. Most participants selected icon input for the self-directed task, but responses addressing the intended need were observed with both input resources. A crossover from text to icon input was also observed during query repair, while icon input sometimes introduced ambiguity and required assistance. We contribute a traceable account linking field observations to interface design and derive implications for inclusive digital-library query interfaces: providing multiple input resources, supporting movement between them, and making the system’s interpretation of a query visible to users. Given the small sample, fixed condition order, and use of generated rather than collection-retrieved answers, the findings should be treated as exploratory design evidence.