human-computer interaction

Specializing LLMs with Custom Datasets: A Four-Round Open vs Closed Comparison

Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluation

Retrieval-Augmented Generation (RAG) systems are emerging as a key approach for grounding Large Language Models (LLMs) in external knowledge, addressing limitations in factual accuracy and contextual relevance. However, there is a lack of empirical studies that report on the development of RAG-based implementations grounded in real-world use cases, evaluated through general user involvement, and accompanied by systematic documentation of lessons learned. This paper presents five domain-specific RAG applications developed for real-world scenarios across governance, cybersecurity, agriculture, industrial research, and medical diagnostics. Each system incorporates multilingual OCR, semantic retrieval via vector embeddings, and domain-adapted LLMs, deployed through local servers or cloud APIs to meet distinct user needs. A web-based evaluation involving a total of 100 participants assessed the systems across six dimensions: (i) Ease of Use, (ii) Relevance, (iii) Transparency, (iv) Responsiveness, (v) Accuracy, and (vi) Likelihood of Recommendation. Based on user feedback and our development experience, we documented twelve key lessons learned, highlighting technical, operational, and ethical challenges affecting the reliability and usability of RAG systems in practice.

LLM-based Multi-Agent System for Intelligent Refactoring of Haskell Code

Refactoring is a constant activity in software development and maintenance. Scale and maintain software systems are based on code refactoring. However, this process is still labor intensive, as it requires programmers to analyze the codebases in detail to avoid introducing new defects. In this research, we put forward a large language model (LLM)-based multi-agent system to automate the refactoring process on Haskell code. The objective of this research is to evaluate the effect of LLM-based agents in performing structured and semantically accurate refactoring on Haskell code. Our proposed multi-agent system based on specialized agents with distinct roles, including code analysis, refactoring execution, verification, and debugging. To test the effectiveness and practical applicability of the multi-agent system, we conducted evaluations using different open-source Haskell codebases. The results of the experiments carried out showed that the proposed LLM-based multi-agent system could average 11.03% decreased complexity in code, an improvement of 22.46% in overall code quality, and increase performance efficiency by an average of 13.27%. Furthermore, memory allocation was optimized by up to 14.57%. These results highlight the ability of LLM-based multi-agent in managing refactoring tasks targeted toward functional programming paradigms. Our findings hint that LLM-based multi-agent systems integration into the refactoring of functional programming languages can enhance maintainability and support automated development workflows.

Anomaly detection in smart power grids with graph-regularized MS-SVDD: a multimodal subspace learning approach

Anomaly detection in smart power grids is a critical challenge due to the complexity, heterogeneity, and dynamic nature of sensor data streams. Existing one-class classification methods, particularly Subspace Support Vector Data Description (SVDD), have been extended to multimodal scenarios but often fail to fully exploit the structural dependencies across modalities, limiting their robustness in real-world applications. In this paper, we address this gap by proposing a generalized Multimodal Subspace Support Vector Data Description (MS-SVDD) model with graph-embedded regularization. The method projects data from multiple modalities into a shared low-dimensional subspace while preserving modality-specific structure through Laplacian regularizers. Our approach is evaluated on a three-modality dataset derived from smart grid event time series, using a dedicated preprocessing pipeline for constructing one-class classification training samples. The results demonstrate that our graph-embedded MS-SVDD improves robustness of event detection compared to conventional approaches, highlighting the potential of integrating graph priors with multimodal subspace learning for advancing anomaly detection in critical infrastructure. More broadly, this work contributes to the wider field of AI by illustrating how relational and structural information can be systematically embedded into one-class models, enabling robust learning under complex, high-dimensional, and multimodal conditions.

From Business Meetings to Requirement Artifacts: An Agentic AI Approach with MARARE

This paper presents MARARE, a real-time multi-agent system that transforms meeting dialogues into structured software requirements. One agent interacts with participants, while background agents extract and verify requirements collaboratively. Evaluation using the LLM-as-a-Judge method across five meetings (5–8 minutes each) shows a mean coverage of 80.0 ± 11.2 % (mean ± SD), semantic similarity of 0.86 ± 0.05, and hallucination rate of 14.3 ± 6.2 %. Preliminary results indicate performance differences across LLMs, suggesting that model choice influences coverage, consistency, and hallucination rates.

Bridging Humans and LLMs: Investigating Human-AI Collaboration in Multi-agent Requirements Analysis for Organizational AI Adoption

Context: Organizations adopting Artificial Intelligence (AI) face challenges in eliciting and analyzing requirements that align with strategic objectives, especially when human oversight and iterative refinement are needed. Large Language Models (LLMs)-based Multi-agent systems provide a potential solution by supporting structured and collaborative Requirements Engineering (RE) processes for AI adoption planning. Objective: The objective of this study is to investigate whether a multi-agent system, built on LLMs and supported by human input, can assist in requirements analysis for AI adoption. Method: We used a mixed-method approach: (i) designed and developed a multi-agent system to support the generation and prioritization of requirements for AI adoption, (ii) conducted multiple case studies with four companies to evaluate the system, and (iii) collected data through post-session questionnaires from nine participants and follow-up interviews, one per company. Results: Questionnaire and interview findings together indicate that the system may assist in identifying relevant and goal-aligned requirements. Seven participants considered the generated requirements relevant, and six found them aligned with organizational goals. Participants noted that iterative feedback improved completeness and feasibility, often within two feedback rounds. Both data sources show that human input was essential to clarify technical details, ensure contextual accuracy, and validate prioritization results. Participants from all companies also identified usability, transparency, and scalability as areas requiring further refinement for broader organizational use. Conclusions: LLM-based multi-agent systems can support strategic AI planning by enabling iterative refinement with human experts. Future work will include more interviews with stakeholders and adjustments to system features to improve transparency, usability, and scalability.

REFINE: A Multi-Agent LLM Approach for Evidence-Guided Code Refactoring

Large Language Models (LLMs) offer new opportunities for automated code refactoring. However, generated changes must reduce targeted quality problems without introducing new issues or altering behaviour-relevant code structures. We introduce REFINE (Refactoring with Evidence-aware Flow for Integrated ageNtic Execution), a tool-agnostic, evidence-aware multi-agent approach for generating Java file-level refactoring candidates. REFINE combines static-analysis-guided smell identification, smell-informed planning, LLM-based transformation, automated re-analysis, preservation checks, and structured reporting. We evaluate REFINE on 450 Java files from 15 open-source systems, producing 1,350 model-pass outputs using OpenAI GPT-5.5, Google Gemini 3.1 Pro Preview, and Anthropic Claude Opus 4.8. REFINE reduces detected code smells by 68.26%, 72.79%, and 68.49% across the three configurations, respectively, with the strongest reductions observed for major smells. A matched 150-file direct-prompt baseline shows that REFINE achieves a higher median code-smell reduction with smaller edits and fewer public-method removals. However, broader quality improvements are inconsistent, and preservation checks reveal residual risks, including assert/fail-call changes and public-method removal. Therefore, REFINE outputs should be treated as refactoring candidates requiring compilation, testing, dependency analysis, and human review before adoption in repository- or system-level settings.

A Simulator-Grounded Framework For Constructing Verifiable Muscle-Grounded QA From 3D Tongue Meshes (extended version)

Existing articulatory corpora based on real-time MRI and electromagnetic articulography capture tongue motion but lack traceable labels for the muscle-driven process underlying each configuration. We introduce a simulator-grounded construction framework, instantiated as 3DTongueQA. Controlled 11-D muscle activations are mapped by the ArtiSynth Badin finite-element model to fixed-topology tongue meshes, converted into reusable biomechanical records, and rendered as deterministic QA on simulator-defined muscle state, geometry, and target-directed change. From 295,157 screened configurations, 295,115 valid meshes yield 891,156 QA records per language. English and Korean naturalizations modify only surface form and are verified against the source records. A swappable SpiralNet++--Qwen3-8B baseline reaches $62.9\pm9.2$ Muscle EM, dropping to 2.2 under mesh shuffling; anchor-held-out training retains 80.4--98.6\% of full-inventory scores, while task-specific readouts reach $88.7\pm0.7$ Muscle EM and $93.3\pm1.0$ Direction EM. The results show geometry-grounded supervision supporting both unified language QA and structured prediction. Code is available at https://github.com/esh0504/muscle-grounded-qa .

FleetScape: A Mixed Reality Sandtable for Spatial Supervision and Control of Scalable Drone Fleets

As autonomous drone deployments scale from individual units to coordinated swarms, the human operator's role shifts from direct piloting to high-level supervision. Current interfaces often treat multi-drone control as a scaled-up version of single-drone operation. We instead investigate how reframing fleet supervision as spatial interaction can better support the spatial, temporal, and safety demands of complex missions. We present FleetScape, a Mixed Reality (MR) sandtable system that externalizes layered real-time mission, safety, and environmental data while enabling fluid transitions between manual intervention and autonomous supervision. We developed a high-fidelity building inspection simulation that generates and streams synchronized multi-drone and environmental data for MR visualizations. We used this prototype to conduct a user study with six experienced drone pilots managing fleets of up to 15 drones. Our findings show that FleetScape supports situational awareness through layered spatial representations and clarifies control mode transitions. However, a limit to situational awareness was observed as fleet size increases, leading to different supervisory strategies. Finally, we derive design implications for supporting scalable drone fleet supervision.

Thinking Inside the Box: Considerations for Putting Data Physicalization Workshops in a Box

Visualization researchers utilize workshops both for applied research and to engage different populations with visualization-based activities. While there are many benefits to running visualization workshops, their utility and impact rely on the presence of a researcher who has deep knowledge about visualization theory and practice. In this work, we introduce workshop-in-a-box as a design concept intended to challenge the researcher-centric approach to data physicalization workshops. Through a design study with a socially innovative organization, we deployed several data physicalization workshops that our collaborator ran instead of us. Based on this experience, along with two accompanying case studies that validate the concept, we present material and procedural considerations for how to put data physicalization workshops into a box and the implications it has for extending visualization research outside the bounds of academia.

Ethical Asymmetry in Human-Robot Interaction - An Empirical Test of Sparrow's Hypothesis

The ethics of human-robot interaction (HRI) have been discussed extensively based on three traditional frameworks: deontology, consequentialism, and virtue ethics. We conducted a mixed within/between experiment to investigate Sparrow's proposed ethical asymmetry hypothesis in human treatment of robots. The moral permissibility of action (MPA) was manipulated as a subject grouping variable, and virtue type (prudence, justice, courage, and temperance) was controlled as a within-subjects factor. We tested moral stimuli using an online questionnaire with Perceived Moral Permissibility of Action (PMPA) and Perceived Virtue Scores (PVS) as response measures. The PVS measure was based on an adaptation of the established Questionnaire on Cardinal Virtues (QCV), while the PMPA was based on Malle et al. [39] work. We found that the MPA significantly influenced the PMPA and perceived virtue scores. The best-fitting model to describe the relationship between PMPA and PVS was cubic, which is symmetrical in nature. Our study did not confirm Sparrow's asymmetry hypothesis. The adaptation of the QCV is expected to have utility for future studies, pending additional psychometric property assessments.

A Multimodal Dataset of Student Oral Presentations with Sensors and Evaluation Data

Oral presentation skills are a critical component of higher education, yet comprehensive datasets capturing real-world student performance across multiple modalities remain scarce. To address this gap, we present SOPHIAS (Student Oral Presentation monitoring for Holistic Insights & Analytics using Sensors), a 12-hour multimodal dataset containing recordings of 50 oral presentations delivered by 65 undergraduate and master's students at the Universidad Autonoma de Madrid, comprising 46 individual presentations with a mean presentation duration of 9 min 48 s (SD = 33 s) followed by a mean Q&A duration of 6 min 28 s (SD = 3 min 02 s), and 4 group presentations with a mean presentation duration of 14 min 11 s (SD = 1 min 40 s) followed by a mean Q&A duration of 8 min 22 s (SD = 1 min 21 s). SOPHIAS integrates eight timestamped sensor streams from high-definition webcams, ambient and webcam audio, eye-tracking glasses, smartwatch physiological sensors, and clicker, keyboard and mouse interactions. In addition, the dataset includes slides and rubric-based evaluations from teachers, peers, and self-assessments, along with timestamped contextual annotations. The dataset captures presentations conducted in real classroom settings, preserving authentic student behaviors, interactions, and physiological responses. SOPHIAS enables the exploration of relationships between multimodal behavioral and physiological signals and presentation performance, supports the study of peer-assessment and provides a benchmark for developing automated feedback and Multimodal Learning Analytics tools. The dataset is available for research, including both academic and legitimate commercial research and development, under controlled access through Science Data Bank, subject to approval of a Data Usage Agreement (DUA), with code provided through GitHub.