Replication package for: When Convenience Meets Risk: Evaluating Usability, Security, and AI Reliability Across Modern Operating Systems

Abstract

Replication package for "When Convenience Meets Risk: Evaluating Usability, Security, and AI Reliability Across Modern Operating Systems" (Alhawarat and Nijem), submitted to Empirical Software Engineering. Contents 01_AI_Reliability: Windows Copilot test set (80 prompts) with both raters' scores and reconciliation notes 02_Security_Privacy: security/privacy rubric (108 cells) with both raters' scores, evidence sources, and reconciliation notes 03_Usability: Windows Copilot clustering (themes and figure); Apple Intelligence and HarmonyOS data (28,422 posts identified by URL, with model and human labels), human-validation labels and rulebook, pipeline and reproduction notebooks, results, and figures Reproducing the results 03_Usability/Apple_HarmonyOS_Analysis/notebooks/Reproduce_Usability_Results.ipynb recomputes every Apple/HarmonyOS usability result reported in the paper from the released files, without API calls. Data notes Posts are released without raw text or usernames (URLs and derived labels only). The Windows Copilot review corpus (Chhetri et al. 2025) is available from its authors. License: code MIT; data and documentation CC BY 4.0. Full Changelog: https://github.com/malhawarat/OS_Trends_NFR/commits/v1.0.0