Perspective API closes at the end of 2026, removing the de facto standard for toxicity measurement and exposing researchers'dependence on a tool they did not control. Drawing on this case, we argue that a research field must build and govern its own measurement infrastructure rather than borrow it. Surveying 241 papers that use or study Perspective, we show what depending on it cost the research community: claims reaching past what the tool could support, results that shifted when its model was silently retrained, and disparities researchers could measure but not explain. These failures were amplified throughout the LLM lifecycle, where Perspective supplied the labels, filtered the corpora, and graded the systems trained on each, rewarding errors rather than catching them. To keep the instrument open to study after shutdown, we release Perspective scores for 5.9 million text snippets from 77 datasets. We further specify ten requirements for measurement infrastructure a field owns, and argue that what blocks such infrastructure is not technical capability but the value the field places on infrastructure work.
Evaluations of model-based engineering (MBE) typically report what a generator produces—coverage, artefacts emitted, lines generated. Yet under industrial delivery pressure much of that structure is later overwritten, bypassed, or absorbed into hand-written code, while other model-derived assets quietly endure. We argu...
Artur Boronat, Jawad Mustafa, Fola-Dami Eyitemi· Proceedings of the ACM/IEEE...· 0 citations
Software supply chain security has become increasingly critical due to the widespread reliance on third-party dependencies and the growing attack surface of modern software ecosystems. However, existing quantitative, measurement-based analysis and vulnerability management approaches remain largely fragmented and ecosys...
SWE-Prometheus is presented, a benchmark for the broader task of improving repository engineering governance, showing why repository-governance evaluation should report improvement, behavior preservation, evidence quality, and coverage together.
Jia-Jun Wu, Lei-Xin Sun, Zi-Hang Tan et al.· 0 citations
IT projects have, compared to other domains, higher failure rates. This phenomenon is even more pronounced in the Data & Analytics (D&A) context. We notice that shortcomings in existing D&A development and operation frameworks are one major reason for D&A project delay, misorientation and failure and that mitigation of...
Sebastian Junker, I. Weinmann· Schmalenbach Journal of Busi...· 0 citations
Modern enterprise cloud infrastructures need to be able to deliver software quickly, remain operationally resilient and follow strict regulatory requirements. Through traditional operations, observability, security and governance are typically treated as separate processes, which leads to longer Mean Time To Repair (MT...
Vishwa Lakhnakiya· 2026 International Conferenc...· 0 citations
This work investigates the data behind vulnerability analysis through a dataset-centric taxonomy that separates what an artifact is (code, metadata, patches, tests/PoCs, reasoning, traces), how good it is (realism, label evidence, scale, diversity, leakage, availability), and what it is used for.
Yu Nong, Yao Du, Tian-Xiang Xu et al.· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduOct 15, 2026
A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.
Microsoft Research Blog· microsoft.comOct 15, 2026
Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduSep 24, 2026