Testing LLMs on superconductivity research questions
Education Innovation
More from the blog
MIT Quantum Initiative launches postdoctoral fellowship program
The Institute welcomes its first cohort of QMIT Fellows this fall to advance interdisciplinary quantum research.
GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models
What if pathology foundation models could do more with less? GigaPath-Flash and GigaTIME-Flash cut computational demands while maintaining strong performance, opening the door to larger studies and broader exploration. The post GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models appeared first on Microsoft Research.
How an MIT research project became a global programming language
With millions of users across the world, Julia has been used to conduct cutting-edge research and to design new drugs, jet engines, heat pumps, and more.
Related papers
Model-Based Agentic Software Engineering
MAGE explains how externalized knowledge, bounded action, independent evaluation, and retained human authority can compose into a governed engineering environment, and proposes tests of when that environment turns commodity intelligence into durable engineering progress.
Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code
LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense.. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model--language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation--utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9--95.2\%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile. Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.
Prospective evaluation of hearing aid fitting adjusted toward one-third functional gain in patients with sensorineural hearing loss.
OBJECTIVE To prospectively evaluate whether modifying DSL version 5-based hearing aid (HA) fittings by adjusting gain on HA fitting software so that measured functional gain (FG) approached a one-third gain (1/3G) target could provide appropriate fitting outcomes in patients with sensorineural hearing loss. METHODS Twenty-four patients (48 ears) with bilateral sensorineural hearing loss underwent initial HA fitting using the DSL version 5 prescription formula. FG was measured at 250-4000 Hz, and HA gain was adjusted on HA fitting software so that FG approached the target 1/3 G. Speech discrimination scores at 65 and 80 dB SPL were evaluated after a two-week trial period using the 67-S Japanese monosyllable word list. Based on speech discrimination test results, ears were classified as well-fitting or non-well-fitting. FG values were compared between the two groups. RESULTS Twenty-one patients (42 ears) completed the study. Thirty-one ears (73%) were classified as well-fitting. Although HA gain was adjusted toward the target 1/3 G, measured FG values at 250 and 500 Hz remained lower than the target values. In well-fitting ears, low-frequency FG values were lower than the target 1/3 G, whereas FG at 2000 Hz was close to the target value. In contrast, non-well-fitting ears showed low-frequency FG values closer to the target 1/3 G, whereas FG values at 2000 and 4000 Hz remained below the target values. CONCLUSIONS Although HAs adjusted toward a 1/3 G target did not achieve the intended FG values, particularly at low frequencies, relatively favorable fitting outcomes were obtained in approximately three-quarters of the ears. In well-fitting ears, low-frequency FG remained below the target 1/3 G, whereas FG in the mid-frequency range around 2000 Hz was close to the target value. These findings provide a basis for future prospective studies to clarify how these FG characteristics should be applied to optimize HA adjustment.
Multiscale Fresh Tea Leaves Sorting Device with Drum-axial Airflow Coupling: Design and Performance Test
To address the problems of low sorting accuracy and poor operation stability caused by physical leaf entanglement in traditional drum screening of fresh tea leaves, a multiscale fresh tea leaf sorting system with drum-axial airflow coupling based on intelligent control was designed. The axial moving distances of fresh tea leaves of different scales at wind speeds of 5, 7, and 9 m/s were calibrated through bench tests, and the optimal wind speed parameter for secondary fine screening was determined. A numerical model of the axial airflow field inside the drum was established via Fluent software, and a coupled sorting test platform was built to compare the sorting performance of the traditional pure drum screening mode and that of the coupled intelligent sorting mode. The results showed that 7 m/s is the optimal axial airflow velocity for secondary fine screening of multiscale fresh tea leaves during this test, which can realize effective back-blowing of small-scale materials and accurate screening of large-scale materials. At this velocity, the flow field is evenly distributed, and the effective thrust area highly matched the sorting demand. The average sorting efficiency of the coupled intelligent sorting mode reached 84.8%, which is 25.6% higher than that of traditional pure drum screening, with favorable sorting accuracy and operation stability. These findings can provide a theoretical basis and technical reference for the optimization, upgrading, and intelligent transformation of high-efficiency fresh tea leaf sorting equipment.