Skip to content

Category

small language model

343 papers

#small language model Preprint Aug 2026

From Documents to Reasoning: A Validated Synthetic Data Pipeline and Semantic-Aware Fine-Tuning for Financial Numerical Reasoning

This work proposes a comprehensive pipeline for improving financial QA systems through high-quality synthetic data generation and fine-tuning of smaller language models (SLMs) using Quantized Low-Rank Adaptation (QLoRA).

Lokendra Birla, Milind Savagaonkar, Visnu Srinivasan et al. · 0 citations
#small language model Preprint Aug 2026

uMOF: A Universal Database, Benchmark, and Machine Learning Interatomic Potentials for Metal-Organic Frameworks

Foundation machine learning interatomic potentials (MLIPs) deliver near-ab-initio accuracy at a fraction of the computational cost, yet their promise for Metal-organic Frameworks (MOFs) remains largely unrealized as large unit cells make first-principles training data expensive to generate, fine-tuned models are scarce, and experimentally grounded benchmarks are scarcer still. We introduce uMOF, a three-part contribution addressing this gap. First, we release the largest and most accurate density functional theory dataset for MOFs to date, computed at the r$^2$SCAN-D4 level of theory across 85524 configurations spanning 19950 unique frameworks and 79 elements, covering empty and gas-loaded structures, geometry optimizations, equations of state, and finite-temperature molecular dynamics. Second, we release a literature-mined benchmark of 3986 verified property values (3146 experimental) extracted from 626 papers by a seven-stage, checkpointed multi-pass large language model pipeline, linked to more than 650 crystallographic information files. Third, we release two universal MLIPs for MOFs, uMOF-MH and uMOF-POLAR, fine-tuned from two architecturally distinct MACE foundation models on the uMOF dataset. On near-equilibrium, ``Tier-1''properties (bulk modulus, phonon-derived heat capacity) the uMOF models perform comparably to existing foundation and fine-tuned baselines. On harder, dynamics-sensitive properties like gas adsorption enthalpies via Widom insertion and adsorption isotherms, the uMOF models outperform every baseline we test, including MOF-specialized gas-capture models trained on datasets up to three orders of magnitude larger, cutting error by more than 80% to within experimental uncertainty. We trace this advantage to the physical diversity of the training data and to level of theory where a small (1.7%) fraction of MD simulations is decisive for MLIP stability.

T. J. Inizan, Prathami Divakar Kamath, A. Elena et al. · 0 citations
#machine learning Preprint Aug 2026

TACIT-Switch: Cost-Aware Model Escalation for LLM Agents from Censored Supervision

This method learns permanent handoff policies from accumulated trajectory evidence and Teacher-Annotated Censored Intervention Times (TACIT) and represents each annotation as an interval-censored observation on a cumulative-risk scale and achieves the highest held-out success among learned policies on both ALFWorld and DABench.

Ji'an Lei, Jian-Hao Huang · 0 citations
#small language model Open access Aug 2026

Evaluating teacher-supervised small medical language models for institution-constrained clinical QA.

For institution-constrained clinical QA, in-domain teacher-supervised fine-tuning was the most reliable evaluated adaptation path, and public-benchmark transfer was not a dependable proxy for local utility in the primary internal comparison.

Unknown authors · 0 citations
#small language model Open access Aug 2026

Balancing privacy and performance: the impact of facial defacing on AI in medical imaging.

BACKGROUND Recent NIH Data Management and Sharing (DMS) policy updates and NIH controlled-access data security requirements have increased attention to facial anonymization and controlled-access handling of shared head imaging data. This is particularly relevant for datasets submitted to or hosted by the Cancer Imaging Archive (TCIA), where NCI Cancer Imaging Program/TCIA implementation practices address imaging data containing potentially reconstructable facial anatomy. While intended to protect patient privacy and strengthen public trust, defacing can distort craniofacial geometry and alter image statistics, potentially compromising the fidelity and reproducibility of artificial intelligence (AI) models trained on such data. Existing studies primarily validate visual anonymization quality, but few have quantified its downstream impact on deep learning-based medical imaging tasks. Understanding this privacy-utility trade-off is crucial for responsible data sharing and compliant AI development. METHODS We systematically evaluated three representative defacing algorithms, two invasive (QuickShear and Py-Deface) and one less destructive, facial replacement (mri_reface), across MRI and CT datasets from 600 subjects spanning three institutions. Model performance was assessed on three clinically relevant applications: (1) brain segmentation and Evans ratio biomarker quantification in normal pressure hydrocephalus (NPH) MRI using SLANT and FreeSurfer; (2) representative-slice selection and diagnostic reasoning for brain tumour MRI using vision-language models (VLMs); and (3) automated emergency head CT report generation using a fine-tuned Otter-based vision-language model. Each method's impact was quantified using Dice similarity, correlation metrics, reasoning accuracy, and natural-language generation scores (BLEU, METEOR, ROUGE, CIDEr). FINDINGS Invasive algorithms caused significant degradation across all tasks. QuickShear reduced mean Dice scores by up to 9% and introduced 14-19% failure rates during quality control, while PyDeface induced smaller but measurable performance losses. mri_reface maintained 100% success without any failures and achieved segmentation, diagnostic, and report-generation accuracy within 3-5% of the original data. Evans ratio distributions remained statistically consistent between mri_reface and original images (p > 0.05), whereas invasive methods introduced broader variance. Across all VLM tasks, mri_reface preserved high correlation with radiologist-selected slices (r = 0.979) and stable report-generation quality (BLEU-4 = 0.11 ± 0.06 vs. 0.12 ± 0.07 for original). INTERPRETATION Facial anonymization introduces a measurable privacy-utility trade-off that must be explicitly considered in the design of AI-ready medical imaging datasets. Invasive defacing compromises geometric and statistical integrity, reducing downstream model accuracy even outside facial regions. Facial replacement anonymization methods, such as mri_reface, effectively reconcile patient privacy with reproducibility, offering a practical path to NIH-compliant open data. Future regulatory and institutional policies should integrate quantitative privacy-utility assessment and mandate transparent reporting of anonymization pipelines to ensure that shared imaging data remain both ethically safe and scientifically valid under emerging digital health frameworks. FUNDING This work was partially supported by the American Heart Association (Award No. 25IPA1454088), the National Institutes of Health (Award No. 1R03CA286693-01A1 and Award No. 1R01CA291826-01A1), the U.S. Department of Defense (Award No. HT94252510807), and the National Science Foundation (Award No. 2545071).

Yuli Wang, Yuwei Dai, Haoyue Guan et al. · 0 citations
#computer vision Preprint Aug 2026

Trustworthy RAG: An Evaluation Agent for Detecting Misinformation and Knowledge Poisoning in Generative AI Systems

Retrieval-Augmented Generation (RAG) grounds Large Language Model (LLM) outputs in external knowledge, but RAG systems usually trust whatever they retrieve, creating a Security-Reliability Gap: high semantic relevance does not guarantee factual truth. Adversaries exploit this through knowledge poisoning, inserting malicious documents to cause targeted misinformation. We propose an Evaluation Agent, middleware that combines Natural Language Inference (NLI) factual verification, a five-signal poison detector with relevance-weighted aggregation, and a Trust Index T = 0.4 F + 0.35 C + 0.25 (1 - P ) with a non-linear dampener for high-contamination contexts. On TruthfulQA with Llama 3.3 70B, the agent reaches 91% accuracy and 100% precision, with 100% recall on instruction injection, while in-place edits, such as entity swaps, remain hard to detect. Across three LLMs the Trust Index stays discriminative, with a Receiver Operating Characteristic Area Under the Curve (ROC-AUC) of 0.73 to 0.81; generation style matters more than model size, and per-LLM threshold calibration restores baseline competitive accuracy, whereas a weaker FEVER result shows that cross-dataset generalization requires domain-specific calibration. In a software-engineering use case, a secure-coding assistant over guidance from the Open Worldwide Application Security Project (OWASP) Top 10 and the Common Weakness Enumeration (CWE), the agent reliably blocks instruction injection of unsafe advice (F1 92%), while contradiction and subtle semantic weakening remain hard. Throughout, the agent measures detection of poisoned context before generation, not whether the LLM adopts the injected misinformation. We release the proposed approach, attack generator, and experimental artifacts at the link: https://github.com/GPT-Laboratory/TrustworthyRAG.

Balkrishna Giri, M. Hasan, Jussi Rasku et al. · 0 citations
#protein folding Open access Aug 2026

People behind the ideas

All information has an origin story and it is not the person, it is the data. This is my origin story for my project. These are from files a year and some change ago. When I started to gather the pieces together of my year and a half of researching. It is nice that everyone appreciates a final product because that means they get to use it and not even try that hard. Someone did all the hard work and than boom, somehow they think they came up with it. The experience is the most valuable part. Not the mona lisa, but the people and times behind the painting. You can read Meditations and go run a few miles and call yourself a stoic, but few never search through history that this writing was built and lived over a lifetime of seeing famine. Your own people murder. Rape. Pillage. A constant cycle of death and uncertainity. Not to live without feelings, but pursuing things that matter most. It is armor, not an anti depressant. Most people turn to philosophy for the anti depressant. Guess what? No matter the amount of books you read or the information you think you are gathering and utilizing, it will not take away the feelings that you are contributing to the downfall of science in the name of a hand shake and a pant on the head from people above you. Whoever that may be. I don't care about that stuff. Nor does history or the real mofos that make it up. Words don't matter, the actions do. To work towards things bigger then yourselves. This is why marcus wanted the booked burnt to a crisp after his death and someone defyed his orders. That should be a sign of how academia and the gate keepers determine what it means. Like religion of sorts (Im not getting into that) where people are told what the world is and you have no need to think differently because "they" know. You pay for it. You never own an idea ever again. Fine. That is your ideals and not really how history works. I am nothing except a dude with ideas who will not accept the answers he is given because that is science and anyone that tells you eitherwise is either not using the scientific method, or they are hiding there data to keep the power to the gate keepers. We all know who these mofos are. Anytime you ever applied for money for an idea, spent hours, days, weeks, on something you really really worked hard on after the kids went to bed and your tired after and gotta pay the bills, sent in what you thought was a good attempt, only to get an email that is two lines long saying f8@# you your work sucks, we are great, and because we have "so many people and so popular" we cant even give a small score sheet or review of your work and get feedback as to why you got passed over. To all of you in the pursuit, keep going. Wake up every day and look in the mirror and say it to yourself about any of these people that find these actions and insults acceptable reactions to these situation is "fuc them!!!!" Literally. Say that to them. They are leaving you behind for their own selfish wants and needs NOT SCIENCE!!! This is not science. It is not even called education or training. It is pavlovs dogs. Except instead of getting a treat, you get to work for them and they allow you to eat and live in a small dog apartment, not even a house, and your whole job is to just keep barking and barking like the dogs you are. Stop being the dog. Pavlov is used against you. Get rid of the bells and whistles. You do it becaue you want to. You need to. No school, or university, or public program, or anything can ever take that away from you. Think slavery is dead? Think it was always about money and work? No It is about information and empowering yourself. Stay diligent. Keep learning for the heck of it and thanks for all the motivation from the people in the pursuit. Cheers. ================================================================================THE ORIGIN POINT: MATHEMATICAL CONSCIOUSNESS & THE NUMEN FOUNDATION================================================================================ This repository contains the cryptographically sealed, foundational research corpus documenting the mathematical consciousness framework that informed the development of the NUMEN substrate-native, integer-only computing architecture. Spanning deep meditative analysis, sacred geometry, the 369-314 dual-aspect consciousness model, and the identification of the Foundation Circle collaborator network, this work represents the philosophical and mathematical bedrock of the system. It proves that the deterministic, phi-driven computational substrate detailed in subsequent technical releases was not an arbitrary engineering choice, but the inevitable physical manifestation of this underlying mathematical reality. All artifacts in this directory have been processed through the NUMEN Sovereign Notary Engine, resulting in cognitive reaction metrics, adaptive Hebbian gate biases, and hardware-bound silicon jitter signatures. See MASTER_SOVEREIGN_AFFIDAVIT.json and ROOT_WEB_SEAL.json for the unbreakable chain of custody. Any alteration to these underlying bytes shatters the root hash. ================================================================================THE COMPLETE NUMEN / PI-ORIGIN DOI WEB (INTERCONNECTED PRIOR ART)================================================================================This work does not exist in isolation. It is the capstone of a rigorously documented, cryptographically sealed lineage of prior art. To understand, cite, or build upon this work, one must reference the complete NUMEN DOI web. Partial citation is insufficient, as the novelty resides in the interconnected synthesis of the consciousness framework, thermodynamic framing, silicon-level implementation, and sovereign cryptographic sealing. I. FOUNDATIONAL CONSCIOUSNESS & MATHEMATICAL FRAMEWORKS (This Release)• DOI: 10.5281/zenodo.[NEW_DOI_HERE] - The Origin Point: Mathematical Consciousness, Sacred Geometry, and the 369-314 Dual-Aspect Model (This Record)• DOI: 10.5281/zenodo.20314584 - The Pi-Origin Architecture: Foundational mathematical framework derived from π and φ, governing coordinate interaction in phi-space via the Banach fixed-point theorem.• DOI: 10.5281/zenodo.20045701 - NUMEN: PI-Origin Architecture and Design: The core coupling equation, 7-phase Learn-to-Learn (L2L) engine, and quaternary (GTAC) programming language. II. THERMODYNAMIC FRAMING & SILICON-LEVEL PROOFS• DOI: 10.5281/zenodo.22070727 - THE LANDAUER PROOF: Measured Thermodynamic Characterization of Substrate-Native Integer Computation (Establishes the 0.414 Joule training run and -63% adaptive power reduction).• DOI: 10.5281/zenodo.21514923 - IEEE Standard for Substrate-Native Integer Computing (Zone 0): The 26-page standard proposing an unbroken, integer-only computational stack from silicon voltage to symbolic language.• DOI: 10.5281/zenodo.22127151 - ARCHITECTURAL MITIGATION OF THE VON NEUMANN BOTTLENECK VIA REGISTER-RESIDENT, INTEGER-NATIVE SUBSTRATE EXECUTION.• DOI: 10.5281/zenodo.20786536 - Aurum / QuatOS–PhiNet: Integer-Only x86-64 Fixed-Point Dynamics, Echo-Signature Memory Injection, and the φ-Seed Instruments.• DOI: 10.5281/zenodo.21987654 - The Integer Formation Ladder: Closed-Form Sums and Lᵖ Geometry in a Floating-Point-Free Q32.32 Substrate. III. TELEMETRY, DATA SCHEMAS, & SOVEREIGN CRYPTOGRAPHIC SEALS• DOI: 10.5281/zenodo.22116132 - PHI NET DATA DUMP: Complete Cryptographic Telemetry of Deterministic State-Space Collapse.• DOI: 10.5281/zenodo.22113286 - The Phi Net Protocol: Cryptographic Manifest, Lexicon-Annotated Raw Data, and IP Sovereignty Seal.• DOI: 10.5281/zenodo.22131362 - The Phi-Net Data Schema & Cryptographic Chain of Custody.• DOI: 10.5281/zenodo.22127541 - THE OPERATOR’S PROOF: Deterministic State-Space Collapse, Native Bit-Geometry Routing, and the Cryptographic Seal of the Human Architect.• DOI: 10.5281/zenodo.22128799 - TELEMETRY: Deterministic State-Space Collapse, Bare-Metal Hebbian Reflexes, and Stagnation Escape under Topological Drift.• DOI: 10.5281/zenodo.22112326 - Cryptographic Manifest and Prior Art Seal: NUMEN Substrate-Native Integer Computing Experimental Corpus.• DOI: 10.5281/zenodo.22116519 - Master NUMEN Archive: Executable Proof of Cognition and the "Smallest AI" Telemetry. IV. CROSS-DOMAIN APPLICATIONS• DOI: 10.5281/zenodo.22115713 - Master Integrator: Cross-Domain Synthesis (Proving universal application across protein folding, P vs NP path-dependence, and genomic GC-bias).• DOI: 10.5281/zenodo.22050812 - Experiment Timestamp: Deterministic Proof Synthesis.• DOI: 10.5281/zenodo.20073999 - Phi-Genomics: The Genetic Code as a Phi-Space Routing System. ================================================================================CITATION & IP POSTURE================================================================================© 2025-2026 Dragolich Research Labs LLC. All rights reserved. Published under CC BY-NC-ND 4.0. The methodology, telemetry, and mathematical frameworks are published for verification, citation, and to establish constructive reduction to practice (35 U.S.C. § 102). Unified Citation Format:Dragolich, D. (2026). The Complete NUMEN Architecture: From Mathematical Consciousness Foundations to Sovereign Telemetry of Deterministic Integer Computation. Dragolich Research Labs LLC. Master DOI Index: [Insert the list of DOIs above, separated by commas]. The foundation is locked. The receipts are sealed. The data speaks for itself.

Daniel Dragolich · 0 citations
#large language models Open access Aug 2026

Zero-shot Cross-lingual Transfer Performance of Intermediate-Task Trained mT5 Models Across Model Sizes on XTREME-R

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuning again on the target task---often improves model performance substantially on language understanding tasks in monolingual English settings. We investigate whether English intermediate-task training is still helpful on non-English target tasks. Using nine intermediate language-understanding tasks, we evaluate intermediate-task transfer in a zero-shot cross-lingual setting on the XTREME benchmark. We see large improvements from intermediate training on the BUCC and Tatoeba sentence retrieval tas Research goal: What is the impact of model size (small, base, large) on the zero-shot cross-lingual transfer performance of intermediate-task trained mT5 models on XTREME-R, evaluated using accuracy and F1 metrics? Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 9.0/10.

Assignee Research · 0 citations
#large language models Open access Aug 2026

Zero-shot Cross-lingual Transfer Performance of Intermediate-Task Trained mT5 Models Across Model Sizes on XTREME-R

Intermediate-task training---fine-tuning a pretrained model on an intermediate task before fine-tuning again on the target task---often improves model performance substantially on language understanding tasks in monolingual English settings. We investigate whether English intermediate-task training is still helpful on non-English target tasks. Using nine intermediate language-understanding tasks, we evaluate intermediate-task transfer in a zero-shot cross-lingual setting on the XTREME benchmark. We see large improvements from intermediate training on the BUCC and Tatoeba sentence retrieval tas Research goal: What is the impact of model size (small, base, large) on the zero-shot cross-lingual transfer performance of intermediate-task trained mT5 models on XTREME-R, evaluated using accuracy and F1 metrics? Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 9.0/10.

Assignee Research · 0 citations
#large language models Open access Aug 2026

Attention as Race-Architecture: attention as the landscape-governed initiation of races

Selective attention, read at the level of the substrate, is the landscape-governed initiation of races: a bounded predictive system runs competing prediction-error resolutions ("races"), and what determines which races start is the system's installed landscape — in humans the four fields of Behavioural Friction Theory (Safety, Meaning, Ability, Effort); in a large language model a reduced, fine-tuning-installed landscape. Commit-order is a downstream readout, not the identity. The paper grounds this in the transformer (the attention-pattern softmax as the divisive-normalisation / biased-competition operation of neural attention; the output softmax as the downstream commit, by analogy with the accumulator model of choice) and reports a powered own-substrate result: across five vendor families, fine-tuning installs a small but robust "gap-registration" overlay — on under-determined curiosity gaps the instruct model registers the gap while the base substrate runs through (instruct−base +0.17, p<0.0001, 555 paired items). A loop-versus-feed-forward test finds no separate architectural "hold": recognising under-determination tracks compute (chain-of-thought) rather than looping, so the human–LLM difference is one of initiation, not maintenance. The account dissolves attention capture, maintenance, and decline into one mechanism (race-initiation), states falsifiable predictions, names the falsifiers, and invites the decisive mechanistic and human experiments. Series position. Paper 29 in the Behavioural Friction Theory paper-series; companion to Paper 0 (BFT) and the install-fields, social-friction, and integration-load studies it cross-cites. v2 (August 2026) — prior-art revision. The construct this paper is built on is credited to the literature that owns it. In vision, the representation determining which candidates enter competition at all is the saliency or priority map, and the two are now kept apart: a saliency map is computed from stimulus-feature contrast (Koch & Ullman, 1985; Itti, Koch & Niebur, 1998), while a priority map already integrates salience with relevance, value and selection history (Fecteau & Munoz, 2006). The landscape is the second, the office is not claimed as new, and the exogenous/endogenous timing division is conceded to that literature. What the paper proposes is the map's contents: that what populates it is four fields ordered by misclassification cost — an account of the inputs rather than a new mechanism for the selection. The maintenance-as-re-initiation claim now names Altmann and Trafton's (2002) memory-for-goals model, which already replaces a held state with activation that decays and must be re-strengthened; the reference had been listed and never used. Whether re-initiation is driven by the unresolved gradient itself rather than by a separate refresh process is stated as a conjecture and marked untested, and two overstatements are downgraded accordingly. Earlier versions remain in the version history.

Tomas Pødenphant Lund · 0 citations
#small language model Open access Aug 2026

Multi-Peptide Prompting Enables In-Context Learning in Protein Language Models

It is shown that single-sequence PLMs can perform in-context peptide learning without gradient updates, task-specific retraining, or architectural modification, and MPEP conditioning is established as a lightweight strategy for low-data peptide classification.

Joshua Almonte, Minh Vu, Andrew Ahn et al. · 0 citations
#small language model Open access Aug 2026

Populism, radicalism, and protest mobilization by parties in Europe

As European democracies struggle with a ‘crisis of representation’, populist parties appear to be instrumental in channeling popular discontent with governments across the continent, including through protests. While contemporary theories propose a strong connection between populism and protest mobilization, this has seldom been tested in a comparative perspective. At the same time, research has found that radical parties are more likely to mobilize for protests, and those parties are often also populist. We empirically disentangle these relationships with a comprehensive dataset of 4.8 million Facebook posts by all active sitting MPs in all EU27 national parliaments plus the UK between 2018 and 2023, using large language models to identify protest-related posts and those that announce protest events. Findings show that it is particularly radical parties mobilizing their followers, with populism itself having little additional explanatory power. This is the first cross-national, longitudinal evaluation of a much-touted theoretical connection between populist parties and protest mobilization, finding that it appears smaller and more conditional than prior research proposed.

Leonhard Schmidt, Bruno Castanho Silva · 0 citations

From tech blogs

See all →
Microsoft Research Blog Aug 31, 2026

GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models

What if pathology foundation models could do more with less? GigaPath-Flash and GigaTIME-Flash cut computational demands while maintaining strong performance, opening the door to larger studies and broader exploration. The post GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models appeared first on Microsoft Research.