Aug 2026· Zenodo (CERN European Organization for Nuclear Research)
Abstract
The Transformer architecture---built on attention rather than recurrence---redrew the landscape of natural language processing and became the substrate of contemporary artificial intelligence. This article presents a narrative review of the architecture's canonical line: Sutskever and colleagues' 2014 sequence-to-sequence learning, Bahdanau and colleagues' 2015 attention alignment, Vaswani and colleagues' 2017 Attention Is All You Need, Devlin and colleagues' 2019 BERT pretraining, Radford and colleagues' 2019 GPT-2, Brown and colleagues' 2020 GPT-3 and few-shot learning, Raffel and colleagues' 2020 T5 transfer, Dosovitskiy and colleagues' 2021 Vision Transformer, Bommasani and colleagues' 2021 foundation-model framing, Hoffmann and colleagues' 2022 Chinchilla scaling laws, Ouyang and colleagues' 2022 InstructGPT alignment, and Touvron and colleagues' 2023 LLaMA openness. The synthesis is organized around three themes: architecture, in which self-attention's parallel sequence processing replaced recurrence and enabled scale; scaling, in which pretraining on text plus parameter growth yielded emergent few-shot capability and then compute-optimal correction; and alignment and access, in which instruction tuning, RL from feedback, and open weights reshaped capability's deployment. It is concluded that the Transformer is machine learning's most consequential architecture to date---its attention mechanism the field's new inductive bias---and that scaling's economics and governance now define its trajectory.
A production platform built on large language models makes two kinds of decision, and most of its trouble comes from writing both into one clause. An optimization decision improves an objective: lower latency, lower cost, higher quality, fewer tests run. A boundary decision fixes a constraint that may not be relaxed for any gain: a residency rule, a least-privilege scope, a human-review threshold. When the two share a clause, improving one silently erodes the other, which is why efficiency and accountability are so often reported as a trade. This specification is built on one invariant: a boundary is a clause the optimizer may not cross, and everything else is optimization. The contribution is a cross-layer architectural method for separating non-negotiable constraints from adaptive optimization and binding both to reconstructable evidence, applied identically across model routing, agent orchestration and AI-native delivery. The seventeen patterns are instances of that method rather than the contribution itself. Each pattern is specified in the classical pattern form and carries three architectural declarations: the boundary it fixes, the optimizer it frees, and the evidence proving the boundary held. Every boundary is assigned to one of five classes covering data, authority, decision, resource and process constraints. Section 3 states the derivation method by which candidates were admitted or rejected, and publishes the rejections alongside the admissions so that the criterion can be examined rather than trusted. Three mechanisms make the language operate as a language rather than a list. A pattern relationship graph names which pattern supplies the artifact, evidence or authority another depends on, including the single cycle by which a workflow improves from its own structural record and the economic chain running the full height of the stack. A normative event identity, with rules for causal parentage, retries, provider boundaries and retention, turns the requirement that evidence be joinable into something an implementation can satisfy or fail. And per-pattern applicability conditions replace categorical requirements, so that a pattern governing a mechanism an institution does not operate is out of scope rather than a gap. Conformance is self-declared and published as a profile carrying the environment, the applicable set, per-pattern status, an evidence date and documented gaps. It is not a certification scheme, and no conformity assessment body operates against it. The contribution is architectural rather than empirical. Every pattern carries an evidence level, and no pattern reaches the highest level, because no implementation unconnected to the author has been evaluated. Nothing has been measured. The specification separates what would falsify the invariant from what would falsify an individual pattern and from what would falsify the composition and adoption sequence, poses six research questions, and records the absence of a real implementation profile as a known deficiency of version 1.0. An appendix reconciles the pattern identifiers with the names used across the author's papers and companion book series, including the acronyms PEVG and PARA, so that the two bodies of work can be cited as one. Version 1.1 names two constructs the specification already contained. The central proposition is named the Boundary Invariant, and the three architectural declarations required of every pattern are together named the BOE Declaration. Neither carries a trademark, both are offered for use with attribution under this document's licence, and neither changes any requirement: the proposition, its wording and its priority date are those of version 1.0. Section 11 gains the two-family naming convention and a precedence rule fixing which document governs where this specification and the Defensible AI Framework Registry describe the same relationship.
What if pathology foundation models could do more with less? GigaPath-Flash and GigaTIME-Flash cut computational demands while maintaining strong performance, opening the door to larger studies and broader exploration. The post GigaPath-Flash and GigaTIME-Flash: Toward population-scale discovery with efficient pathology foundation models appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduAug 31, 2026
With millions of users across the world, Julia has been used to conduct cutting-edge research and to design new drugs, jet engines, heat pumps, and more.