Henry S. Hunt

An exact information theory of generalization phase transitions in Bayesian diffusion models

A fundamental role for information restriction in generative AI to circumvent the curse of dimensionality is revealed: both spatially local BIRD models and early-training diffusion models track the memorization-generalization phase boundary by increasingly restricting information over time.