Data, data science, and official statistics

Abstract

The emergence of data science, big data, and new digital sources has profoundly changed the environment in which official statistics are produced. However, many of the challenges associated with these developments are not new: they concern the definition of statistical units, the construction and validation of data, the treatment of errors, and the extrapolation of partial information to well-defined populations. This article examines the historical and methodological relationships between official statistics and modern data science. It argues that data are not simply available objects but the outcome of complex production processes involving measurement, collection, correction, and validation. The evolution from exhaustive censuses to probability sampling, administrative data, and integrated data sources is reviewed, with particular emphasis on the continuing importance of inferential principles. The paper also discusses the role of modern machine learning methods, showing that many techniques presented as innovations are extensions or rediscoveries of classical statistical ideas. Finally, it examines the main challenges facing official statistics, including declining survey response rates, the use of administrative and alternative data sources, data integration, confidentiality, technological change, and the need for highly qualified statistical expertise. The future of official statistics will depend on combining new computational opportunities with the methodological foundations that ensure reliable and trustworthy information.