Skip to content
#natural language processing Preprint Open access

ProfileFoundry: A Synthetic Person-Object Substrate for Privacy, Memory, and Tool-Use Evaluation in LLM Agent

Sriram Selvam Anneswa Ghosh
Aug 2026
Natural Language Processing

Abstract

Foundation-model research increasingly needs data about people: user state, personal histories, relationships, contact-like fields, documents, and longitudinal updates. Real user data is difficult to share, perturb, audit, or redistribute responsibly, while independently generated fake fields rarely preserve the cross-field and temporal consistency needed for controlled evaluation. We present ProfileFoundry, a deterministic generator and fixed reference release of 100,000 adult synthetic Person Objects across eight locales. Each object combines a typed current snapshot, household, family, and employer links, snapshot-aligned events, normalized relational views, and generation provenance. The release contains 709,228 events, 40,338 households, 52,491 employers, and 518,564 directed relationship edges. We report evidence in separate categories: selected population-marginal comparisons, per-object invariant checks, release-wide referential and temporal closure, and coincidence/provenance screens. A pilot case study, MatchDesk, uses certified coincidences, typed events, and history truncation to evaluate whether models distinguish corroborated identity evidence from underdetermined matches. ProfileFoundry is not a population-fidelity model, a rendered-text corpus, or a formal privacy mechanism. Instead, it is a responsible synthetic source layer for constructing downstream foundation-model evaluations involving memory, privacy, document understanding, record linkage, and agent state while keeping the synthetic person behind each artifact inspectable.

View source

Similar papers

Jailbreaking in the Haystack

NINJA (short for Needle-in-haystack jailbreak attack), a method that jailbreaks aligned LMs by appending benign, model-generated content to harmful user goals to reveal fundamental vulnerabilities in modern LMs.

R. Shah, C. Wu, Shashwat Saxena et al. · 4 citations

SimulRAG: Simulator-based RAG for Grounding LLMs in Long-form Scientific QA

SimulRAG, a simulator-based RAG framework with a generalized retrieval interface that translates between text and simulator parameters/outputs, is proposed, which improves informativeness and factuality over the strongest adapted RAG baselines, while UE+SBA enhances claim-level efficiency and quality.

Haozhou Xu, D. Wu, M. Chinazzi et al. · 3 citations

Convergent Evolution: How Different Language Models Learn Similar Number Representations

This paper identifies two different routes through which models can acquire geometrically separable features: they can learn them from complementary co-occurrence signals in general language data, including text-number co-occurrence and cross-number interaction, or from multi-token addition problems.

Deqing Fu, Tianyi Zhou, Mikhail Belkin et al. · 3 citations
#artificial intelligence Preprint Aug 2026

Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactions

Interactions are introduced as a fine-grained tool to analyze prompt sensitivity of LLMs and it is discovered that subtle changes to prompts can trigger severe instability in interactions, even when the outputs of the LLM remain the same.

Ruiyang Qin, Qingzhuo Wang, Tianhao Wang et al. · 2 citations · ⚡1

When Audio-Language Models Fail to Leverage Multimodal Context for Dysarthric Speech Recognition

A benchmark built on the Speech Accessibility Project (SAP) dataset is introduced that tests whether diagnosis labels, clinician-derived speech ratings, and progressively richer clinical descriptions improve transcription accuracy for dysarthric speech, finding that current models do not meaningfully use this context.

P. Moure, Niclas Pokel, Bilal Bounajma et al. · 2 citations

Making Implicit Premises Explicit in Logical Understanding of Enthymemes

A pipeline that integrates a large language model to generate intermediate implicit premises based on the explicit premise and claim, a neuro-symbolic reasoner based on a SAT solver to determine entailment, and a neuro-symbolic reasoner based on a SAT solver to determine entailment is proposed.

Xuyao Feng, Anthony Hunter · 2 citations

Related blog posts

MIT News · Artificial Intelligence Aug 27, 2026

Looking beyond natural sequences

A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.