New method aims to keep kids safe from illegal AI-generated content
Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs.
More from the blog
How an MIT research project became a global programming language
With millions of users across the world, Julia has been used to conduct cutting-edge research and to design new drugs, jet engines, heat pumps, and more.
Looking beyond natural sequences
A new machine-learning framework aims to improve the success rate of computational protein design while moving away from results that reproduce sequences found in nature.
AI helps design new materials that work in the real world
The “CrysVCD” tool developed at MIT could cut the huge amounts of time and money spent on screening out chemically unstable designs.
AgentHands: Generating interactive hand gestures for spatially grounded agent conversations in XR
Human-Computer Interaction and Visualization
Related papers
Constitutional Midtraining: Content Presence Drives Alignment Gains
Post-training alignment is often shallow, eroding under fine-tuning. It remains untested as to whether constitutional midtraining interventions can produce durable alignment when cleanly isolated from post-training. We build a 394M-token constitutional corpus from Anthropic's Constitution and apply constitutional midtraining at 120B scale, where principled, values-based content is inserted into midtraining. A 2x2 design (curriculum ordering x deliberative reasoning) was used to produce four constitutionally midtrained conditions, plus a control, which were evaluated on self-generated and established benchmarks including alignment under pressure, value conflict resolution, blackmail, and emergent misalignment. All models were evaluated across three stages: post-midtraining, post-SFT, and post-benign fine-tuning. Constitutionally midtrained models outperformed the control on alignment generalization and durability, notably on blackmail: SFT instilled a blackmail propensity in all models, but constitutional midtraining blunted it, with the advantage surviving benign fine-tuning (-17.5pp). This durability did not extend to settings that required active resistance to in-context pressure or conflict, where the advantage attenuates after SFT. The presence of constitutional content at midtraining also mattered more than its structure, and constitutional midtraining incurred no capability cost, on average, at any stage (MMLU, ARC-Easy, piqa, GSM8K). A modest amount of constitutional content at midtraining could therefore yield broad, persistent alignment gains, offering a cheap, complementary addition to SFT-centered pipelines. Code, data, and models are available.
MEASURING THE IMPACT OF GENERATIVE AI ON SOFTWARE TEAM PRODUCTIVITY AND OUTPUT QUALITY IN AGILE ENVIRONMENTS
Generative artificial intelligence (GAI) is becoming more incorporated into software engineering functions like code creation, debugging, requirement analysis, testing, and sharing knowledge. This research looks at how GAI affects software teams in terms of productivity and quality of the output in Agile environments. The research design used is quantitative, cross-sectional survey type using a questionnaire prepared for this research. The data used consists of 35 responses, with 34 usable cases in analyzing 30 Likert items. The measuring instrument consists of six concepts: use of GAI, efficiency of the software team, quality of the software output, GAI in Agile, team collaboration, and communication, and overall impact perceived. The descriptive results show positive feelings about the six concepts. The values on the mean for the different concepts varied from 3.54 to 3.78 on a scale of five, with GAI being the concept that received the highest mean (M = 3.78, SD = 0.47) while productivity was the one that received the lowest (M = 3.54, SD = 0.69). The instrument has a high level of internal consistency with α = 0.799 for the entire scale of 30 items. In terms of specific items, productivity, quality, and team collaboration had acceptable reliability, whereas GAI had low internal consistency and Agile and general have the upper limit of reliability therefore, construct-level findings should be interpreted cautiously. Pearson correlation analysis showed statistically significant positive associations between overall perceived impact and software output quality (r = 0.365, p = 0.034) and team collaboration and communication (r = 0.371, p = 0.031). Productivity was positively associated with overall impact but did not reach the conventional 0.05 significance level (r = 0.312, p = 0.073). In a multiple regression model, the five dimensions explained 23.5% of the variance in overall perceived impact (R² = 0.235); however, the overall model was not statistically significant (F(5, 28) = 1.719, p = 0.163). These findings support a cautious interpretation: respondents generally perceive GAI positively, but the present small sample does not provide strong evidence for broad causal claims.
AI and Bullshit
It is argued that both AI and bullshitters are untrustworthy informants, and for similar reasons, it is natural to describe AI’s informational outputs as bullshit, as it signals their distinctive kind of epistemic deficiencies, which they share with bullshit.
LLMs Leak Training Data Beyond Verbatim Memorization: Extraction via Membership Decoding
The Membership Decoding method is a plug-and-play replacement for standard decoding that requires only black-box token probabilities, and a new token-level membership inference method is proposed by leveraging likelihood from reference models, shifting the generation from the original token distribution to the member token distribution.