Attend to Fragments: How Key Information Affects Large Language Models for Factual Inconsistency Detection

Thesis Proposal: Intentional Inference for Insight Generation

Large language models (LLMs) show strong capabilities in natural language generation (NLG) and have been applied to translate complex structured data into human-readable insights. While these models excel at surface-level fluency, they remain unreliable as they produce factually inaccurate outputs and struggle with consistent logical inference beyond surface-level patterns. Moreover, they often lack a clear sense of relevance and produce shallow or un-informative insights. This proposal argues that a key source of these limitations is task underspecification, which requires models to make implicit assumptions about missing context. We investigate how such underspecification leads to unintentional assumptions and how these affect faithfulness and evaluation. We examine how models can identify missing premises and surface multiple plausible interpretations to make evaluation more rigorous. We also explore how to improve reasoning to enable deeper inferences, focusing on code generation and qualitative reasoning. Finally, we will evaluate how the underlying assumptions and depth of inference influence the perceived interestingness of the insights. By shifting focus from surface-level generation to assumption-aware deeper inferences, this work aims to improve reliability, interpretability, and user controllability in NLG.

Tokens to Types: Context Editing with Selective Entity Abstraction for Grounded Generation

Large language models (LLMs) frequently prioritize parametric world knowledge over provided context -- a failure mode that is particularly catastrophic in enterprise or counterfactual settings where local facts contradict web-scale training data. While modern reasoning models improve general response quality, they fail to resolve these underlying prior knowledge biases even when generating a high volume of costly thinking tokens. We propose a context-editing framework that addresses this by performing selective abstraction over entities that appear in both the context and the question. Our approach replaces these overlapping entities with typed, indexed identifiers (e.g., Paris with City łangle1\rangle), suppressing spurious lexical associations while preserving the relational structure required for grounded reasoning. This mechanism is decoupled into an offline preprocessing stage and a lightweight inference-time substitution, requiring no model retraining. Experiments across counterfactual and multi-hop knowledge-conflict benchmarks demonstrate consistent accuracy gains across many model families, open- and closed-sourced, especially for small to medium sized LLMs (0.5B to 18B). Notably, our framework yields up to a 25% improvement over state-of-the-art instruct models and 13% over reasoning models, establishing symbolic abstraction as a highly cost-efficient solution for ensuring context fidelity in LLMs.

Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

The results demonstrate that suppressing entity-linked supervision during pretraining induces a shift in epistemic behavior: KLLMs rely less on parametric knowledge and more on external evidence, leading to improved reliability under realistic conditions, suggesting that pretraining-time control over knowledge acquisition can complement retrieval-augmented and tool-based systems by providing a more evidence-sensitive base model.

Reversing Arrows in Large Language Models

This work presents the first systematic study of inverse relation directionality in LLMs, using a benchmark consisting of 5,457 instances spanning 27 distinct inverse relation labels and reveals systematic asymmetries in inverse relation classification across LLMs.

When Absence Is Evidence: Evaluating Completeness-Sensitive Negative Reasoning in Large Language Models

Large language models (LLMs) are often asked whether something is absent from a record, list, or retrieved context. Yet non-observation licenses a negative answer only when evidence completely covers the query scope; otherwise, the answer should remain unknown. We call this completeness-sensitive negative reasoning. We introduce CROWN-QA, comprising CROWN-Synth, a controlled paired core that fixes the question and observed facts while varying only query-relative coverage, and CROWN-Real, a real-document contrast-set evaluation with controlled coverage variants. Across three LLM families, models show unstable closure judgments and substantial over-closure, failing to reliably distinguish a justified negative answer (Certified-Negative) from insufficient evidence (Unknown). The dominant CROWN-Synth failure is asymmetric: models often recognize implicitly complete evidence yet treat implicitly partial evidence as query-covering. Prompting redistributes errors between over- and under-closure rather than consistently resolving them. Structured certificate elicitation traces many errors to evidence-coverage mischaracterization. CROWN-Real shows that the core partial-coverage asymmetry persists on real-document content, while its strength and the balance between over- and under-closure vary by model, prompt, and source.

TEMPORAL PROMPTING FOR LARGE LANGUAGE MODELS-BASED ON ENTITY LINKING