Skip to content

Representation, Retrieval, and Decision Space: A Case Study on Diagnosing LLM Failures in Column Type Annotation

· 0 citations · 17 references

TL;DR

A case study diagnosing LLM failures in CTA from three perspectives: representation bias, retrieval incompleteness, and decision-space complexity suggests that improving LLM-based CTA requires targeted interventions tailored to error profiles rather than a generic fix.

View source

Similar papers

#artificial intelligence Preprint Sep 2026

From Confusion to Clarity: Confusion-Aware Retrieval and Knowledge Injection for Text Classification

A framework is proposed that identifies which label pairs the model struggles to distinguish, expands the candidate set to include confusable labels, and generates targeted rules to differentiate between similar candidates, which requires no fine-tuning and transfers to smaller, cheaper models.

Manish Gupta, Chaitanya Giri, Jayasimha Talur · 0 citations
Preprint Aug 2026

From Diagnosis to Correction: Benchmarking and Improving Real-World Table Parsing

Decompose--Enhance--Correct (Decompose--Enhance--Correct), a visual-consistency-guided agentic framework that improves frozen table parsers without retraining is proposed, which derives a 1,977-table Consensus-Hard Set from 4,556 candidates through offline metrics and cross-model consensus.

Jutao Xiao, Yuan Qu, Dongsheng Ma et al. · 0 citations
#artificial intelligence Preprint Aug 2026

When Do LLMs Actually Help? Evaluating LLMs as Data Quality Annotators

Testing an LLM on two e-commerce data quality tasks, entity matching and brand mislabeling, against rule based baselines and human verified ground truth, under both zero-shot and few-shot prompting suggests that the value of using an LLM over traditional methods depends heavily on the task.

Praphulla Lal Shrestha · 0 citations
Preprint Aug 2026

Factorized Hypothesis Search for Evidence-to-Taxonomy Retrieval

This work proposes Factorized Hypothesis Search (FHS), which maintains multiple partial interpretations over named semantic dimensions that support structured query rendering, multi-hypothesis retrieval, and dimension-level candidate verification.

Lin-Hai Ma, Ethan F. Wei, Xue-Qing Peng et al. · 0 citations
#machine learning Preprint Aug 2026

R2VC: Modular Fact-Checking with Retrieval, Verification, and Confidence Calibration

Large language models are increasingly used for automated fact checking, but end-to-end prompting often entangles evidence retrieval, reasoning, and uncertainty estimation, making failures difficult to diagnose and confidence difficult to trust. We present R2VC, a modular retrieve, reason, verify, calibrate architectur...

Dhruv Dixit, Paritosh Pandey · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.