Author

Samuel Madden

1 paper indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Low-Latency Semantic Processing with Optimal Prompt-Level Batching

This paper measures and analytically derive an optimal size for in-prompt document batching to effectively amortize this overhead of LLM calls over public APIs, cutting end-to-end semantic operator latency by up to 14 × with no meaningful accuracy loss.

Jingyi Qu, Samuel Madden, Tianyu Li · 0 citations