Ying-Ying Cheng

InferOpt: Constrained Multi-Objective Search for LLM Inference Configurations

Serving an LLM means setting dozens of inference-time knobs, from per-layer KV retention to per-layer expert counts. Practice sets them with mechanism-specific heuristics that return a single operating point and do not scale to layer-wise search spaces. We recast inference configuration as constrained multi-objective b...