Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey
This survey provides a systematic analysis of multi-LLM routing and cascading approaches, focusing on systems that route queries across a pool of independently trained LLMs at inference time, and demonstrates that effective multi-LLM routing requires balancing competing objectives.