Bin-Bin Feng

Dynamic Pipeline Inference Optimization for LLMs: Load-Aware Partitioning, Updating, and Migration

With the rapid advancement of deep learning technology, the parameter scale of large language models has grown exponentially, expanding from hundreds of millions in the early stages to hundreds of billions or even trillions today. Pipeline inference is a crucial approach enabling efficient inference in large language m...