Memory-Aware Joint Optimisation of Partitioning and Scheduling for Pipeline-Parallel Training
OptPipe is presented, a unified framework that jointly optimises partitioning and scheduling for pipeline parallelism and introduces a memory-aware directed acyclic graph (DAG) that captures both task dependencies and the lifetime of intermediate tensors, enabling explicit reasoning about the trade-off between executio...