AI Networking Cookbook: Practical recipes for AI-assisted network automation and development
TL;DR
This work proposes EHV, an Efficient Hierarchical ViT architecture designed for massive scaling, and introduces a Sparse Mixture-of-Experts (SMoE) variant, pushing the scale to an industry-leading 30B parameters.
Recent advances in single image super-resolution (SISR) have leveraged convolutional neural networks (CNNs) and vision transformers to model pixel-level statistics, often relying on increasingly complex architectures to capture spatial correlations. However, these approaches generally overlook a fundamental distinction...
Asynchronous Learning Rate is proposed, which assigns depth-dependent initial learning rates to network modules according to their topological depth, and Smoothed Synchronous Decay is proposed, which coordinates the subsequent decay of heterogeneous parameter groups.
This work systematically study MoE designs for vision encoder scaling and finds that fine-grained MoE topologies yield substantial gains over both dense and standard MoE counterparts, and proposes an auxiliary-loss-free balancing variant for better expert utilization, and designs a specialized MoE kernel to mitigate in...
This work introduces HiRAE (Hierarchical Representation Autoencoder), which learns a hierarchical fusion framework over the full encoder hierarchy to improve reconstruction fidelity while maintaining compatibility with generative modeling.
While Mamba-based models have shown strong potential for long sequence modeling, adapting them to vision is challenging due to the requirement of local neighborhood correlations and multi-directional spatial contexts for visual understanding. In this paper, we present LiAuto-MindViT, a novel hybrid vision backbone that...
Scaling has become a primary driver of progress in language and vision foundation models, yet its role in precise correspondence matching remains underexplored. In this work, we present Flow Any Scene Transformer (FAST), a scalable correspondence model driven by two key insights. First, we reveal that the query-key pro...
We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.