This paper attempts to solve the well known text classifi-cation task of fine-grained domain classification using BERT and RoBERTa and performs a comparative analysis of the same and evaluates the impact of data preprocessing specially in the context of fine-grained domain classification.
This work proposes a student design based on simple, homogeneous blocks mirroring those of the teacher, distilling knowledge between corresponding blocks, showing that intermediate block-wise distillation, guided appropriately, is key to building compact data-efficient models without sacrificing accuracy.
Irene Trigueros-Lorca, Leonardo Concepción, Christian Wagner et al.· 0 citations
Mastering Transformers will help you understand and implement multimodal solutions, including text-to-image and computer vision solutions that are based on transformers, in NLP and CV.
Treating supervision format as a first-class hyperparameter for multi-task reasoning SFT in large language models—at least in this benchmark-and-model setting—rather than a mere rendering detail is supported.
Nhat Thanh Vu, M. Rashid, Fariza Sabrina· Electronics· 0 citations
An algorithm framework named DR-EFT (Domain-Representative Experts for Fine-Tuning), which explores and loads the domain-representative experts for subsequent retraining and reincorporation and demonstrates robustness through validations on popular MoE LLMs, including Qwen, DeepSeek, and Ernie.
Zhaomeng Cheng, Zhong Ji, Yan Zhang et al.· Neural Networks· 0 citations
This article presents a procedure to deal with and treat unknown languages or languages without available resources such as Kicongo, a Bantu matrix language, commonly spoken in the northern regions of Angola.
Henriques Fernando, Rodrigo Florêncio da Silva· Revista Brasileira de Comput...· 0 citations
Recursion Transformers are studied, reusing a shared block across depth to scale compute, together with factorized embeddings to reduce vocabulary-map parameters, and find that they outperform standard Transformers at 10M and 100M words, while remaining competitive with BabyLM Challenge 2025 winners.
Serdar Gülbahar, Lukas Edman, Alexander Fraser· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
