Adding a Thai N-gram Engram to Qwen3-ASR-0.6B: effects on transcription error, memory, and latency

Abstract

Technical report, version 1.0.1, in English and Thai, deposited as a preprint. Both editions are included as PDF, in full, one file per language. This version expands the background and introduction, revises the academic prose in both languages, and corrects the parameter count and evaluation-protocol details. The report asks whether a token-level N-gram memory (Engram) reduces Thai transcription errors when added to Qwen3-ASR-0.6B (the evaluation code freezes its base weights), and what three numerical precisions cost in quality, model-tensor storage, peak allocated VRAM, and latency. It is written for a reader who is new to ASR terminology: the terms that carry the argument are defined where they are used. Contents: five chapters covering the problem and scope, the background and model design, the methodology, the results, and the conclusions, followed by acknowledgements, a reproducibility-artifact appendix, and the bibliography. Findings, with the limits that apply to each: - Engram step 750 in BF16 reduced corpus character error rate from 17.8051% to 9.0448% on a fixed 300-clip Thai evaluation set, with character-level micro F1 rising from 86.3235% to 92.9302%. The clips come from the dataset's streaming train split; overlap with Engram's original training data cannot be ruled out because the complete training procedure is unavailable. No confidence intervals were computed. - Quantization cost quality in this setup. NF4 raised CER to 28.7086% for the baseline and 20.9178% for Engram; LLM.int8 raised it to 18.8901% and 10.8413%. Engram NF4 remained 3.1128 CER percentage points above the BF16 baseline, which is a difference in CER and not a 3% loss of accuracy. - Smaller weights were not faster. Both quantized variants ran slower per clip than their BF16 counterparts on this GPU and backend. NF4 reduced model-tensor storage by about 62% and peak allocated VRAM by about 36%; LLM.int8 reduced model-tensor storage by about 29% but increased peak allocated VRAM by about 53–54%. - The two quantizers do not touch the same parameters: NF4 applies to Linear and Embedding modules, LLM.int8 to Linear modules only, leaving embeddings in BF16. This is a limitation of the comparison and the report says so. The report is explicit that these are measurements of one ROCm/bitsandbytes configuration on one GPU, not general performance claims, and that the Engram training loop is not part of the deposit. Quantized GGUF inference is out of scope and no number comes from llama.cpp. The Thai speech data comes from the Porjai-Thai-voice-dataset-central corpus (CMKL University, CC BY-SA 4.0), which is not redistributed; the 300 clips used are identified only by offset and by a frozen reference list. The base model is Qwen3-ASR-0.6B from the Qwen Team.