A Lightweight BERT-Variant Optimized for WebGPU-Based Real-TimeInference in Web Browsers

Abstract

Large Language Models (LLMs) achieve state-of-the-art performance across natural language processingtasks but remain computationally intensive, limiting their deployment in browser-based environments.This study investigates the feasibility of real-time transformer inference entirely within web browsers usingWebGPU acceleration. We propose a lightweight BERT-inspired architecture optimized for GPUparallelmatrix operations through TensorFlow.js with the WebGPU backend. The model is evaluated ona 4,000-sample IMDB sentiment classification dataset and achieves 65–70% classification accuracy withper-sample inference latency of 8–9 ms in Google Chrome using an NVIDIA T400 GPU (4 GB VRAM).Batch inference throughput reaches approximately 6,600 inferences per minute, while GPU utilization remainsstable between 26–77%. Compared to CPU-based TensorFlow.js execution, WebGPU significantlyreduces inference latency and enables fully client-side training and inference without server dependencies.Although accuracy is lower than full-scale BERT benchmarks, the results demonstrate that simplified transformerarchitectures can operate efficiently in browser environments under resource constraints. This workestablishes a practical framework for deploying real-time edge NLP applications using open web standardsand GPU acceleration.