Medical Specialty Classification of Clinical Transcriptions using TF-IDF N-Grams and a Linear Support Vector Machine

Abstract

Automated classification of clinical documents by medical specialty is critical for tasks such as document routing, billing code assignment, and clinical decision support. In this study, we perform 10-class medical specialty classification on the publicly available MTSamples corpus using a Linear Support Vector Classifier (LinearSVC) trained on combined word (1-2) and character (4-6) n-gram TF-IDF features extracted from the description, keywords, and full transcription fields of 2,904 clinical records. Using an 80/20 stratified train/test split, the model achieved 76.62% test accuracy (95% bootstrap CI: 73.59-79.50%), with a weighted F1 of 0.770 (95% CI: 0.741-0.797) and a macro F1 of 0.750, exceeding the 10% random baseline by 7.66×. Seven of ten classes attained F1 scores above 0.75, with Surgery (0.92), SOAP/Chart/Progress Notes (0.88), Radiology (0.87), and Urology (0.85) performing best, while Consult-History & Physical remained the most challenging (F1 = 0.53). Key findings highlight that character ngram features improve performance over word n-grams alone, inclusion of the clinical keywords field significantly enhances discriminative ability, and balanced class weighting mitigates corpus imbalance. This work establishes a transparent, interpretable baseline and provides per-class insights for clinical NLP validation.