Multimodal Electronic Health Record Extraction and IoT-Based Patient Monitoring using Artificial Intelligence

Abstract

Electronic Health Records (EHRs) form the backbone of modern healthcare, yet a large fraction of clinical data remains trapped in handwritten or scanned documents, constraining efficient data access and real-time clinical decisions. The Proposed System presents a multimodal AI pipeline that utilizes OCR, biomedical NLP, LLM-based validation, and IoT based patient monitoring to automatically extract, validate, and structure medical information from unstructured EHRs. The pipeline consumes scanned medical records, performing image preprocessing, layout analysis, OCR, biomedical NER and semantic validation, before fusing this knowledge with real-time physiological data from IoT sensors. The proposed system was evaluated on 10 anonymized medical records comprising printed text, Prescription, Lab-reports, Bills, Discharge Summaries using a 70:15:15 train-validation-test split. Performance was assessed using OCR accuracy, entity extraction precision, recall, F1-score, and end-to-end extraction accuracy of 85%. The framework includes strategies for dealing with noisy scans and making sense of contradictory clinical information. The framework provide a promising approach offers a hopeful path forward to smarter healthcare data digitization, clinical decision support and continuous patient monitoring with less manual note-taking overhead and more smart healthcare interoperability.