Recognition of Doctors’ Cursive Handwritten Medical Words by using Bidirectional LSTM and SRP Data Augmentation

Inability to read doctors’ handwritten prescriptions causes 7,000 deaths a year in a developed country like the US. The situation should be worse in developing countries where more doctors use handwriting prescriptions. In Bangladesh, the writings become more indecipherable as they contain both English and Bangla words with Latin abbreviations of medical terms. As a result, patients and pharmacists find them difficult to read and the pharmacists provide wrong medicines. In order to ease the difficulty of reading doctors’ prescriptions, this paper proposes an online handwritten recognition system to predict the doctors’ handwriting and develop a digital prescription. To build this system, the “Handwritten Medical Term Corpus” dataset is introduced which contains 17,431 data samples of 480 words (360 English and 120 Bangla) from 39 Bangladeshi doctors and medical professionals. A bigger sample size can improve the recognition efficiency. A new data augmentation technique SRP (Stroke Rotation and Parallel shift) method is proposed to widen the variety of handwriting styles and increase the sample size. A sequence of line data is extracted from the augmented image dataset of 1,591,100 samples which is fed to a Bidirectional LSTM model. The proposed method has achieved 89.5% accuracy which is 16.1% higher than the recognition accuracy with no data expansion. This technology can reduce medical errors and save medical cost and ensure healthy living.