Search

This paper presents an attention-based row-column encoder-decoder (ARCED) model for recognizing an input image of multiple text lines from Japanese historical documents without explicit segmentation of lines. The recognition system has three main parts: a feature extractor, a row-column encoder, and a decoder. We introduce a row-column BLSTM in the encoder and a residual LSTM network in the decoder. The whole system is trained end-to-end by a standard cross-entropy loss function, requiring only document images and their ground-truth text. We experimentally evaluate the performance of ARCED on the dataset of Japanese historical documents: Kana-PRMU. The results of the experiments show that ARCED outperforms the state-of-the-art recognition methods on the dataset. Furthermore, we demonstrate that the row-column BLSTM in the encoder and the residual LSTM in the decoder improves the performance of the encoder-decoder model for the recognition of Japanese historical document.

Offline handwritten text recognition is still a big challenging problem due to various backgrounds, noises, diversity of writing styles, and multiple touches between characters. In this paper, we propose a model of 2D Self-Attention Convolutional Recurrent Network (2D-SACRN) for recognizing handwritten text lines. The 2D-SACRN model consists of three main components: 1) a 2D self-attention based convolutional feature extractor that extracts a feature sequence from an input image; 2) a recurrent encoder that encodes the feature sequence into a sequence of label probabilities; and 3) a CTC-decoder that decodes the sequence of label probabilities into the final label sequence. In this model, we present a 2D self-attention mechanism in the feature extractor to capture the relationships between widely separated spatial regions in an input image. In the experiment, we evaluate the performance of the proposed model on the three datasets: IAM Handwriting, Rimes, and TUAT Kondate. The experimental results show that the proposed model achieves similar or better accuracy when compared to state-of-the-art models in all datasets.

In this paper, we propose an RNN-Transducer model for recognizing Japanese and Chinese offline handwritten text line images. As far as we know, it is the first approach that adopts the RNN-Transducer model for offline handwritten text recognition. The proposed model consists of three main components: a visual feature encoder that extracts visual features from an input image by CNN and then encodes the visual features by BLSTM; a linguistic context encoder that extracts and encodes linguistic features from the input image by embedded layers and LSTM; and a joint decoder that combines and then decodes the visual features and the linguistic features into the final label sequence by fully connected and softmax layers. The proposed model takes advantage of both visual and linguistic information from the input image. In the experiments, we evaluated the performance of the proposed model on the two datasets: Kuzushiji and SCUT-EPT. Experimental results show that the proposed model achieves state-of-the-art performance on all datasets.

Handwritten Japanese text recognition is still a big challenging task due to the large character set, diversity of writing styles, and multiple-touches between characters. In this paper, we propose a model of Attention Augmented Convolutional Recurrent Network (AACRN) for recognizing handwritten Japanese text lines. The AACRN model has three main parts: a convolutional feature extractor, a self-attention based encoder, and a CTC-decoder. The whole model can be trained end-to-end. In the experiment, we evaluate the performance of the AACRN model on the TUAT Kondate dataset and the Kuzushiji dataset. The results of the experiments show that the proposed model achieves higher performance than the state-of-the-art recognition accuracies on the test set of TUAT Kondate and the Kuzushiji dataset.

The Wartegg Zeichen Test (WZT) is a method of personality evaluation developed by the psychologist Ehrig Wartegg. Three new scoring categories for the WZT consist of Evocative Character, Form Quality, and Affective Quality. In this paper, we present the object detection model in scoring the Affective Quality of WZT. Our works consist of two main parts: 1) using the YOLO (You Only Look Once) model to detect the hand-drawn square boxes from the WZT form, and 2) using YOLO to detect the object in the hand-drawn square box. In the experiments, YOLOv3 achieved 88.94% of mAP for hand-drawn square box detection and 46.90% of mAP for hand-drawn object detection.

Inspired by the recent successes of attention based encoder-decoder (AED) approach on image captioning, machine translation, we present an AED model as an end-to-end recognition system for recognizing Japanese historical documents. The recognition system has two main modules: a dense convolution neural network for extracting features, and a Long Shor Term Memory (LSTM) decoder integrating with attention model for generating target text. We can train the model end-to-end. The model requires only input text line images and corresponding output characters. Therefore, we don't need annotations for characters and save a lot of time for making annotations. We also present a method to generate artificial text lines to solve the imbalance problem of the current annotated database. The results of experiments on the annotated and artificial databases demonstrate the effectiveness of the text line generation. Our recognition system achieved Character Error Rate of 23.76% and 22.52% by training with and without artificial text lines, respectively. Moreover, our recognition system outperforms the CNN-LSTM system, which achieved the state-of-art results in other document recognition tasks.

This paper presents an attention-based convolutional sequence to sequence (ACseq2seq) model for recognizing an input image of multiple text lines from Japanese historical documents without explicit segmentation of lines. The recognition system has three main parts: a feature extractor using Convolutional Neural Network (CNN) to extract a feature sequence from an input image; an encoder employing bidirectional Long Short-Term Memory (BLSTM) to encode the feature sequence; and a decoder using a unidirectional LSTM with the attention mechanism to generate the final target text based on the attended pertinent features. We also introduce a residual LSTM network between the attention vector and softmax layer in the decoder. The system can be trained end-to-end by a standard cross-entropy loss function. In the experiment, we evaluate the performance of the ACseq2seq model on the anomalously deformed Kana datasets in the PRMU contest. The results of the experiments show that our proposed model achieves higher recognition accuracy than the state-of-the-art recognition methods on the anomalously deformed Kana datasets.

2013年度～2017年度科学研究費補助金基盤研究(S) 研究成果報告書（課題番号25220401)

This paper presents an end-to-end model of Deep Convolutional Recurrent Network (DCRN) for recognizing offline handwritten Japanese text lines. The end-to-end DCRN model has three parts: a convolutional feature extractor using Deep Convolutional Neural Network (DCNN) to extract a feature sequence from a text line image; recurrent layers employing a Deep Bidirectional LSTM to predict pre-frame from the feature sequence; and a transcription layer using Connectionist Temporal Classification (CTC) to convert the pre-frame predictions into the label sequence. Since our end-to-end model requires a large data for training, we synthesize handwritten text line images from sentences in corpora and handwritten character patterns in the Nakayosi and Kuchibue database with elastic distortions. In the experiment, we evaluate the performance of the end-to-end model and the effectiveness of the synthetic data generation method on the test set of the TUAT Kondate database. The results of the experiments show that our end-to-end model achieves higher than the state-of-the-art recognition accuracy on the test set of TUAT Kondate with 96.35% and 98.05% character level recognition accuracies without and with the generated synthetic data, respectively.

This paper presents methods for three different tasks of recognizing anomalously deformed Kana in Japanese historical documents, which were contested by IEICE PRMU1 2017. The tasks have three levels: single character recognition, three Kana characters sequence recognition and unrestricted Kana recognition. We compare several methods for each task. For the level 1, we evaluate CNN based methods and BLSTM based methods. For the level 2, we consider several variations of a combined architecture of CNN and BLSTM. For the level 3, we compare an extension of the method for the level 2 and a segmentation based method. We achieve the single character recognition accuracy of 96.8%, the three Kana characters sequence recognition accuracy of 87.12% and the unrestricted Kana recognition accuracy of 73.3%. These results prove the performance of CNN and BLSTM on these tasks.

This paper presents a model of Deep Convolutional Recurrent Network (DCRN) for recognizing offline handwritten Japanese text lines without explicit segmentation of characters. Most of traditional offline handwritten Japanese text recognizers perform segmentation of text image into characters before individually recognizing each character. Although segmentation by recognition and context are employed to recover from segmentation errors, errors made at this stage directly make an impact on the performance of the whole system. The DCRN model consists of three parts: a convolutional feature extractor using Convolutional Neural Network (CNN) and sliding window to extract features from text image; recurrent layers using BLSTM to predict pre-frame from an input sequence; and a transcription layer using a CTC-decoder to translate the predictions into the label sequence. Experimental results on the database: TUAT Kondate database demonstrates the effectiveness of the proposed method.