Search

International audience; This short paper investigates how to improve the confidence of speech balloon segmentation algorithms from comic book images. It comes from the need of precise indications about the quality of automatic processing in order to accept or not each segmented regions as a valid result, according to the application and without requiring any ground truth. We discuss several applications like result quality assessment for companies and automatic ground truth creation from high confidence results to train machine learning based systems.We present some ideas to combine several domain knowledge information (e.g. shape, text, etc.) and produce an improved confidence criterion.

International audience

International audience; Speech text in comic books is written in a particular manner by the scriptwriter which raises unusual challenges for text recognition. We first detail these challenges and present different approaches to solve them. We compare the performances of pre-trained OCR and segmentation-free approach for speech text of comic books written in Latin script. We demonstrate that few good quality pre-trained OCR output samples, associated with other unlabeled data with the same writing style, can feed a segmentation-free OCR and improve text recognition. Thanks to the help of the lexi-cality measure that automatically accept or reject the pre-trained OCR output as pseudo ground truth for a subsequent segmentation-free OCR training and recognition.

International audience; Les bandes dessinées et les mangas sont l'une des formes les plus populaires de contenu graphique dans le monde et jouent un rôle majeur dans la diffusion de la culture. Aujourd'hui, la numérisation massive permet la lecture page par page mais nous pensons que d'autres usages peuvent émerger. Dans cet article, nous nous concentrons sur l'extraction de phylactères qui est une étape majeure pour permettre l'association texte/graphique dans les images de bandes dessinées. En effet, les bulles de dialogues sont à l'interface entre le texte et les personnages, elles informent le lecteur du ton et de la position des locuteurs. Nous présentons une méthode de segmentation de phylactères génériques et indépendante du texte, basée sur les niveaux de gris, la forme et l'organisation topologique des composantes connexes. La méthode a été évaluée sur les ensembles de données publics eBDtheque et Manga109, les résultats de F-mesure obtenus sont respectivement de 78,24% et 80.04%.

International audience

International audience; Speech text in comic books is placed and written in a particular manner by the letterers which raises unusual challenges for text recognition. We first detail these challenges and present different approaches to solve them. We compare the performances of generic versus specifically trained OCR systems for typewritten and handwritten text lines from French comic books. This work is evaluated over a subset of public (eBDtheque) and private (Sequencity) datasets. We demonstrate that generic OCR systems perform best on typewritten-like and lowercase fonts while specifically trained OCR can be very powerful on skewed, uppercase and even cursive fonts.

National audience; Les auteurs proposent une méthode de localisation des personnages dans des cases de bandes dessinées en s'appuyant sur les caractéristiques des bulles de dialogue. L'évaluation montre un taux de localisation des personnages allant jusqu'à 65%.

International audience; Comics and manga are one of the most important forms of publication and play a major role in spreading culture all over the world. In this paper we focus on balloons and their association to comic characters or more generally text and graphic links retrieval. This information is not directly encoded in the image, whether scanned or digital-born, it has to be understood according to other information present in the image. Such high level information allows new browsing experience and story understanding (e.g. dialog analysis, situation retrieval). We propose a speech balloon and comic character association method able to retrieve which character is emitting which speech balloon. The proposed method is based on geometric graph analysis and anchor point selection. This work has been evaluated over various comic book styles from the eBDtheque dataset and also a volume of the Kingdom manga series.

In this thesis, we review, highlight and illustrate the challenges related to comic book image analysis in order to give to the reader a good overview about the last research progress in this field and the current issues. We propose three different approaches for comic book image analysis that are composed by several processing. The first approach is called "sequential'' because the image content is described in an intuitive way, from simple to complex elements using previously extracted elements to guide further processing. Simple elements such as panel text and balloon are extracted first, followed by the balloon tail and then the comic character position in the panel. The second approach addresses independent information extraction to recover the main drawback of the first approach : error propagation. This second method is called “independent” because it is composed by several specific extractors for each elements of the image without any dependence between them. Extra processing such as balloon type classification and text recognition are also covered. The third approach introduces a knowledge-driven and scalable system of comics image understanding. This system called “expert system” is composed by an inference engine and two models, one for comics domain and another one for image processing, stored in an ontology. This expert system combines the benefits of the two first approaches and enables high level semantic description such as the reading order of panels and text, the relations between the speech balloons and their speakers and the comic character identification., Dans ce manuscrit de thèse, nous détaillons et illustrons les différents défis scientifiques liés à l'analyse automatique d'images de bandes dessinées, de manière à donner au lecteur tous les éléments concernant les dernières avancées scientifiques en la matière ainsi que les verrous scientifiques actuels. Nous proposons trois approches pour l'analyse d'image de bandes dessinées. La première approche est dite "séquentielle'' car le contenu de l'image est décrit progressivement et de manière intuitive. Dans cette approche, les extractions se succèdent, en commençant par les plus simples comme les cases, le texte et les bulles qui servent ensuite à guider l'extraction d'éléments plus complexes tels que la queue des bulles et les personnages au sein des cases. La seconde approche propose des extractions indépendantes les unes des autres de manière à éviter la propagation d'erreur due aux traitements successifs. D'autres éléments tels que la classification du type de bulle et la reconnaissance de texte y sont aussi abordés. La troisième approche introduit un système fondé sur une base de connaissance a priori du contenu des images de bandes dessinées. Ce système permet de construire une description sémantique de l'image, dirigée par les modèles de connaissances. Il combine les avantages des deux approches précédentes et permet une description sémantique de haut niveau pouvant inclure des informations telles que l'ordre de lecture, la sémantique des bulles, les relations entre les bulles et leurs locuteurs ainsi que les interactions entre les personnages.

National audience; Les bandes dessinées représentent un patrimoine culturel important dans de nombreux pays. La numérisation en masse offre l'opportunité d'effectuer des recherches sur le contenu des albums et pas uniquement sur des métadonnées associées (e.g. nom de l'auteur ou de la collection). Peu de travaux ont été menés à ce jour. Seule l'extraction des cases et des bulles de dialogues a été étudiée et ce, pour des structures de pages relativement simples. En effet, la structure des pages est propre à chaque auteur, ce qui engendre une très grande diversité de des- sins. Malgré cette diversité, les dessins ont une particularité commune de part leurs méthodes de conception : ils sont constitués ou entourés d'un trait noir. Dans cet article, nous proposons de nous appuyer sur cette particularité des bandes dessinées pour extraire automatiquement les cases et le texte avec une méthode basée sur la classification de composantes connexes. Nous comparerons notre méthode avec des outils de la littérature et discuterons des résultats.

This paper describes the second round of the ICDAR 2019 competition on post-OCR text correction and presents the different methods submitted by the participants. OCR has been an active research field for over the past 30 years but results are still imperfect, especially for historical documents. The purpose of this competition is to compare and evaluate automatic approaches for correcting (denoising) OCR-ed texts. The present challenge consists of two tasks: 1) error detection and 2) error correction. An original dataset of 22M OCR-ed symbols along with an aligned ground truth was provided to the participants with 80% of the dataset dedicated to training and 20% to evaluation. Different sources were aggregated and contain newspapers, historical printed documents as well as manuscripts and shopping receipts, covering 10 European languages (Bulgarian, Czech, Dutch, English, Finish, French, German, Polish, Spanish and Slovak). Five teams submitted results, the error detection scores vary from 41 to 95% and the best error correction improvement is 44%. This competition, which counted 34 registrations, illustrates the strong interest of the community to improve OCR output, which is a key issue to any digitization process involving textual data. Dataset In addition to the paper, you may also be interested in the datasets of the ICDAR 2019 Competition on Post-OCR Text Correction.

International audience; Comic books constitute an important cultural heritage asset in many countries. Digitization combined with subsequent comic book understanding would enable a variety of new applications, including content-based retrieval and content retargeting. Document understanding in this domain is challenging as comics are semi-structured documents, combining semantically important graphical and textual parts. Few studies have been done in this direction. In this work we detail a novel approach for closed and non-closed speech balloon localization in scanned comic book pages, an essential step towards a fully automatic comic book understanding. The approach is compared with existing methods for closed balloon localization found in the literature and results are presented.