Search

Optical Music Recognition (OMR) promises to make accessible the content of large amounts of musical documents, an important component of cultural heritage. However, the field does not have an adequate dataset and ground truth for benchmarking OMR systems, which has been a major obstacle to measurable progress. Furthermore, machine learning methods for OMR require training data. We design and collect MUSCIMA++, a new dataset for OMR. Ground truth in MUSCIMA++ is a notation graph, which our analysis shows to be a necessary and sufficient representation of music notation. Building on the CVC-MUSCIMA dataset for staffline removal, the MUSCIMA++ dataset v1.0 consists of 140 pages of handwritten music, with 91254 manually annotated notation symbols and 82247 explicitly marked relationships between symbol pairs. The dataset allows training and directly evaluating models for symbol classification, symbol localization, and notation graph assembly, and indirectly musical content extraction, both in isolation and jointly. Open-source tools are provided for manipulating the dataset, visualizing the data and annotating further, and the data is made available under an open license.

Dataset creation for graphics recognition, especially for hand-drawn inputs, is often an expensive and time-consuming undertaking. The MUSCIMarker tool used for creating the MUSCIMA++ dataset for Optical Music Recognition (OMR) led to efficient use of annotation resources, and it provides enough flexibility to be applicable to creating datasets for other graphics recognition tasks where the ground truth can be represented similarly. First, we describe the MUSCIMA++ ground truth to define the range of tasks for which using MUSCIMarker to annotate ground truth is applicable. We then describe the MUSCIMarker tool itself, discuss its strong and weak points, and share practical experience with the tool from creating the MUSCIMA++ dataset.

A major roadblock for Optical Music Recognition, especially for handwritten music notation, is symbol detection: recovering the locations of musical symbols from the input page. This has been attempted both with bottom-up approaches exploiting visual features, and top-down approaches based on the strong constraints that music notation syntax imposes on possible symbol configurations; sometimes joined together at appropriate points in the recognition process. The bottom-up approach has recently greatly improved with the boom of neural networks. However, the reduction in uncertainty that music notation syntax can provide has not yet been married to the power of these neural network models. This extended abstract brainstorms ways in which this can be done, and analyzes the difficulties the various combined approaches will have to address. We hope our work will foster further discussion to clarify the issues involed, provoke OMR researchers to try some of these approaches experimentally, and entice researchers from other parts of the graphics recognition community to share relevant experience.