Search

Given the need to describe, analyze and index large quantities of marine imagery data for exploration and monitoring activities, a range of specialized image annotation tools have been developed worldwide. Image annotation - the process of transposing objects or events represented in a video or still image to the semantic level, may involve human interactions and computer-assisted solutions. Marine image annotation software (MIAS) have enabled over 500 publications to date. We review the functioning, application trends and developments, by comparing general and advanced features of 23 different tools utilized in underwater image analysis. \{MIAS\} requiring human input are basically a graphical user interface, with a video player or image browser that recognizes a specific time code or image code, allowing to log events in a time-stamped (and/or geo-referenced) manner. \{MIAS\} differ from similar software by the capability of integrating data associated to video collection, the most simple being the position coordinates of the video recording platform. \{MIAS\} have three main characteristics: annotating events in real time, in posteriorly to annotation and interact with a database. These range from simple annotation interfaces, to full onboard data management systems, with a variety of toolboxes. Advanced packages allow to input and display of data from multiple sensors or multiple annotators via intranet or internet. Posterior human-mediated annotation often include tools for data display and image analysis, e.g. length, area, image segmentation, point count; and in a few cases the possibility of browsing and editing previous dive logs or to analyze annotation data. The interaction with a database allows the automatic integration of annotations from different surveys, repeated annotation and collaborative annotation of shared datasets, browsing and querying of data. Progress in the field of automated annotation is mostly in post processing, for stable platforms or still images. Integration into available \{MIAS\} is currently limited to semi-automated processes of pixel recognition through computer-vision modules that compile expert-based knowledge. Important topics aiding the choice of a specific software are outlined, the ideal software is discussed and future trends are presented.

Deep-sea sponge aggregations are widely recognised as features of conservation interest and vulnerable marine ecosystems that may particularly require protection from the impact of commercial bottom trawl fishing. In 2011 we revisited deep-sea sponge aggregations in the Porcupine Seabight (NE Atlantic, c. 1200 m water depth) originally described by Rice, Thurston and New (1990, Prog. Oceanogr. 24: 179-196) from surveys in 1983/4. Using an off-bottom towed camera system, broadly comparable to the bottom-towed system originally employed, we resurveyed four key transects detailed in that publication. In the intervening years, there has been a substantial increase in deep-water fishing activity; our primary objectives were therefore to establish the continued presence of Pheronema carpenteri (Hexactinellida, Pheronematidae), the current status of the sponge population, and whether there was any evidence of bottom trawl fishing impact on the sponges and their associated fauna. We noted a very substantial reduction in the standing stock of sponges: in Rice et al.'s (loc. cit.) peak abundance depth range (1210 – 1250 m) numerical density declined from 1.09 to 0.03 ind m-2, and biomass density from 246 to 4 gwwt m-2, between the surveys. Our assessment of available vessel monitoring data suggested that commercial bottom trawling had been occurring in the area, with some indication of focussed effort in the sponge's bathymetric range. We also recorded the presence of multiple apparent seafloor trawl marks on two of the transects. Despite the potential disturbance, the presence of sponge aggregations continued to exert a statistically significant positive influence on the diversity of the local megafaunal assemblage. Similarly, faunal composition also exhibited a statistically significant trend with P. carpenteri numerical density. Megafaunal numerical density, particularly that of ascideans, appeared to be enhanced in the core of Rice et al.'s (loc. cit.) peak abundance depth range potentially reflecting the residual effect of sponge spicule mats. Our observations were suggestive of a substantive impact by bottom trawl fishing; however, a definitive assessment of cause and effect was not possible, being hampered by a lack of temporal studies in the intervening period. Other causes and interpretations were plausible and suggested the need for: (i) a precautionary approach to management, (ii) an improved understanding of sponge natural history, and (iii) temporal monitoring (e.g. seafloor sponge habitat cover).

Machine learning is rapidly developing as a tool for gathering data from imagery and may be useful in identifying (classifying) visible specimens in large numbers of seabed photographs. Application of an automated classification workflow requires manually identified specimens to be supplied for training and validating the model. These training and validation datasets are generally generated by partitioning the available manual identified specimens; typical ratios of training to validation dataset sizes are 75:25 or 80:20. However, this approach does not facilitate the desired scalability, which would require models to successfully classify specimens in hundreds of thousands to millions of images after training on a relatively small subset of manually identified specimens. A second problem is related to the ‘class imbalance’, where natural community structure means that fewer specimens of rare morphotypes are available for model training. We investigated the impact of independent variation of the training and validation dataset sizes on the performance of a convolutional neural network classifier on benthic invertebrates visible in a very large set of seabed photographs captured by an autonomous underwater vehicle at the Porcupine Abyssal Plain Sustained Observatory. We tested the impact of increasing training dataset size on specimen classification in a single validation dataset, and then tested the impact of increasing validation set size, evaluating ecological metrics in addition to computer vision metrics. Computer vision metrics (recall, precision, F1-score) indicated that classification improved with increasing training dataset size. In terms of ecological metrics, the number of morphotypes recorded increased, while diversity decreased with increasing training dataset size. Variation and bias in diversity metrics decreased with increasing training dataset size. Multivariate dispersion in apparent community composition was reduced, and bias from expert-derived data declined with increasing training dataset size. In contrast, classification success and resulting ecological metrics did not differ significantly with varying validation dataset sizes. Thus, the selection of an appropriate training dataset size is key to ensuring robust automated classifications of benthic invertebrates in seabed photographs, in terms of ecological results, and validation may be conducted on a comparatively small dataset with confidence that similar results will be obtained in a larger production dataset. In addition, our results suggest that automated classification of less common morphotypes may be feasible, providing that the overall training dataset size is sufficiently large. Thus, tactics for reducing class imbalance in the training dataset may produce improvements in the resulting ecological metrics.