Search

In settings requiring synthetic data generation based on a clinical cohort, e.g., due to data protection regulations, heterogeneity across individuals might be a nuisance that we need to control or faithfully preserve. The sources of such heterogeneity might be known, e.g., as indicated by sub-groups labels, or might be unknown and thus reflected only in properties of distributions, such as bimodality or skewness. We investigate how such heterogeneity can be preserved and controlled when obtaining synthetic data from variational autoencoders (VAEs), i.e., a generative deep learning technique that utilizes a low-dimensional latent representation. To faithfully reproduce unknown heterogeneity reflected in marginal distributions, we propose to combine VAEs with pre-transformations. For dealing with known heterogeneity due to sub-groups, we complement VAEs with models for group membership, specifically from propensity score regression. The evaluation is performed with a realistic simulation design that features sub-groups and challenging marginal distributions. The proposed approach faithfully recovers the latter, compared to synthetic data approaches that focus purely on marginal distributions. Propensity scores add complementary information, e.g., when visualized in the latent space, and enable sampling of synthetic data with or without sub-group specific characteristics. We also illustrate the proposed approach with real data from an international stroke trial that exhibits considerable distribution differences between study sites, in addition to bimodality. These results indicate that describing heterogeneity by statistical approaches, such as propensity score regression, might be more generally useful for complementing generative deep learning for obtaining synthetic data that faithfully reflects structure from clinical cohorts.

Ocean Networks Canada (ONC) operates large seafloor cabled observatories in the Arctic, Atlantic and Pacific, with some of its long-term observations surpassing 16 years. We present snapshot results from long-term video time-series observations and in-situ experiments studying the benthic boundary layer in two coastal and one continental margin setting of Canada¿s Atlantic and Pacific Oceans. Using video imagery spanning for 7 years (2013-2020) we studied the deep-sea pink urchin Strongylocentrotus fragilis with respect to the expanding oxygen minimum zone in Barkley Canyon (420 m), NE Pacific. In a second case study, we analyzed 6 months of hourly videos from the newly installed Holyrood observatory in Conception Bay, Newfoundland, Atlantic, to investigate benthic-pelagic coupling following the onset of the 2021 spring bloom. From a series of short-term experiments, we combined video and acoustic imagery (dual-frequency identification sonars) and passive acoustics data to better understand poorly understood fish vocalizations, overall temporal changes in benthic abundance and diversity, and behavioural responses to artificial lighting. In a first experiment, in turbid waters of the Fraser River Delta (150 m), Strait of Georgia, the acoustic camera proved to be the most efficient device for measuring faunal densities, while the video was more efficient in detecting a moderately diverse assemblage of fish and invertebrates. Light avoidance behaviour was detected in a large number of species while light attraction was verified for the spotted ratfish Hydrolagus colliei. In the second and third experiments, deployed at 640 m depth adjacent to Barkley Canyon, sequential bait-introduction was employed for the study of benthic successional patterns of deep-sea scavenger communities under limiting dissolved oxygen conditions. Lastly, we present an example of machine learning using a deep learning neural network applied to the automatic detection of commercially harvested s

Ocean observatories collect large volumes of video data, with some data archives now spanning well over a few decades, and bringing the challenges of analytical capacity beyond conventional processing tools. The analysis of such vast and complex datasets can only be achieved with appropriate machine learning and Artificial Intelligence (AI) tools. The implementation of AI monitoring programs for animal tracking and classification becomes necessary in the particular case of deep-sea cabled observatories, as those operated by Ocean Networks Canada (ONC), where Petabytes of data are now collected each and every year since their installation. Here, we present a machine-learning and computer vision automated pipeline to detect and count sablefish (Anoplopoma fimbria), a key commercially exploited species in the N-NE Pacific. We used 651 hours of video footage obtained from three long-term monitoring sites in the NEPTUNE cabled observatory, in Barkley Canyon, on the nearby slope, and at depths ranging from 420 to 985 m. Our proposed AI sablefish detection and classification pipeline was tested and validated for an initial 4.5 month period (Sep 18 2019-Jan 2 2020), and was a first step towards validation for future processing of the now decade-long video archives from Barkley Canyon. For the validation period, we trained a YOLO neural network on 2917 manually annotated frames containing sablefish images to obtain an automatic detector with a 92% Average Precision (AP) on 730 test images, and a 5-fold cross-validation AP of 93% (± 3.7%). We then ran the detector on all video material (i.e., 651 hours from a 4.5 month period), to automatically detect and annotate sablefish. We finally applied a tracking algorithm on detection results, to approximate counts of individual fishes moving on scene and obtain a time series of proxy sablefish abundance. Those proxy abundance estimates are among the first to be made using such a large volume of video data from deep-sea settings. We

Deep-sea ecosystems are reservoirs of biodiversity that are largely unexplored, but their exploration and biodiscovery are becoming a reality thanks to biotechnological advances (e.g., omics technologies) and their integration in an expanding network of marine infrastructures for the exploration of the seas, such as cabled observatories. While still in its infancy, the application of environmental DNA (eDNA) metabarcoding approaches is revolutionizing marine biodiversity monitoring capability. Indeed, the analysis of eDNA in conjunction with the collection of multidisciplinary optoacoustic and environmental data, can provide a more comprehensive monitoring of deep-sea biodiversity. Here, we describe the potential for acquiring eDNA as a core component for the expanding ecological monitoring capabilities through cabled observatories and their docked Internet Operated Vehicles (IOVs), such as crawlers. Furthermore, we provide a critical overview of four areas of development: (i) Integrating eDNA with optoacoustic imaging; (ii) Development of eDNA repositories and cross-linking with other biodiversity databases; (iii) Artificial Intelligence for eDNA analyses and integration with imaging data; and (iv) Benefits of eDNA augmented observatories for the conservation and sustainable management of deep-sea biodiversity. Finally, we discuss the technical limitations and recommendations for future eDNA monitoring of the deep-sea. It is hoped that this review will frame the future direction of an exciting journey of biodiscovery in remote and yet vulnerable areas of our planet, with the overall aim to understand deep-sea biodiversity and hence manage and protect vital marine resources., This research has been funded within the framework of the following project activities: ARIM (Autonomous Robotic Sea-Floor Infrastructure for Benthopelagic Monitoring; MarTERA ERA-Net Cofound); RESBIO (TEC2017-87861-R; Ministerio de Ciencia, Innovación y Universidades); JERICO-S3: (Horizon 2020; Grant Agreement no. 871153); ENDURUNS (Research Grant Agreement H2020-MG-2018-2019-2020 n.824348); Slovenian Research Agency (Research Core Funding Nos. P1-0237 and P1-0255 and project ARRS-RPROJ-JR-J1-3015). We also profited of the funding from the Spanish Government through the “Severo Ochoa Centre of Excellence” accreditation (CEX2019-000928-S) and Italian Ministry of Education (MIUR) under the “Bando premiale FOE 2015” (nota prot. N. 850, dd. 27 ottobre 2017) with the project EarthCruisers “EARTH’s CRUst Imagery for Investigating Seismicity, Volcanism, and Marine Natural Resources in the Sicilian Offshore”. Ocean Networks Canada was funded through Canada Foundation for Innovation-Major Science Initiative (CFI-MSI) fund 30199., Peer Reviewed, Postprint (published version)

Deep-sea ecosystems are reservoirs of biodiversity that are largely unexplored, but their exploration and biodiscovery are becoming a reality thanks to biotechnological advances (e.g., omics technologies) and their integration in an expanding network of marine infrastructures for the exploration of the seas, such as cabled observatories. While still in its infancy, the application of environmental DNA (eDNA) metabarcoding approaches is revolutionizing marine biodiversity monitoring capability. Indeed, the analysis of eDNA in conjunction with the collection of multidisciplinary optoacoustic and environmental data, can provide a more comprehensive monitoring of deep-sea biodiversity. Here, we describe the potential for acquiring eDNA as a core component for the expanding ecological monitoring capabilities through cabled observatories and their docked Internet Operated Vehicles (IOVs), such as crawlers. Furthermore, we provide a critical overview of four areas of development: (i) Integrating eDNA with optoacoustic imaging; (ii) Development of eDNA repositories and cross-linking with other biodiversity databases; (iii) Artificial Intelligence for eDNA analyses and integration with imaging data; and (iv) Benefits of eDNA augmented observatories for the conservation and sustainable management of deep-sea biodiversity. Finally, we discuss the technical limitations and recommendations for future eDNA monitoring of the deep-sea. It is hoped that this review will frame the future direction of an exciting journey of biodiscovery in remote and yet vulnerable areas of our planet, with the overall aim to understand deep-sea biodiversity and hence manage and protect vital marine resources

Synthetic data generation is of great interest in diverse applications, such as for privacy protection. Deep generative models, such as variational autoencoders (VAEs), are a popular approach for creating such synthetic datasets from original data. Despite the success of VAEs, there are limitations when it comes to the bimodal and skewed marginal distributions. These deviate from the unimodal symmetric distributions that are encouraged by the normality assumption typically used for the latent representations in VAEs. While there are extensions that assume other distributions for the latent space, this does not generally increase flexibility for data with many different distributions. Therefore, we propose a novel method, pre-transformation variational autoencoders (PTVAEs), to specifically address bimodal and skewed data, by employing pre-transformations at the level of original variables. Two types of transformations are used to bring the data close to a normal distribution by a separate parameter optimization for each variable in a dataset. We compare the performance of our method with other state-of-the-art methods for synthetic data generation. In addition to the visual comparison, we use a utility measurement for a quantitative evaluation. The results show that the PTVAE approach can outperform others in both bimodal and skewed data generation. Furthermore, the simplicity of the approach makes it usable in combination with other extensions of VAE.

Seafloor multiparametric fibre-optic-cabled video observatories are emerging tools for standardized monitoring programmes, dedicated to the production of real-time fishery-independent stock assessment data. Here, we propose that a network of cabled cameras can be set up and optimized to ensure representative long-term monitoring of target commercial species and their surrounding habitats. We highlight the importance of adding the spatial dimension to fixed-point-cabled monitoring networks, and the need for close integration with Artificial Intelligence pipelines, that are necessary for fast and reliable biological data processing. We then describe two pilot studies, exemplary of using video imagery and environmental monitoring to derive robust data as a foundation for future ecosystem-based fish-stock and biodiversity management. The first example is from the NE Pacific Ocean where the deep-water sablefish (Anoplopoma fimbria) has been monitored since 2010 by the NEPTUNE cabled observatory operated by Ocean Networks Canada. The second example is from the NE Atlantic Ocean where the Norway lobster (Nephrops norvegicus) is being monitored using the SmartBay observatory developed for the European Multidisciplinary Seafloor and water column Observatories. Drawing from these two examples, we provide insights into the technological challenges and future steps required to develop full-scale fishery-independent stock assessments., This work was funded by the following project activities: ARIM (Autonomous Robotic sea-floor Infrastructure for benthopelagic Monitoring; MartTERA ERA-Net Cofound), ARCHES (Autonomous Robotic Networks to Help Modern Societies; German Helmholtz Association), RESBIO (TEC2017-87861-R; Ministerio de Ciencia, Innovación y Universidades, Spanish Government), RESNEP (CTM2017-82991-C2-1-R; Ministerio de Ciencia, Innovación y Universidades, Spanish Government), and SmartLobster (EMSO-LINK Trans National Access-TNA). The EMSO_SmartBay cabled observatory was funded by Science Foundation Ireland (SFI) as part of a SFI Research Infrastructure Award Grant No. 12/RI/2331., Peer Reviewed, Postprint (author's final draft)

Seafloor multiparametric fibre-optic-cabled video observatories are emerging tools for standardized monitoring programmes, dedicated to the production of real-time fishery-independent stock assessment data. Here, we propose that a network of cabled cameras can be set up and optimized to ensure representative long-term monitoring of target commercial species and their surrounding habitats. We highlight the importance of adding the spatial dimension to fixed-point-cabled monitoring networks, and the need for close integration with Artificial Intelligence pipelines, that are necessary for fast and reliable biological data processing. We then describe two pilot studies, exemplary of using video imagery and environmental monitoring to derive robust data as a foundation for future ecosystem-based fish-stock and biodiversity management. The first example is from the NE Pacific Ocean where the deep-water sablefish (Anoplopoma fimbria) has been monitored since 2010 by the NEPTUNE cabled observatory operated by Ocean Networks Canada. The second example is from the NE Atlantic Ocean where the Norway lobster (Nephrops norvegicus) is being monitored using the SmartBay observatory developed for the European Multidisciplinary Seafloor and water column Observatories. Drawing from these two examples, we provide insights into the technological challenges and future steps required to develop full-scale fishery-independent stock assessments

Measuring biodiversity simultaneously in different locations, at different temporal scales, and over wide spatial scales is of strategic importance for the improvement of our understanding of the functioning of marine ecosystems and for the conservation of their biodiversity. Monitoring networks of cabled observatories, along with other docked autonomous systems (e.g., Remotely Operated Vehicles [ROVs], Autonomous Underwater Vehicles [AUVs], and crawlers), are being conceived and established at a spatial scale capable of tracking energy fluxes across benthic and pelagic compartments, as well as across geographic ecotones. At the same time, optoacoustic imaging is sustaining an unprecedented expansion in marine ecological monitoring, enabling the acquisition of new biological and environmental data at an appropriate spatiotemporal scale. At this stage, one of the main problems for an effective application of these technologies is the processing, storage, and treatment of the acquired complex ecological information. Here, we provide a conceptual overview on the technological developments in the multiparametric generation, storage, and automated hierarchic treatment of biological and environmental information required to capture the spatiotemporal complexity of a marine ecosystem. In doing so, we present a pipeline of ecological data acquisition and processing in different steps and prone to automation. We also give an example of population biomass, community richness and biodiversity data computation (as indicators for ecosystem functionality) with an Internet Operated Vehicle (a mobile crawler). Finally, we discuss the software requirements for that automated data processing at the level of cyber-infrastructures with sensor calibration and control, data banking, and ingestion into large data portals

Measuring biodiversity simultaneously in different locations, at different temporal scales, and over wide spatial scales is of strategic importance for the improvement of our understanding of the functioning of marine ecosystems and for the conservation of their biodiversity. Monitoring networks of cabled observatories, along with other docked autonomous systems (e.g., Remotely Operated Vehicles [ROVs], Autonomous Underwater Vehicles [AUVs], and crawlers), are being conceived and established at a spatial scale capable of tracking energy fluxes across benthic and pelagic compartments, as well as across geographic ecotones. At the same time, optoacoustic imaging is sustaining an unprecedented expansion in marine ecological monitoring, enabling the acquisition of new biological and environmental data at an appropriate spatiotemporal scale. At this stage, one of the main problems for an effective application of these technologies is the processing, storage, and treatment of the acquired complex ecological information. Here, we provide a conceptual overview on the technological developments in the multiparametric generation, storage, and automated hierarchic treatment of biological and environmental information required to capture the spatiotemporal complexity of a marine ecosystem. In doing so, we present a pipeline of ecological data acquisition and processing in different steps and prone to automation. We also give an example of population biomass, community richness and biodiversity data computation (as indicators for ecosystem functionality) with an Internet Operated Vehicle (a mobile crawler). Finally, we discuss the software requirements for that automated data processing at the level of cyber-infrastructures with sensor calibration and control, data banking, and ingestion into large data portals., Peer Reviewed, Postprint (published version)