Search

Abstract Detection and diagnosis of colon polyps are key to preventing colorectal cancer. Recent evidence suggests that AI-based computer-aided detection (CADe) and computer-aided diagnosis (CADx) systems can enhance endoscopists' performance and boost colonoscopy effectiveness. However, most available public datasets primarily consist of still images or video clips, often at a down-sampled resolution, and do not accurately represent real-world colonoscopy procedures. We introduce the REAL-Colon (Real-world multi-center Endoscopy Annotated video Library) dataset: a compilation of 2.7 M native video frames from sixty full-resolution, real-world colonoscopy recordings across multiple centers. The dataset contains 350k bounding-box annotations, each created under the supervision of expert gastroenterologists. Comprehensive patient clinical data, colonoscopy acquisition information, and polyp histopathological information are also included in each video. With its unprecedented size, quality, and heterogeneity, the REAL-Colon dataset is a unique resource for researchers and developers aiming to advance AI research in colonoscopy. Its openness and transparency facilitate rigorous and reproducible research, fostering the development and benchmarking of more accurate and reliable colonoscopy-related algorithms and models.

BackgroundEnsuring accurate polyp detection during colonoscopy is essential for preventing colorectal cancer (CRC). Recent advances in deep learning-based computer-aided detection (CADe) systems have shown promise in enhancing endoscopists’ performances. Effective CADe systems must achieve high polyp detection rates from the initial seconds of polyp appearance while maintaining low false positive (FP) detection rates throughout the procedure.MethodWe integrated four open-access datasets into a unified platform containing over 340,000 images from various centers, including 380 annotated polyps, with distinct data splits for comprehensive model development and benchmarking. The REAL-Colon dataset, comprising 60 full-procedure colonoscopy videos from six centers, is used as the fifth dataset of the platform to simulate clinical conditions for model evaluation on unseen center data. Performance assessment includes traditional object detection metrics and new metrics that better meet clinical needs. Specifically, by defining detection events as sequences of consecutive detections, we compute per-polyp recall at early detection stages and average per-patient FPs, enabling the generation of Free-Response Receiver Operating Characteristic (FROC) curves.ResultsUsing YOLOv7, we trained and tested several models across the proposed data splits, showcasing the robustness of our open-access platform for CADe system development and benchmarking. The introduction of new metrics allows for the optimization of CADe operational parameters based on clinically relevant criteria, such as per-patient FPs and early polyp detection. Our findings also reveal that omitting full-procedure videos leads to non-realistic assessments and that detecting small polyp bounding boxes poses the greatest challenge.ConclusionThis study demonstrates how newly available open-access data supports ongoing research progress in environments that closely mimic clinical settings. The introduced metrics and FROC curves illustrate CADe clinical efficacy and can aid in tuning CADe hyperparameters.

In the Industry 4.0 scenario, human–robot collaboration (HRC) plays a key role in factories to reduce costs, increase production, and help aged and/or sick workers maintain their job. The approaches of the ISO 11228 series commonly used for biomechanical risk assessments cannot be applied in Industry 4.0, as they do not involve interactions between workers and HRC technologies. The use of wearable sensor networks and software for biomechanical risk assessments could help us develop a more reliable idea about the effectiveness of collaborative robots (coBots) in reducing the biomechanical load for workers. The aim of the present study was to investigate some biomechanical parameters with the 3D Static Strength Prediction Program (3DSSPP) software v.7.1.3, on workers executing a practical manual material-handling task, by comparing a dual-arm coBot-assisted scenario with a no-coBot scenario. In this study, we calculated the mean and the standard deviation (SD) values from eleven participants for some 3DSSPP parameters. We considered the following parameters: the percentage of maximum voluntary contraction (%MVC), the maximum allowed static exertion time (MaxST), the low-back spine compression forces at the L4/L5 level (L4Ort), and the strength percent capable value (SPC). The advantages of introducing the coBot, according to our statistics, concerned trunk flexion (SPC from 85.8% without coBot to 95.2%; %MVC from 63.5% without coBot to 43.4%; MaxST from 33.9 s without coBot to 86.2 s), left shoulder abdo-adduction (%MVC from 46.1% without coBot to 32.6%; MaxST from 32.7 s without coBot to 65 s), and right shoulder abdo-adduction (%MVC from 43.9% without coBot to 30.0%; MaxST from 37.2 s without coBot to 70.7 s) in Phase 1, and right shoulder humeral rotation (%MVC from 68.4% without coBot to 7.4%; MaxST from 873.0 s without coBot to 125.2 s), right shoulder abdo-adduction (%MVC from 31.0% without coBot to 18.3%; MaxST from 60.3 s without coBot to 183.6 s), and right wrist flexion/extension rotation (%MVC from 50.2% without coBot to 3.0%; MaxST from 58.8 s without coBot to 1200.0 s) in Phase 2. Moreover, Phase 3, which consisted of another manual handling task, would be removed by using a coBot. In summary, using a coBot in this industrial scenario would reduce the biomechanical risk for workers, particularly for the trunk, both shoulders, and the right wrist. Finally, the 3DSSPP software could be an easy, fast, and costless tool for biomechanical risk assessments in an Industry 4.0 scenario where ISO 11228 series cannot be applied; it could be used by occupational medicine physicians and health and safety technicians, and could also help employers to justify a long-term investment.

Abstract Artificial Intelligence (ai) systems are precious support for decision-making, with many applications also in the medical domain. The interaction between mds and ai enjoys a renewed interest following the increased possibilities of deep learning devices. However, we still have limited evidence-based knowledge of the context, design, and psychological mechanisms that craft an optimal human–ai collaboration. In this multicentric study, 21 endoscopists reviewed 504 videos of lesions prospectively acquired from real colonoscopies. They were asked to provide an optical diagnosis with and without the assistance of an ai support system. Endoscopists were influenced by ai ( $$\textsc {or}=3.05$$ O R = 3.05 ), but not erratically: they followed the ai advice more when it was correct ( $$\textsc {or}=3.48$$ O R = 3.48 ) than incorrect ( $$\textsc {or}=1.85$$ O R = 1.85 ). Endoscopists achieved this outcome through a weighted integration of their and the ai opinions, considering the case-by-case estimations of the two reliabilities. This Bayesian-like rational behavior allowed the human–ai hybrid team to outperform both agents taken alone. We discuss the features of the human–ai interaction that determined this favorable outcome.

Abstract Accurate in-vivo optical characterization of colorectal polyps is key to select the optimal treatment regimen during colonoscopy. However, reported accuracies vary widely among endoscopists. We developed a novel intelligent medical device able to seamlessly operate in real-time using conventional white light (WL) endoscopy video stream without virtual chromoendoscopy (blue light, BL). In this work, we evaluated the standalone performance of this computer-aided diagnosis device (CADx) on a prospectively acquired dataset of unaltered colonoscopy videos. An international group of endoscopists performed optical characterization of each polyp acquired in a prospective study, blinded to both histology and CADx result, by means of an online platform enabling careful video assessment. Colorectal polyps were categorized by reviewers, subdivided into 10 experts and 11 non-experts endoscopists, and by the CADx as either “adenoma” or “non-adenoma”. A total of 513 polyps from 165 patients were assessed. CADx accuracy in WL was found comparable to the accuracy of expert endoscopists (CADxWL/Exp; OR 1.211 [0.766–1.915]) using histopathology as the reference standard. Moreover, CADx accuracy in WL was found superior to the accuracy of non-expert endoscopists (CADxWL/NonExp; OR 1.875 [1.191–2.953]), and CADx accuracy in BL was found comparable to it (CADxBL/CADxWL; OR 0.886 [0.612–1.282]). The proposed intelligent device shows the potential to support non-expert endoscopists in systematically reaching the performances of expert endoscopists in optical characterization.

Collaborative Robots-CoBots-are emerging as a promising technological aid for workers. To date, most CoBots merely share their workspace or collaborate without contact, with their human partners. We claim that robots would be much more beneficial if they physically collaborated with the worker, on high payload tasks. To move high payloads, while remaining safe, the robot should use two or more lightweight arms. In this work, we address the following question: to what extent can robots help workers in physical human-robot collaboration tasks? To find an answer, we have gathered an interdisciplinary group, spanning from an industrial end user to cognitive ergonomists, and including biomechanicians and roboticists. We drew inspiration from an industrial process realized repetitively by workers of the SME HANKAMP (Netherlands). Eleven participants replicated the process, without and with the help of a robot. During the task, we monitored the participants' biomechanical activity. After the task, the participants completed a survey with usability and acceptability measures; seven workers of the SME completed the same survey. The results of our research are the following. First, by applying-for the first time in collaborative robotics-Potvin's method, we show that the robot substantially reduces the participants' muscular effort. Second: we design and present an unprecedented method for measuring the robot reliability and reproducibility in collaborative scenarios. Third: by correlating the worker's effort with the power measured by the robot, we show that the two agents act in energetic synergy. Fourth: the participant's increasing level of experience with robots shifts his/her focus from the robot's overall functionality towards finer expectations. Last but not least: workers and participants are willing to work with the robot and think it is useful.

Artificial intelligence (AI) has the potential to assist in endoscopy and improve decision making, particularly in situations where humans may make inconsistent judgments. The performance assessment of the medical devices operating in this context is a complex combination of bench tests, randomized controlled trials, and studies on the interaction between physicians and AI. We review the scientific evidence published about GI Genius, the first AI-powered medical device for colonoscopy to enter the market, and the device that is most widely tested by the scientific community. We provide an overview of its technical architecture, AI training and testing strategies, and regulatory path. In addition, we discuss the strengths and limitations of the current platform and its potential impact on clinical practice. The details of the algorithm architecture and the data that were used to train the AI device have been disclosed to the scientific community in the pursuit of a transparent AI. Overall, the first AI-enabled medical device for real-time video analysis represents a significant advancement in the use of AI for endoscopies and has the potential to improve the accuracy and efficiency of colonoscopy procedures.

The objective of this paper is to present a systematic review of existing sensor-based control methodologies for applications that involve direct interaction between humans and robots, in the form of either physical collaboration or safe coexistence. To this end, we first introduce the basic formulation of the sensor-servo problem, and then, present its most common approaches: vision-based, touch-based, audio-based, and distance-based control. Afterwards, we discuss and formalize the methods that integrate heterogeneous sensors at the control level. The surveyed body of literature is classified according to various factors such as: sensor type, sensor integration method, and application domain. Finally, we discuss open problems, potential applications, and future research directions.

In this article, we present a new scheme that approximates unknown sensorimotor models of robots by using feedback signals only. The formulation of the uncalibrated sensor-based regulation problem is first formulated, then, we develop a computational method that distributes the model estimation problem amongst multiple adaptive units that specialize in a local sensorimotor map. Different from traditional estimation algorithms, the proposed method requires little data to train and constrain it (the number of required data points can be analytically determined) and has rigorous stability properties (the conditions to satisfy Lyapunov stability are derived). Numerical simulations and experimental results are presented to validate the proposed method.

Mapping brain functions is crucial for neurosurgical planning in patients with drug-resistant seizures. However, presurgical language mapping using either functional or structural networks can be challenging, especially in children. In fact, most of the evidence on this topic derives from cross-sectional or retrospective studies in adults submitted to anterior temporal lobectomy. In this prospective study, we used fMRI and DTI to explore patterns of language representation, their predictors and impact on cognitive performances in 29 children and young adults (mean age at surgery: 14.6 ± 4.5 years) with focal lesional epilepsy. In 20 of them, we also assessed the influence of epilepsy surgery on language lateralization. All patients were consecutively enrolled at a single epilepsy surgery center between 2009 and 2015 and assessed with preoperative structural and functional 3T brain MRI during three language tasks: Word Generation (WG), Rhyme Generation (RG) and a comprehension task. We also acquired DTI data on arcuate fasciculus in 24 patients. We first assessed patterns of language representation (relationship of activations with the epileptogenic lesion and Laterality Index (LI)) and then hypothesized a causal model to test whether selected clinical variables would influence the patterns of language representation and the ensuing impact of the latter on cognitive performances. Twenty out of 29 patients also underwent postoperative language fMRI. We analyzed possible changes of fMRI and DTI LIs and their clinical predictors. Preoperatively, we found atypical language lateralization in four patients during WG task, in one patient during RG task and in seven patients during the comprehension task. Diffuse interictal EEG abnormalities predicted a more atypical language representation on fMRI (p = 0.012), which in turn correlated with lower attention (p = 0.036) and IQ/GDQ scores (p = 0.014). Postoperative language reorganization implied shifting towards atypical language representation. Abnormal postoperative EEG (p = 0.003) and surgical failures (p = 0.015) were associated with more atypical language lateralization, in turn correlating with worsened fluency. Neither preoperative asymmetry nor postoperative DTI LI changes in the arcuate fasciculus were observed. Focal lesional epilepsy associated with diffuse EEG abnormalities may favor atypical language lateralization and worse cognitive performances, which are potentially reversible after successful surgery.

Intuitive user interfaces are indispensable to interact with the human centric smart environments. In this paper, we propose a unified framework that recognizes both static and dynamic gestures, using simple RGB vision (without depth sensing). This feature makes it suitable for inexpensive human-robot interaction in social or industrial settings. We employ a pose-driven spatial attention strategy, which guides our proposed Static and Dynamic gestures Network—StaDNet. From the image of the human upper body, we estimate his/her depth, along with the region-of-interest around his/her hands. The Convolutional Neural Network (CNN) in StaDNet is fine-tuned on a background-substituted hand gestures dataset. It is utilized to detect 10 static gestures for each hand as well as to obtain the hand image-embeddings. These are subsequently fused with the augmented pose vector and then passed to the stacked Long Short-Term Memory blocks. Thus, human-centred frame-wise information from the augmented pose vector and from the left/right hands image-embeddings are aggregated in time to predict the dynamic gestures of the performing person. In a number of experiments, we show that the proposed approach surpasses the state-of-the-art results on the large-scale Chalearn 2016 dataset. Moreover, we transfer the knowledge learned through the proposed methodology to the Praxis gestures dataset, and the obtained results also outscore the state-of-the-art on this dataset.

Due to the epochal changes introduced by “Industry 4.0”, it is getting harder to apply the varying approaches for biomechanical risk assessment of manual handling tasks used to prevent work-related musculoskeletal disorders (WMDs) considered within the International Standards for ergonomics. In fact, the innovative human–robot collaboration (HRC) systems are widening the number of work motor tasks that cannot be assessed. On the other hand, new sensor-based tools for biomechanical risk assessment could be used for both quantitative “direct instrumental evaluations” and “rating of standard methods”, allowing certain improvements over traditional methods. In this light, this Letter aims at detecting the need for revising the standards for human ergonomics and biomechanical risk assessment by analyzing the WMDs prevalence and incidence; additionally, the strengths and weaknesses of traditional methods listed within the International Standards for manual handling activities and the next challenges needed for their revision are considered. As a representative example, the discussion is referred to the lifting of heavy loads where the revision should include the use of sensor-based tools for biomechanical risk assessment during lifting performed with the use of exoskeletons, by more than one person (team lifting) and when the traditional methods cannot be applied. The wearability of sensing and feedback sensors in addition to human augmentation technologies allows for increasing workers’ awareness about possible risks and enhance the effectiveness and safety during the execution of in many manual handling activities.

Significant corpus callosum (CC) involvement has been found in relapsing–remitting multiple sclerosis (RRMS), even if conventional magnetic resonance imaging measures have shown poor correlation with clinical disability measures. In this work, we tested the potential of multimodal imaging of the entire CC to explain physical and cognitive disability in 47 patients with RRMS. Values of thickness, fractional anisotropy (FA) and mean diffusivity (MD) were extracted from 50 regions of interest (ROIs) sampled along the bundle. The relationships between clinical, neuropsychological and imaging variables were assessed by using Spearman's correlation. Multiple linear regression analysis was employed in order to identify the relative importance of imaging metrics in modeling different clinical variables. Regional fiber composition of the CC differentially explained the response variables (Expanded Disability Status Scale [EDSS], cognitive impairment). Increases in EDSS were explained by reductions in CC thickness and MD. Cognitive impairment was mainly explained by FA reductions in the genu and splenium. Regional CC imaging properties differentially explained disability within RRMS patients revealing strong, distinct patterns of correlation with clinical and cognitive status of patients affected by this specific clinical phenotype.

The aim of this study was to investigate intra-regional activation and inter-regional connectivity during passive movement. During fMRI, a mechanic device was used to move the subject's index and middle fingers. We assessed four movement conditions (unimanual left/right, bimanual symmetric/asymmetric), plus Rest. A conventional intra-regional analysis identified the passive stimulation network, including motor cortex, primary and secondary somatosensory cortex, plus the cerebellum. The posterior (sensory) part of the sensory-motor activation around the central sulcus showed a significant modulation according to the symmetry of the bimanual movement, with greater activation for asymmetric compared to symmetric movements. A second set of fMRI analyses assessed condition-dependent changes of coupling between sensory-motor regions around the superior central sulcus and the rest of the brain. These analyses showed a high inter-regional covariation within the entire network activated by passive movement. However, the specific experimental conditions modulated these patterns of connectivity. Highest coupling was observed during the Rest condition, and the coupling between homologous sensory-motor regions around the left and right central sulcus was higher in bimanual than unimanual conditions. These findings demonstrate that passive movement can affect the connectivity within the sensory-motor network. We conclude that implicit detection of asymmetry during bimanual movement relies on associative somatosensory region in post-central areas, and that passive stimulation reduces the functional connectivity within the passive movement network. Our findings open the possibility to combine passive movement and inter-regional connectivity as a tool to investigate the functionality of the sensory-motor system in patients with very poor mobility.