Search

Gynecologic laparoscopy as a type of minimally invasive surgery (MIS) is performed via a live feed of a patient’s abdomen surveying the insertion and handling of various instruments for conducting treatment. Adopting this kind of surgical intervention not only facilitates a great variety of treatments, the possibility of recording said video streams is as well essential for numerous post-surgical activities, such as treatment planning, case documentation and education. Nonetheless, the process of manually analyzing surgical recordings, as it is carried out in current practice, usually proves tediously time-consuming. In order to improve upon this situation, more sophisticated computer vision as well as machine learning approaches are actively developed. Since most of such approaches heavily rely on sample data, which especially in the medical field is only sparsely available, with this work we publish the Gynecologic Laparoscopy ENdometriosis DAtaset (GLENDA) – an image dataset containing region-based annotations of a common medical condition named endometriosis, i.e. the dislocation of uterine-like tissue. The dataset is the first of its kind and it has been created in collaboration with leading medical experts in the field.

Having participated in the three most recent iterations of the annual Video Browser Showdown (VBS2017–VBS2019) as well as in both newly established Lifelog Search Challenges (LSC2018–LSC2019), the actively developed Deep Interactive Video Exploration (diveXplore) system combines a variety of content-based video analysis and processing strategies for interactively exploring large video archives. The system provides a user with browseable self-organizing feature maps, color filtering, semantic concept search utilizing deep neural networks as well as hand-drawn sketch search. The most recent version improves upon its predecessors by unifying deep concepts for facilitating and speeding up search, while significantly refactoring the user interface for increasing the overall system performance.

Laparoscopic skill training and evaluation as well as identifying technical errors in surgical procedures have become important aspects in Surgical Quality Assessment (SQA). Typically performed in a manual, time-consuming and effortful post-surgical process, evaluating technical skills for a large part involves assessing proper instrument handling as the main cause for these type of errors. Therefore, when attempting to improve upon this situation using computer vision approaches, the automatic identification of instruments in laparoscopy videos is the very first step toward a semi-automatic assessment procedure. Within this work we summarize existing methodologies for instrument recognition, while proposing a state-of-the-art instance segmentation approach. As a first experiment in the domain of gynecology, our approach is able to segment instruments well but a much higher precision will be required, since this early step is critical before attempting any kind of skill recognition.

We present the most recent version of our Deep Interactive Video Exploration (diveXplore) system, which has been successfully used for the latest two Video Browser Showdown competitions (VBS2017 and VBS2018) as well as for the first Lifelog Search Challenge (LSC2018). diveXplore is based on a plethora of video content analysis and processing methods, such as simple color, texture, and motion analysis, self-organizing feature maps, and semantic concept extraction with different deep convolutional neural networks. The biggest strength of the system, however, is that it provides a variety of video search and rich interaction features. One of the novelties in the most recent version is a Feature Map Autopilot, which ensures time-efficient inspection of feature maps without gaps and unnecessary visits.

Past editions of the annual Video Browser Showdown (VBS) event have brought forward many tools targeting a diverse amount of techniques for interactive video search, among which sketch-based search showed promising results. Aiming at exploring this direction further, we present a custom approach for tackling the problem of finding similarities in the TRECVID IACC.3 dataset via hand-drawn pictures using color compositions together with contour matching. The proposed methodology is integrated into the established Collaborative Feature Maps (CFM) system, which has first been utilized in the VBS 2017 challenge.