Search

PhD thesis in Information technology This thesis investigates possibilities for applying automatic video analysis in the medical context of resuscitation of a patient. Two situations are investigated: 1) Out-of-hospital cardiac arrest (OHCA) where there is a need for cardiopulmonary resuscitation (CPR) and 2) newborn resuscitation where the newborn is in need of various resuscitation activities, such as stimulation and ventilation support. Both situations suffer from high mortality rates and measurement of resuscitation parameters and activities to evaluate if the performed resuscitation complies with the recommended guidelines, could contribute to ensure provision of quality treatment. Currently there are no clinical solutions utilizing automatic video analysis to improve the quality of the resuscitation in the two situations approached in this thesis. In this work, conventional image processing methods, such as segmentation and frequency analysis approaches have been used to perform measurementof the CPR quality during simulated OHCA situations. The methods for measurement of chest compression rate and CPR summary parameters are implemented in a smartphone app which performs real-time measurements and communicate the information to a webserver that could be monitored by the emergency unit. The system performance is satisfactory with accurate measurements and could add valuable information to the communication between the caller and the emergency unit in OHCA situations. Deep learning and convolutional neural network (CNN) approaches have been used for activity recognition from newborn resuscitation videos. The proposed system, ORAA-net, is a two-step approach consisting of 1) Object detection and Region proposal using a 2D CNN and post-processing, and 2) Activity recognition and generation of Activity timelines using 3D CNNs. The system provides promising results on a dataset of noisy low quality newborn resuscitation videos. By detecting and quantifying the amount of the relevant activities for each episode, a better understanding of the effect of the different resuscitation activities can be achieved, and potentially contribute to optimize patient treatment in newborn resuscitations situations.

Master's thesis in Cybernetics and signal processing I rapporten er mulighetene en har ved å benytte video til deteksjon av hjerte- og respirasjonsrate på nyfødte babyer utforsket. Datamaterialet som inngår i forsøkene og utviklingen av algoritmene er hentet inn ved Universitet i Stavanger (UIS). Testobjektene har vært voksne personer med ulik alder, kjønn, og hudfarge. Det er benyttet to kameratyper til opptakene, et speilreflekskamera i høy prisklasse og et webcamera i lav prisklasse. Ulike lysforhold i rommet hvor opptakene utføres er også testet ut. Løsningene som er utviklet er også testet ut i praksis, og her er datamateriale hentet inn fra nyfødt intensivavdeling ved Stavanger Universitetsykehus (SUS). Hjerteratedeteksjon utføres ved å studere små endringer i fargekomponenter i bilderammene over tid. Ansiktet til testobjektene er utsnittsområdet deteksjonen foretas i. Det er sammenlignet 11 ulike metoder som tar utgangspunkt i fargekomponentene i bilderammene for å generere hjerteratesignalet. I deteksjon av respirasjonsrate er det tatt utgangspunkt i metoden utviklet av Tveit [3], men hvor det er utført noen mindre modifikasjoner. Metoden studerer lokal fase og amplitude i bilderammer ved hjelp av Riesz transform, og genererer den globale bevegelsen i bilderammene ved å finne differansen i disse over tid. Deteksjonene er følsomme for bevegelser, og en er avhengige av at personen ligger i ro for å kunne foreta en god deteksjon. Det er derfor implementert en forstyrrelsesdeteksjon som kutter ut deteksjon av hjerte- og respirasjonsrate dersom bevegelsene i bilderammene er for store. Denne metoden blir også benyttet for å detektere en mulig respirasjonsstopp. Dersom det i den siste perioden har vært for lite bevegelse mellom bilderammene gir detektoren alarm. Resultatene er gode for deteksjon av både hjerte- og respirasjonsrate i testopptakene utført ved UIS. Kameratype spilte ingen vesentlig rolle og deteksjonene kan utføres med et webcamera i lav prisklasse. Lysskilder som forårsaker varierende lysforhold for bilderammene i opptaket gjør en hjerteratedeteksjon mer utfordrende, men fortsatt mulig. Da metodene ble testet ut i praksis på SUS fikk vi varierende resultat for hjerteratedeteksjon, og videre testopptak må her utføres for å kunne lokalisere støykilden som befant seg i to av opptakene. For respirasjonsdeteksjon var resultatene gode også ved SUS. Kjøretiden på deteksjonene tilsier at en total sanntidsdeteksjon av både hjerte- og respirasjonsrate, forstyrrelse og respirasjonsstopp er mulig. Resultatene samlet sett er lovende for at detektoren skal kunne implementeres i en overvåkingsløsning av nyfødte.

Medical emergencies and trauma situations are stressful events. Training and repetition in controlled environment is used for health professionals to gain experience and retain the learning for longer. Laerdal Medical creates medical equipment and training equipment meant for health personnel. SimMan is a high-fidelity patient simulator created by Laerdal Medical to train teams in treating medical emergencies and trauma. To make the simulations more realistic to increase the training effect work has been done to make prototypes that can replicate realistic behaviour. The project in this thesis builds on a head prototype with LCD monitors as eyes and with a joystick and switch controller attached for changing eye modes. One of those modes were used in this thesis for receiving target angles the eyes should be rotated to look at a detected object. The detected object was found using a neural network trained on detecting faces. Edge devices like Raspberry Pi with lower computing capability are cheap and flexible for many use cases. The effectiveness object detection network can achieve on these edge devices makes this eye prototype system flexible for further implementation and more advanced functionality. The modified eye prototype and object detection pipeline developed for this thesis performs well and appear realistic when there is a single target person in the depth camera's field of view. Multiple people visible will make the eyes change who it looks at in a way that does not seem realistic. There are also some angles relative to the prototype where the eye contact looks unfocused. Medical emergencies and trauma situations are stressful events. Training and repetition in controlled environment is used for health professionals to gain experience and retain the learning for longer. Laerdal Medical creates medical equipment and training equipment meant for health personnel. SimMan is a high-fidelity patient simulator created by Laerdal Medical to train teams in treating medical emergencies and trauma. To make the simulations more realistic to increase the training effect work has been done to make prototypes that can replicate realistic behaviour. The project in this thesis builds on a head prototype with LCD monitors as eyes and with a joystick and switch controller attached for changing eye modes. One of those modes were used in this thesis for receiving target angles the eyes should be rotated to look at a detected object. The detected object was found using a neural network trained on detecting faces. Edge devices like Raspberry Pi with lower computing capability are cheap and flexible for many use cases. The effectiveness object detection network can achieve on these edge devices makes this eye prototype system flexible for further implementation and more advanced functionality. The modified eye prototype and object detection pipeline developed for this thesis performs well and appear realistic when there is a single target person in the depth camera's field of view. Multiple people visible will make the eyes change who it looks at in a way that does not seem realistic. There are also some angles relative to the prototype where the eye contact looks unfocused.

The new paradigm shift with the expansion of data requires us to plan better ways to manage it. This data can be in the form of text or digital images and can be found in different domains. This data when managed well, can be of much importance to us. While handling of text data is considered to be an easy process because it has a specific structure. But data in digital images is much larger and requires considerable effort to handle. Handling of such a large data (video format) to smaller amounts (text and numbers in different data structures) while preserving the important information, comes with a lot of benefits such as sorting, easy transmission and searching. In the medical field, the data from Patient monitor screens are available in the form of video recordings. SimCapture uses these recordings for their medical training purposes. It is believed that handling such data can be very helpful to serve soulful purposes in this domain. The manual handling of such data can be a very time-consuming task for which automated solutions are required. For this purpose, a model is proposed that has been developed using different OCR techniques to extract important information from patient monitor screens. This proposed algorithm follows three main steps where at the first step, the input (Stream of videos from SimCapture) is pre-processed and the frames are captured based on a certain threshold for the video feeds. The next step is to apply OCR techniques to identify bounding regions of information. There are different OCRs available with their own computational complexities and limitations on different kinds of data. Lastly, the information that has been extracted is stored into different data-frames for further use. Such solutions can be computationally very exhausting but the project also aims at providing the low-complexity solution for companies having resource limitations by reducing the information size for each video from MegaBytes to Kilobytes of text. In order to achieve the aims set for this project, we are using two different OCRs to test with our data set i.e. tesseract and easyOCR. While tesseract is considered a good solution for other problems, it has presented a very low accuracy when used for SimCapture’s dataset but the algorithm is computationally very cheap. On the other hand, EasyOCR solves the problem with much better accuracy but is computationally expensive. Thus, if the resource is not a limitation, EasyOCR is the better model to extract the information from a patient monitor screen and present it in the form of data frames to be fed to a training model.

Birth asphyxia is a global problem which has resulted in a high mortality rate of newborn babies all over the globe, it is a newborn’s inability to establish breathing at birth. A notable breakthrough is the marrying of medical technology with information technology in an attempt to tackle this global health problem. An example of this is the Safer Births project which is focused on establishing technological advancement to curb newborn deaths. In the year 2013, the Safer Births project started and has till date gathered a lot of data captured during resuscitation sessions. The Haydom data used for the Safer Births project and additional data from Nepal and SUS will be used with the aim of comparing a specialized and generalized model trained on activity recognition system I3D and RGB stream excluding optical flow. With focus on only the newborn region, the reason for this is to simplify the existing model. The experiment was conducted in view of the possibility of achieving a system that can generalize or specialize with a combination of different hospital data on some specific activities of interest namely Ventilation, Suction, Stimulation. A new simplified pipeline, which is a reduction of the previous work done by the saferbirth group, showed a very poor performance when generalized.

Master's thesis in Automation and Signal Processing 99% of deaths of children under 28 days of age takes place in low- and middle-income countries. Most of these deaths are due to complications during childbirth. Newborn resuscitation heavily revolves around ventilation, as the cause of newborn cardiac arrests are predominantly asphyxial. Having properly trained and equipped health care providers is recognized as a key element in lowering newborn mortality. Since 2013 the Safer Births project has been collecting data during resuscitation episodes at Haydom Lutheran hospital in Tanzania. The Safer Births project is a collaborative project between several actors, including the University of Stavanger, Laerdal Global Health and Stavanger University Hospital. The goal of the project is to gain knowledge and develop equipment in order to better train and equip health care providers to save lives at birth. Among the collected data are video recordings of the resuscitation episodes. The objective of the thesis is to locate and classify objects in the video recordings. This is done through object detection networks: Supervised machine learning systems based on deep convolutional neural networks. The object detection will supplement a larger video interpretation project which final aim is to provide timelines for different activities surrounding newborn resuscitation. The resulting information will be used in analysis of resuscitation activities and the responses of the newborn. A baseline for the object detection was available through previous work, where the YOLOv3 object detection network was used. The previous results were encouraging, but some objects were found hard to detect. The largest potential for improvement lay in the detection of a suctioning device, called the suction penguin. From tests conducted on the same set of images, object detection networks RetinaNet and SSD MultiBox collectively outperformed YOLOv3 for all different object classes. The best overall results was achieved through training the networks on original and augmented image frames from the video recordings, in addition to synthetic images. Here, the best attained average precision for the objects were 82.77 for the heart rate sensor, 78.34 for health care provider hands, 69.75 for the bag-mask resuscitator and 44.22 for the suction penguin. Detection rates on the suction penguin are promising considering the networks were not trained on split images, which for the previous work heavily improved the detection of this object.