Search

In this paper we present our initial findings from investigating the dampening of acoustic signals through dense fish layers, referred to as shadowing. These findings are based on measurements which were performed in the Mediterranean Sea and off the Norwegian coast using gilthead seabream and Atlantic salmon respectively. The paper also presents the experimental setup for data acquisition. The results from the initial data analysis show that the shadowing effect is measurable and indicate that it can be compensated for. Future work will include the creation of a compensation strategy for this effect.

The CageReporter project adapts the use of autonomous and tetherless underwater vehicles as a carrier of sensor systems for data acquisition, where the data are transferred from sea-based fish cages to a centralized land base (Figure 1). The vehicle will use active motion con-trol and acquire data from the cage environment while exploring the fish cages. The main project objective is to develop technology for autonomous functionality for adaptive mission planning to achieve high quality data acquisition from the cage space. One of the most im-portant capabilities within this context is to operate in a dynamically changing environment in interaction with the biomass (bio-interactive) and the aquaculture structures. The project addresses many challenges within the aquaculture industry related to poor accuracy and representative sampling of important variables from the whole volume of the cage. A suc-cessful project outcome will lead to new technology for collection of high-resolution data that could be utilized for assessment of the fish farm state, grouped within three main areas: A) fish, B) aquaculture structures and C) production environment. Examples of areas of applica-tions are detection of abnormal fish behaviour, net inspection and mapping of water quality. CageReporter will provide a solution for continuous 24/7 inspection of the current situation and will be the mobile eyes of the fish farmer in the cage environment. The project idea is based on using low-cost technology for underwater communication, vehicle positioning, and camera systems for 3D vision.

This report presents results obtained in the CageReporter project regarding the development of a 3D vision system to be used for data capture in fish cages. The developed system enables to obtain high-quality data with the overall goal to identify fish conditions and perform cage inspections during daily operations, as well as the robotic vision for an underwater vehicle during the adaptive operation planning in the cage. A compact and robust sensor with optical components and lighting system was developed. In addition, this activity presents development of methods to evaluate the quality of the captured data. Based on defined quality criteria associated with fish conditions and cage inspection operations, algorithms have been developed to evaluate whether the quality criteria are met. The algorithms have been validated using image data obtained from 24/7 video streams from a full-scale fish cage. The work furthermore includes the development of image processing algorithms to estimate the distance and orientation relative to the inspected object of interest, such as the fish or the net. The developed algorithms have been validated based on vision data obtained during tests both in lab- and full scale.

Automatic recognition of objects in images is a difficult and challenging task in computer vision which has been tackled in many different ways. Based on the powerful and widely used concept to represent objects and scenes as relational structures, the problem of graph matching, i.e. to find correspondences between two graphs is a part of the object recognition problem. Belonging to the field of combinatorial optimization graph matching is considered to be one of the most complex problems in computer vision: It is known to be NP-complete in the general case. In this thesis, two novel approaches to the graph matching problem are proposed and investigated. They are based on recent progress in the mathematical literature on convex programming. Starting out from describing the desired matchings by suitable objective functions in terms of binary variables, relaxations of combinatorial constraints and an adequate adaption of the objective function lead to continuous convex optimization problems which can be solved without parameter tuning and in polynomial time. A subsequent post-processing step results in feasible, sub-optimal combinatorial solutions to the original decision problem. In the first part of this thesis, the connection between specific graph-matching problems and the quadratic assignment problem is explored. In this case, the convex relaxation leads to a convex quadratic program , which is combined with a linear program for post-processing. Conditions under which the quadratic assignment representation is adequate from the computer vision point of view are investigated, along with attempts to relax these conditions by modifying the approach accordingly. The second part of this work focuses directly on the matching of subgraphs -- representing a model -- to a considerably larger scene graph. A bipartite matching is extended with a quadratic regularization term to take into account relations within each set of vertices. Based on this convex relaxation, post-processing and the application to computer vision are investigated and discussed. Numerical experiments reveal both the power and the limitations of the approach. For problems of sizes which occur in applications the approach is quite reasonable and often the combinatorial optimal solution is found. For larger instances the intrinsic combinatorial nature of the problem comes out and leads to sub-optimal solutions which, however, are still good.

I havbruksnæringen har det blitt mer populært å benytte seg av datasyn som et resultat av et stadig økende fokus på mindre ingripende observasjonsmetoder. Oppdrettere kan i dag benytte seg av høykvalitets kameraer for å automatisk analysere laksens adferd, uten behov for menneskelig innblanding. Denne oppgaven demostrerer hvordan kontinuerlig bevegelsesbaner og hastighet til laks kan estimeres ved å benytte stereokameraer sammen med konvensjonelle metoder for stereomatching. Dette muliggjør 3D posisjonsmålinger av matchende punkter, basert på forskyvning forskjellen mellom to stereobilder med pikselnøyaktighet. Tidligere arbeid innen relaterte datasynsfelt ga inspirasjon til å forbedre nøyaktigheten ytterligere ved bruk av stereomatching metoder som opererer på et sub-piksel-nivå. Seks forskjellige metoder har blitt undersøkt i dette arbeidet. Gjennom det presenterte arbeidet har det blitt vist at sub-piksel nøyaktighet var nødvendig for å observere ørsmå bevegelser av f.eks. laksens hodebevegelser. Det presenterte arbeidet med bevegelsesbanene ble utvidet til å estimere posituren og størrelsen på laksen, basert på en tett 3D-rekonstruksjon av lakseoverflaten med sub-piksel nøyaktighet. Det ble kvalitativt vist at posituren kan estimeres ved å tilpasse en ellipsoide til overflatepunktskyrepresentasjonen av laksen. Denne ellipsoiden kan også utnyttes til å estimere lengden og størrelsen på laksen, og viser at den foreslåtte metoden har potensial til å gi mer konsistente og pålitelige målinger sammenlignet med målinger som er utført for hånd. Gjennom dette arbeidet har alle metoder blitt evaluert basert på nøyaktighet og robusthet. Det har også blitt bevist og dokumentert at stereomatching med sub-piksel nøyaktighet er en potensiell metode for å innhente nyttig informasjon fra undervannsstereobilder og -videoer. In the aquaculture industry, the use of computer vision has gained popularity with the emerging focus on more non-intrusive procedures. Nowadays, farmers can use cameras that provide good image-quality to automatically analyse fish behaviour without the need for any human intervention. This thesis demonstrates how continuous motion trajectories and speed of individual salmon can be estimated by using stereo cameras along with conventional matching techniques. This allows for 3D position measurements of matching features, based on the disparity shift observed between the two stereo views with pixel accuracy. Inspired by previous work in related fields of computer vision, sub pixel refinement was applied to further enhance the matching accuracy. Six state of the art methods were selected for investigation in this work. The proposed methods were selected to show a variety of approaches exploiting both the spatial and frequency domain. It was also shown that tiny movements that were not previously apparent, such as the head wiggling of the fish, can be observed when using sub pixel refinement. The presented work on the motion trajectories was extended to estimate the pose and size of the fish based on a dense 3D reconstruction of the salmon surface with sub-pixel accuracy. It was qualitatively shown that the pose can be estimated by fitting an ellipsoid to the point cloud representation of the fish. This ellipsoid can also be exploited to estimate the length and size of the fish, showing that the proposed method has the potential to provide more consistent and reliable measurements compared to manual performed measurements. Throughout this thesis, all proposed methods have been evaluated in terms of accuracy and robustness. It was then demonstrated that sub pixel refinement, combined with stereo matching was a viable method for extracting useful information from underwater stereo images and videos.

Sanntidsestimering av posisjon er en teknologi vi i dag er godt vant med. Systemer som GPS har gitt oss muligheter til å kunne finne posisjonen vår raskt og enkelt. Men det er ikke alle steder vi kan være avhengige av slike systemer. Et slikt sted er under vann. Og når det da er et ønske om å kunne finne posisjonen til undervanns fartøyer i fiskemerder, oppstår det et problem. Hvordan finne posisjoner uten bruk av GPS? Først er det laget et dataset for å kunne lage of presentere resultater og for å få høy kvalitet på dataen. Datasettet er syntetisk, som betyr at det er laget digitalt. Ved å lage ett syntetisk datasett får vi også med den faktiske posisjonen som kameraet bevegde seg. Deretter er en metode for å posisjonsestimering presentert. Ved bruk av Fourier Analyse, en teknikk som bruker frekvensene i bildet, kan bevegelsen til kameraet estimeres. Til sammenligning bruker tilsvarende systemer på land egenskaper i bildene brukes for å oppnå posisjonsestimater. Disse egenskapene er gjerne kanter, hjørner og andre steder med stor kontrast. Bruk av Fourier Analys er ikke uhørt i sammenheng med posisjonsestimering, det som er nytt her er bruken i en merde konstruksjon. Resultatene fra testene med denne metoden viser at bruken av denne typen metode kan ha noe for seg. Det er fortsatt en vei å gå med utviklingen for at systemet skal være robust og pålitelig, men resultatene viser at det er en mulighet å fortsette med. By now we are used to realtime position estimating, with systems such as GPS. These types of systems have given quick and easy access to our position. However, we can not rely on these systems everywhere. Under water for instance. There arises a challenge with the desire for positioning of underwater vehicles in aquacultural fish cages. How do we find positions without the use of GPS? To start it is produced a dataset. This is done in order to present results from the position estimates, and control the quality of the data. The dataset is a so called synthetic dataset, which means it is produced digitally. By producing a synthetic dataset we obtain the actual positions of the camera. Then a method for position estimating is presented. Using Fourier Analysis, a technique that uses the frequencies in an image, the motion of the camera can be estimated. In comparison, equivalent systems on land uses features in the image to obtain a position estimate. In this context features are corners, edges and other contrasting regions in the image. The use of Fourier Analysis is not unheard of in regards to position estimation, new here is the use in an aquacultural fish cage. The results with this method shows some promise. There are still steps to be made in the development of this method in order for it to be robust and reliable, but the results show possibilities for further work.

I 2019 alene ble nesten 300.000 atlanterhavslaks rapportert rømt fra norske oppdrettsanlegg. Dette antallet tilsvarer over halve den gjenværende villaksbestanden. En vanlig fluktrute går gjennom hull i nota, og regelmessig kontroll av nettintegriteten på et oppdrettsanlegg er derfor et nødvendig preventivt tiltak. Et komplett algoritmisk rammeverk har blitt initiert for å automatisere søket etter hull i nettmaskene i en videosekvens filmet av undervannsrobot under notvask. Rammeverket presenterer en ny strategi for å segmentere nettmasker ved hjelp av en U-Net-variant kalt MultiRes U-Net. En liten endring i U-Nettet foreslås for å fremme koherensen til segmenteringene i både tid og rom uten å øke behovet for regnekraft stort. Å bruke kunstig intelligens, i motsetning til tradisjonelle metoder som kantdeteksjon eller histogramanalyse, mulliggjør en sømløs segmenteringsporosess hvor fisk, utstyr, grumsete vann og eventuelle fremmedlegemer ignoreres. For å oppdage uregelmessigheter i nota foreslås en adaptiv prosedyre basert på morfologiske operasjoner og segmentsvulming, som analyserer videosekvensen bilde-for-bilde. Et filter ser de oppdagede uregelmessighetene i sammenheng med tidligere oppdagelser og verifiserer de som oppstår på omtrentlig samme sted med omtrentlig samme utstrekning. Et dypt konvulerende nevralt nettverk basert på VGG16-modellen har videre blitt spesialisert på å se forskjell på nettstruktur, fisk, og vrøvl. Verifiserte uregelmessigheter blir inspisert av dette nettverket, og siden hull utelukkende kan oppstå innad i nettstruktur, avvises uregelmessigheten dersom den klassifiseres som fisk eller vrøvl. Lovende resultater ble oppnådd, og de fleste hullene i ti krevende 10-sekunders testvideoer ble identifisert. Spesielt overbevisende var prestasjonene til de dype nevrale nettverkene brukt til segmentering og klassifisering. Bilde-for-bildeprosedyren for uregelmessighetsdeteksjon og den påfølgende filtreringen behøver videre arbeid for å gjøre rammeverket ytterligere robust og effektivt. In 2019 alone, close to 300,000 Atlantic Salmon reportedly fled Norwegian aquaculture sites, which is more than half the number of the remaining wild stock. A common escape route is through net holes, and frequent inspection of fish cage integrity is therefore a necessary preventative measure. A complete algorithmic framework has been initiated to fully automatise the search for net holes in a video sequence captured by a remotely controlled vehicle carrying out a net cleaning operation. The framework presents a new strategy for net thread segmentation utilising a U-Net variant called MultiRes U-Net. A computationally efficient alteration to the U-Net’s input layer is proposed to encourage its spatiotemporal coherency. The introduction of artificial intelligence for segmentation, contrary to traditional edge detection or histogram thresholding, allows seamless discrimination of intelligible net structure from noisy surroundings such as turbulent water, fish, and equipment. An adaptive scheme based on morphological operations and region growing is suggested as a frame-by-frame irregularity detector, and a spatiotemporal filter to verify irregularities that occur in the same area over a certain period of time. A deep convolutional neural network based on the VGG16 model has been specialised on separating net structure from fish and nonsense to classify verified irregularities – a final barricade to prevent objects falsely included in the segmentation from being reported as holes. Promising results were achieved, and most holes present in a set of ten 10- second test videos shot in challenging real-world scenes were correctly identified. Particularly satisfactory were the performances of the deep learning approaches to scene segmentation and irregularity classification, whilst the local irregularity detector and the spatiotemporal filter require further work to improve the robustness and the efficiency of the framework.

Bevaring av fiskevelferd er et omfattende tema i oppdrettsnæringen. En industriell laksemerd kan inneholde så mange som 200 000 fisk, og det er utfordrende å overvåke velferden til denne mengden enkeltdyr. Manuelle observasjoner og målinger av atferd, miljøfaktorer og fiskehelse har vært dominerende. Oppdrettslaksproduksjonen utvides imidlertid fortsatt, og det er behov for grundigere og mer effektive metoder. Et problem er ineffektiviteten ved manuell operasjon, et annet er mangelen på en fullstendig romlig og tidsmessig oversikt over merden. De fleste laksemerder i dag har undervannskameraer installert brukt til manuelle observasjoner, og flere indikatorer for fiskevelferd kan overvåkes visuelt. Dette kan utnyttes av datasynteknologi. Undervannsbilder er underlagt komplekse utfordringer, men utviklingen innen datasyn og dyplæringsteknologi viser muligheter til å bruke slike metoder for praktiske formål, for bedre og mer effektiv overvåkning av fiskevelferd. Denne oppgaven foreslår et proof-of-concept for estimering av svømmehastighet for oppdrettslaks på et datasett med stereovideo fra industrielle laksemerder. Dette ved å kombinere objektdeteksjon ved dyp læring, IOU-basert objektsporing og dybdeestimering av bildescener ved semi-global blokkmatching. En rekke laksevelferdsindikatorer ble evaluert og klassifisert ut fra deres visuelle og tidsmessige detekterbarhet. Svømmeaktivitet, eller nærmere bestemt svømmehastighet, ble ansett som den mest gjennomførbare indikatoren å studere, gitt datasettet. Spredning, lav kontrast, fargeforvrengninger og lysrefleksjoner representerer ytterligere utfordringer for synsbaserte teknikker under vann, sammenlignet med terrestriske applikasjoner. Svømme-hastigheten ble beregnet ved å spore hver fisk ved å måle posisjonen til et fast punkt på fiskekroppen. Midtpunktet for deteksjonsboksene fungerte som dette faste punktet. Fiskeanatomi ble studert for å bestemme hvilket objekt som skulle spores, sammen med hensyn til begrensningene i selve datasettet, dybdeestimering og gjenstandsdeteksjon. Kaudalfinnen så ut til å være det beste valget. For objektgjenkjenning ble modellen YOLOv4 valgt på grunn av parallelle databehandlingsmuligheter på GPU og overlegen ytelse når det gjelder hastighet og nøyaktighet. Bilder ble hentet fra videoene og annotert. Hvert treningsbilde ble augmentert fire ganger, noe som resulterte i et treningssett med fem versjoner av hvert bilde. De beste nettverksvektene fra treningen ble lagret og brukt til den endelige modellen. For å oppnå 3D-posisjoner for laksen ble stereomatching ved semi-global blokkmatching implementert. Fra de resulterende dybdeestimatene kunne 3D-posisjonen til kaudalfinner estimeres ved å bruke det gjennomsnittlige dybdeestimatet til et forhåndsdefinert indre midtområde av deteksjonsboksen og midtpunktet for deteksjonsboksen. Deteksjoner med et gjennomsnittlig dybdeestimat i dette indre området over en viss terskel, som indikerer en ikke-jevn dybdeestimering av finneflaten, ble forkastet. Den foreslåtte objektsporingsmetoden var basert på overlapp mellom deteksjonsbokser, noe som betyr at ingen bilde- eller bevegelsesinformasjon var nødvendig. Algoritmen ble modifisert til å tillate "hull" mellom deteksjoner i påfølgende bilder. De enkelte komponentene ble kombinert for å danne en endelig eksperimentell pipeline for å estimere laksens svømmehastighet. Finneposisjoner i 3D ble sporet i to videoeksperimenter. For hvert 3D-spor ble gjennomsnittshastigheten beregnet ved hjelp av avstanden mellom påfølgende punkter, antall bilder mellom hvert bestemte punkt og bildefrekvensen til videoene. Videre ble den gjennomsnittlige lakseshastigheten og standardavviket beregnet for hvert eksperiment. Deteksjonsmodellen ble testet på hastighet og ytelse. Modellen hadde en ytelse på 87.49\% mAP@0.50 etter trening på 208 originale bilder. For 3D-rekonstruksjonsdelen var den totale reprojeksjonsfeilen for kamerakalibreringen 0,2234 piksler. Videre viste dybdeestimeringene forbedring etter påføring av et vektet minste kvadrat-filter. Hull i dybdeestimeringene ble fjernet, og overflatene ble jevnere. Målinger av kjente lengder i rekonstruerte kalibreringsbilder samsvarte med de sanne størrelsene. Sporingsalgoritmen ble testet på YOLOv4-deteksjonene. Ved å bruke en moderat grense for IOU-overlapp oppnådde vi pålitelige spor. Maksimal og gjennomsnittlig sporlengde økte når det ble tillatt hull, og antall spor ble redusert. Den endelige sporingsalgoritmen var gjenstand for noen problemer forårsaket av deteksjonsresultatene, for eksempel spor konstruert av falske positive deteksjoner og splittede spor forårsaket av falske negative deteksjoner. Visualiseringer og 3D-plott av spor fra eksperimentene med den endelige pipelinen, sammen med målinger av gjennomsnittshastighet og standardavvik over alle spor i hver video, ble analysert. Gjennomsnittlig laksesvømmehastighet ble målt til henholdsvis 0,5479 ms-1 og 0,6561 ms-1, og standardavvikene til 0,3175 ms-1 og 0,3149 ms-1. Uskarphet var den viktigste bidragsyteren til feil i den endelige pipelinen for estimering av svømmehastighet. Objektdeteksjonsmodellen detekterte kaudalfinner i sanntid. Modifiseringen av sporingsalgoritmen induserte en forbedring i sporlengden og reduserte antall spor, noe som indikerer at sporene som ble splittet på grunn av hull i den opprinnelige algoritmen, ble slått sammen i den modifiserte algoritmen. For sporingsalgoritmen var kameraets egenbevegelse, som påvirker IOU-overlapp, som det største problemet. Et neste steg kan være å inkludere en bevegelsesmodell for egenbevegelse eller bruke en algoritme som tar hensyn til bildeinformasjon. Samlet sett oppnådde vi en eksperimentell pipeline som var i stand til å estimere laksens svømmehastighet i de to eksperimentene, ved å fjerne utilstrekkelige deteksjoner og dybdeestimeringer fra sporene. 3D-plott av spor samsvarte med den observerte bevegelsen til fisken i forhold til kameraet. Gjennomsnittlige hastigheter beregnet fra sporene var rimelige for svømmehastigheten til oppdrettslaks. Som et proof-of-concept viser systemet at det er stort potensial i å anvende og kombinere velkjente visjonsbaserte teknikker for å overvåke fiskevelferd i industrielle laksemerder, og at bildekvalitet er den viktigste begrensningen per i dag. Preserving fish welfare is an extensive issue in the fish farming industry. An industrial salmon sea cage can contain as many as 200 000 fish, and monitoring the welfare of this amount of individual animals is challenging. Manual observations and measurements of behavior, environmental factors, and fish health have been predominant. However, farmed salmon production is still expanding, and there is a need for more thorough and efficient methods. One problem is the inefficiency of manual operations, another is the lack of a complete spatial and temporal overview of the sea cage. Most sea cages today have underwater cameras installed used for manual observations, and several indicators of fish welfare can be monitored visually. This can be exploited by computer vision technology. Underwater images are subject to complex challenges, but the evolution in computer-vision and deep-learning technology shows promise for applying such methods for practical purposes to better and more efficiently monitor fish welfare. This thesis proposes a proof-of-concept for estimating farmed salmon swimming velocity on a dataset of stereo video from industrial sea cages by combining deep-learning object detection, IOU-based object tracking, and image scene depth estimation by semi-global block matching. A number of salmon welfare indicators were evaluated and classified based on their visual and temporal detectability. Swimming activity, or more specifically swimming velocity, was considered as the most feasible indicator to study, given the dataset. Scattering, low contrast, color distortions, and light reflections represent additional challenges for vision-based techniques underwater, compared to terrestrial applications. Swimming velocity was to be estimated by tracking each fish by measuring the position of a fixed point on the fish body. The middle point of object detection bounding boxes served as this fixed point. Fish anatomy was studied to decide which object to track, together with considerations of the limitations of the dataset itself, depth estimation, and object detection. The caudal fin appeared to be the best choice. For object detection, the YOLOv4 model was chosen due to its GPU parallel computing capabilities and its superior performance in terms of speed and accuracy. Image frames were extracted from the videos and annotated. Each training image was augmented four times, resulting in a training set with five versions of each image. The best-performing network weights through the training were saved and used for the final model. To obtain 3D positions for the salmon, stereo matching by semi-global block matching was implemented. From the resulting disparity maps, the 3D position of caudal fins could be estimated by using the average disparity value of a predefined inner middle area of the bounding box and the center point of the bounding box. Detections with an average disparity value in this inner area above a certain threshold, indicating a non-smooth depth estimation of the fin surface, were discarded. The proposed object tracking method was based on overlap between detection boxes, meaning no image or motion information was required. The algorithm was modified to allow "gaps" between detections in consecutive frames. The individual components were combined to form a final experimental pipeline to estimate salmon swimming velocity. The 3D caudal fin positions were tracked in two video experiments. For each 3D track, the mean velocity was calculated using the distance between consecutive points, the number of frames between each certain point, and the frame rate of the videos. Further, the mean salmon swimming velocity and standard deviation were calculated for each experiment. The detection model was tested on speed and performance. The model performed at 87.49\% mAP@0.50 after training on 208 original images. For the stereo 3D reconstruction part, the total reprojection error of the camera calibration was 0.2234 pixels. Further, the disparity maps showed improvement after applying a weighted least squares filter. Speckles were removed, and surfaces appeared smoother. Measurements of known sizes in reconstructed calibration images corresponded with the ground truth sizes. The tracking algorithm was tested on the YOLOv4 detections. By using a moderate IOU overlap threshold, we achieved reliable tracks. The maximum and average track length increased when allowing gaps and the number of total tracks was reduced. The final tracking algorithm was subject to some issues caused by the detection results, e.g., tracks constructed of false positives and split tracks caused by false negatives. Visualizations and 3D plots of tracks from the final pipeline experiments, together with measurements of mean velocity and standard deviation over all tracks in each video, were analyzed. The mean salmon swimming velocities were measured to 0.5479 ms-1 and 0.6561 ms-1, and the standard deviations to 0.3175 ms-1 and 0.3149 ms-1, respectively. Motion blur was the main contributor to errors in the final pipeline of swimming velocity estimation. The object detection model performed sufficiently on detecting caudal fins in real-time. The modification of the tracking algorithm to allow gaps induced an improvement in the length of tracks and decreased the number of tracks, indicating that tracks split due to gaps in the original algorithm were merged in the modified algorithm. For the tracking algorithm, camera egomotion affecting IOU overlap appeared as the largest issue. A next step might be to include a motion model for the egomotion or use an algorithm that considers image information. Overall, by removing insufficient detections and disparity estimations from the tracks, we obtained an experimental pipeline that was able to estimate salmon swimming velocities in the two experiments conducted. 3D plots of tracks corresponded with the observed motion of the fish relative to the camera. The mean velocities computed from the tracks were reasonable for the swimming velocity of farmed salmon. As a proof-of-concept, the system shows that there is large potential in applying and combining well-known vision-based techniques for monitoring fish welfare in industrial sea cages and that image quality is the main limitation as of today.

This thesis is a paper collection that focuses on unconventional methods of biometric recognition. Four new approaches are presented and discussed. The first two introduce and explore the concepts behind transient biometrics. Transient biometrics relaxes the hard permanence requirement that is common to biometric identifiers, creating a biometric signature with expiration date which increases acceptability. The third approach investigates a novel method for extracting a capable biometric identifier using Electroencephalography (EEG) and a visual stimulus. The final approach studies the use of synthetic biometric data for training a machine learning approach in the recognition of non-collaborative subjects under the context or person re-identification. Four new datasets have been created for the purposes of this thesis and have been made publicly available. Contributions are on the interface between computer vision, biometrics and machine learning. Ethical implications of this work are discussed, concluding that it is preferable to perform such work in the public domain.

In recent years, computer vision has been used in increasing amounts in aqua culture and will be essential for the development of automated solutions for fish farming. In this thesis we study the possibility of using unsupervised learning based on motion patterns to classify main groups of objects in a fish farm. The main focus will be separating fish from feed. The approach is based on the hypothesis that fish and feed have distinct motion patterns that are suitable to use as classification criteria. The approach is based on optical flow by using KLT-tracking to estimate motion in the image. Similar motion patterns are grouped together using cluster analysis. Mean shift and DBSCAN were chosen as the algorithms to be used in the experiments, based on a preliminary analysis of the motion data. Mean shift is centroid based, while DBSCAN is density based which gives a useful combination of differing properties to compare. Further, the effect of adding object sizes to the clustering was studied. Object sizes were estimated by using image segmentation. The segmentation algorithm is based on edge detection, using a Sobel operator to create gradient images that can be used as basis for finding contours of objects. Results showed that classifying fish and feed based on motion patterns is plausible under certain conditions. There are some requirements for the camera position that improves the classification. For instance the clustering performance increases when numerous objects is visible in the image. Accurately selecting clustering parameters are also necessary to avoid cluster merging. In cases where several clusters are merged together, all valuable information about the objects are lost. The effect of adding object sizes to the clustering proved to be as expected. It resulted in improved separability of the motion patterns, although the segmentation accuracy required, made the proposed approach not robust enough to calculate the object sizes automatically. The main issues were incomplete contours because of too low contrast towards the background and overlapping objects. As a cause of the inaccurate segmentation, the number of data samples were too small to draw any conclusions, although the tendencies are that by using a better suited segmentation algorithm, a more consistent classification can be obtained.

This thesis looks into the applicability of visual saliency as a basis for autonomous camera orientation. Autonomous re-orientation is expected to be a key component in future computer vision-based monitoring systems: a fish-cage is a highly dynamic environment, and even a well-placed camera may no longer be optimally oriented for data gathering if left static over time. The underwater performance of visual saliency algorithms are tested and discussed. A few different attempts at addressing a major concern related to the cage net itself are made. Most notable is a filtering scheme based on an optical flow algorithm by Gunnar Farnebäck, the use of which successfully solves the presented challenge but also introduces a new problem. In conclusion, visual saliency can provide a viable basis for camera orientation, but only under certain conditions.

Develop a technique for measuring erosion using 3D-scans.

Registration of three-dimensional objects has vast applications in many areas of science. This research is linked to the international PRESIOUS project, concerning measurement of erosion effects on cultural heritage objects. The data in question is several magnitudes larger than found in research so far, often reaching several hundred million points. This thesis presents a novel approach to registration of such large data sets by co-dividing the sets into corresponding parts and registering them locally. The same division algorithm can be used to precisely measure the distance between the data sets. We test experimentally how the local registration results can be used to obtain an optimal global transformation for the entire data set.

In this thesis we have implemented and developed the necessary hardware and software solutions to transform a radio controlled quad copter into an autonomous drone. The main goal for this thesis was to safely land the drone by visual navigation on a given landing spot. To make this possible the we started with the assembly of a radio controlled quad copter from a almost-ready-to-fly kit (ARF) from AeroQuad. We then applied the necessary hardware to the drone by adding a web camera for visual abilities, an embedded computer for processing power and wireless access point for telemetry. We implemented computer vision algorithms the enable visual navigation and the linear Kalman filter to give robustness to the visual tracking and to cope with noisy sensor observation.A proportional-integral-derivative controller (PID controller) was developed to control the drones pose and fly to desired positions.These implementations was tested in experiments in a real-world environment and the results shows the drones ability to perform autonomous tasks based on visual navigation are presented in results.