Search

With the development of intelligent manufacturing and robotic technologies, the capability of grasping unknown objects in unstructured environments is becoming more prominent for robots with extensive applications. However, current robotic three-finger grasping studies only focus on grasp generation for single objects or scattered scenes, and suffer from high time expenditure to label grasp ground truth, making them incapable of predicting grasp poses for cluttered objects or generating large-scale datasets. To address such limitations, we first introduce a novel three-finger grasp representation with fewer prediction dimensions, which balances the training difficulty and representation accuracy to obtain efficient grasp performance. Based on this representation, we develop an auto-annotation pipeline and contribute a large-scale three-finger grasp dataset (TF-Grasp Dataset). Our dataset contains 222,720 RGB-D images with over 2 billion grasp annotations in cluttered scenes. In addition, we also propose a three-finger grasp pose detection network (TF-GPD), which detects globally while fine-tuning locally to predict high-quality collision-free grasps from a single-view point cloud. In sum, our work addresses the issue of high-quality collision-free three-finger grasp generation in cluttered scenes based on the proposed pipeline. Extensive comparative experiments show that our proposed methodology outperforms previous methods and improves the grasp quality and efficiency in clutters. The superior results in real-world robot grasping experiments not only prove the reliability of our grasp model but also pave the way for practical applications of three-finger grasping. Our dataset and source code will be released., QC 20240725

Template matching is a fundamental problem in computer vision with applications in fields including object detection, image registration, and object tracking. Current methods rely on nearest-neighbour (NN) matching, where the query feature space is converted to NN space by representing each query pixel with its NN in the template. NN-based methods have been shown to perform better in occlusions, appearance changes, and non-rigid transformations; however, they scale poorly with high-resolution data and high feature dimensions. We present an NN-based method that efficiently reduces the NN computations and introduces filtering in the NN fields (NNFs). A vector quantization step is introduced before the NN calculation to represent the template with k features, and the filter response over the NNFs is used to compare the template and query distributions over the features. We show that state-of-the-art performance is achieved in low-resolution data, and our method outperforms previous methods at higher resolution.

Imaging through the multi-mode fiber (MMF) becomes an attractive approach for gaining visual access to confined spaces. However, current imaging techniques through a MMF still encounter challenges including modal dispersion, complex wave-front shaping mechanism, and expensive light sources and modulations. This work proposed a cost-efficient setup with three light-emitting diodes as the illumination light source (including red, green, and blue light) and a hybrid model including the principal component analysis and a variational auto-encoder (PCAVAE) for reconstructing the transmitted images. The reconstructed images demonstrate high fidelity compared with their ground truth images. The average similarity index value of the reconstructed images is as high as 0.99. Experimental works indicated that the proposed approach was capable of rejecting 10% white noise in the imaging process. The proposed triple-color illumination method paves a cost-effective way of transmitting images through an MMF. The PCAVAE model established in this work demonstrates great potential for processing scrambled images transmitted by the MMF., QC 20240318

Over the years, research in neuroscience-driven marketing has progressively delved into the conscious and subconscious behaviors of consumers. Existing Electroencephalography (EEG)-based studies related to consumer preferences toward products are not comprehensive. Due to non-stationarity issues of EEG, a significant variance is observed in inter-trial and inter-session EEG signals of a subject, which leads to challenges in building a universal consumer preference model across diverse subjects, sessions, and tasks. Transfer learning mitigates this challenge by utilizing data or knowledge from similar subjects, sessions, or tasks to improve the learning process for a new subject, session, or task, thereby enhancing overall model performance. Moreover, high-dimensional EEG features often lead to poor classification results. Therefore, selecting meaningful or refined features is of utmost importance for classification. Therefore, we propose a robust EEG-based neuromarketing framework combining deep transfer learning, spatial attention model, and deep neural networks. The proposed framework predicts the consumer choices (in terms of “likes” and “dislikes”) for e-commerce products. Initially, the knowledge distillation is performed from the pre-trained network to the proposed model, and the model is trained on the connectivity features of EEG. Next, the attention-based features are extracted from high-level connectivity features using the spatial attention model (Convolutional Block Attention Module: CBAM). CBAM extracts the attention feature maps along channel and spatial dimensions for adaptive feature refinement. The refined features improve the classification accuracy. Finally, the attention-based features are passed to the 2D CNN-based deep learning model to evaluate consumer choices. The proposed model achieves 95.60% classification accuracy with the experimental dataset. The proposed model achieves a significant improvement of 2.60% over the existing neuromarketin, Validerad;2024;Nivå 2;2024-03-18 (hanlid);Full text license: CC BY-NC-ND

The current technological revolution driven by advances in machine learning has motivated a wide range of applications aiming to improve our quality of life. Representative of such applications are autonomous and semiautonomous Powered Wheelchairs (PWs), where the focus is on providing a degree of autonomy to the wheelchair user as a matter of guidance and interaction with the environment. Based on these perspectives, the focus of the current research has been on the design of lightweight systems that provide the necessary accuracy in the navigation system while enabling an embedded implementation. This motivated us to develop a real-time measurement methodology that relies on a monocular RGB camera to detect the caregiver’s feet based on a deep learning method, followed by the distance measurement of the caregiver from the PW. An important contribution of this article is the metrological characterization of the proposed methodology in comparison with measurements made with dedicated depth cameras. Our results show that despite shifting from 3D imaging to 2D imaging, we can still obtain comparable metrological performances in distance estimation as compared with Light Detection and Ranging (LiDAR) or even improved compared with stereo cameras. In particular, we obtained comparable instrument classes with LiDAR and stereo cameras, with measurement uncertainties within a magnitude of 10 cm. This is further complemented by the significant reduction in data volume and object detection complexity, thus facilitating its deployment, primarily due to the reduced complexity of initial calibration, positioning, and deployment compared with three-dimensional segmentation algorithms.

In ophthalmic diagnostics, achieving precise segmentation of retinal blood vessels is a critical yet challenging task, primarily due to the complex nature of retinal images. The intricacies of these images often hinder the accuracy and efficiency of segmentation processes. To overcome these challenges, we introduce the cognitive DL retinal blood vessel segmentation (CoDLRBVS), a novel hybrid model that synergistically combines the deep learning capabilities of the U-Net architecture with a suite of advanced image processing techniques. This model uniquely integrates a preprocessing phase using a matched filter (MF) for feature enhancement and a post-processing phase employing morphological techniques (MT) for refining the segmentation output. Also, the model incorporates multi-scale line detection and scale space methods to enhance its segmentation capabilities. Hence, CoDLRBVS leverages the strengths of these combined approaches within the cognitive computing framework, endowing the system with human-like adaptability and reasoning. This strategic integration enables the model to emphasize blood vessels, accurately segment effectively, and proficiently detect vessels of varying sizes. CoDLRBVS achieves a notable mean accuracy of 96.7%, precision of 96.9%, sensitivity of 99.3%, and specificity of 80.4% across all of the studied datasets, including DRIVE, STARE, HRF, retinal blood vessel and Chase-DB1. CoDLRBVS has been compared with different models, and the resulting metrics surpass the compared models and establish a new benchmark in retinal vessel segmentation. The success of CoDLRBVS underscores its significant potential in advancing medical image processing, particularly in the realm of retinal blood vessel segmentation.

This work examines the adaptability of Deep Perceptual Similarity (DPS) metrics to context beyond those that align with average human perception and contexts in which the standard metrics have been shown to perform well. Prior works have shown that DPS metrics are good at estimating human perception of similarity, so-called perceptual similarity. However, it remains unknown whether such metrics can be adapted to other contexts. In this work, DPS metrics are evaluated for their adaptability to different contradictory similarity contexts. Such contexts are created by randomly ranking six image distortions. Metrics are adapted to consider distortions more or less disruptive to similarity depending on their place in the random rankings. This is done by training pretrained CNNs to measure similarity according to given contexts. The adapted metrics are also evaluated on a perceptual similarity dataset to evaluate whether adapting to a ranking affects their prior performance. The findings show that DPS metrics can be adapted with high performance. While the adapted metrics have difficulties with the same contexts as baselines, performance is improved in 99% of cases. Finally, it is shown that the adaption is not significantly detrimental to prior performance on perceptual similarity. The implementation of this work is available online.

This work presents EAT (Environment Agnostic Traversability for Reactive Navigation) a novel framework for traversability estimation in indoor, outdoor, subterranean (SubT) and other unstructured environments. The architecture provides updates on traversable regions online during the mission, adapts to varying environments, while being robust to noisy semantic image segmentation. The proposed framework considers terrain prioritization based on a novel decay exponential function to fuse the semantic information and geometric features extracted from RGB-D images to obtain the traversability of the scene. Moreover, EAT introduces an obstacle inflation mechanism on the traversability image, based on mean-window weighting module, allowing to adapt the proximity to untraversable regions. The overall architecture uses two LRASPP MobileNet V3 large Convolutional Neural Networks (CNN) for semantic segmentation over RGB images, where the first one classifies the terrain types and the second one classifies see-through obstacles in the scene. Additionally, the geometric features profile the underlying surface properties of the local scene, extracting normals from depth images. The proposed scheme was integrated with a control architecture in reactive navigation scenarios and was experimentally validated in indoor and outdoor environments as well as in subterranean environments with a Pioneer 3AT mobile robot., Validerad;2024;Nivå 2;2024-02-12 (joosat);Funder: European Unions Horizon 2020 Research and Innovation Programme (101003591 NEXGEN-SIMS);Full text license: CC BY

Over the past few years, there has been a noticeable surge in efforts to design novel tools and approaches that incorporate Artificial Intelligence (AI) into rehabilitation of persons with lower-limb impairments, using robotic exoskeletons. The potential benefits include the ability to implement personalized rehabilitation therapies by leveraging AI for robot control and data analysis, facilitating personalized feedback and guidance. Despite this, there is a current lack of literature review specifically focusing on AI applications in lower-limb rehabilitative robotics. To address this gap, our work aims at performing a review of 37 peer-reviewed papers. This review categorizes selected papers based on robotic application scenarios or AI methodologies. Additionally, it uniquely contributes by providing a detailed summary of input features, AI model performance, enrolled populations, exoskeletal systems used in the validation process, and specific tasks for each paper. The innovative aspect lies in offering a clear understanding of the suitability of different algorithms for specific tasks, intending to guide future developments and support informed decision-making in the realm of lower-limb exoskeleton and AI applications.

Sketch-based image retrieval (SBIR) utilizes sketches to search for images containing similar objects or scenes. Due to the proliferation of touch-screen devices, sketching has become more accessible and therefore has received increasing attention. Deep learning has emerged as a potential tool for SBIR, allowing models to automatically extract image features and learn from large amounts of data. To the best of our knowledge, there is currently no systematic literature review (SLR) of SBIR with deep learning. Therefore, the aim of this review is to incorporate related works into a systematic study, highlighting the main contributions of individual researchers over the years, with a focus on past, present and future trends. To achieve the purpose of this study, 90 studies from 2016 to June 2023 in 4 databases were collected and analyzed using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) framework. The specific models, datasets, evaluation metrics, and applications of deep learning in SBIR are discussed in detail. This study found that Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN) are the most widely used deep learning methods for SBIR. A commonly used dataset is Sketchy, especially in the latest Zero-shot sketch-based image retrieval (ZS-SBIR) task. The results show that Mean Average Precision (mAP) is the most commonly used metric for quantitative evaluation of SBIR. Finally, we provide some future directions and guidance for researchers based on the results of this review. © 2013 IEEE.

Hyperspectral image (HSI) classification is an essential task in remote sensing with substantial practical significance. However, most existing convolutional neural network (CNN)-based classification methods focus only on local spatial features while neglecting global spectral dependencies. Meanwhile, Transformer-based methods exhibit robust capabilities for global spectral feature modeling but struggle to extract local spatial features effectively. To fully exploit the local spatial feature extraction capabilities of CNN-based networks and the global spectral feature extraction capabilities of Transformer-based networks, this article proposes a dual-branch convolutional Transformer method with efficient interactive self-attention (EISA) for HSI classification, namely the dual-branch convolutional transformer network (DCTN), which can aggregate local and global spatial-spectral features fully. Specifically, DCTN includes two core modules: the spatial-spectral fusion projection module (SFPM) and the EISA module. The former utilizes 3-D convolution with adaptive pooling and 2-D group convolution with residual connection to parallel extract fused and grouped spatial-spectral features, respectively. The latter performs EISA across height, width, and spectral dimensions, enabling deep fusion of spatial-spectral features. Extensive experiments on three real HSI datasets demonstrate that the proposed DCTN method outperforms existing classification methods, yielding state-of-the-art classification performance. The code is available at https://github.com/AllFever/DeepHyperX-DCTN for reproducibility., QC 20240304

Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where the data from previously seen domains cannot be stored, e.g., due to data protection regulations, the above becomes a challenging continual learning problem. To this end, we propose to use generative feature-driven image replay in conjunction with a dual-purpose discriminator that not only enables the generation of images with realistic features for replay, but also promotes feature alignment during domain adaptation. We evaluate our approach extensively on a sequence of three histopathological datasets for tissue-type classification, achieving state-of-the-art results. We present detailed ablation experiments studying our proposed method components and demonstrate a possible use-case of our continual UDA method for an unsupervised patch-based segmentation task given high-resolution tissue images. Our code is available at: https://github.com/histocartography/multi-scale-feature-alignment.

Modern autonomous systems typically use several sensors for perception. For best performance, accurate and reliable extrinsic calibration is necessary. In this research, we proposea reliable technique for the extrinsic calibration of several lidars on a vehicle without the need for odometry estimation or fiducial markers. First, our method generates an initial guess of the extrinsics by matching the raw signals of IMUs co-located with each lidar. This initial guess is then used in ICP and point cloud feature matching which refines and verifies this estimate. Furthermore, we can use observability criteria to choose a subset of the IMU measurements that have the highest mutual information — rather than comparing all the readings. We have successfully validated our methodology using data gathered from Scania test vehicles., Submitted to IEEE IV 2024​ 35th IEEE Intelligent Vehicles Symposium, June 2 - 5, 2024. Jeju Shinhwa World, Jeju Island, KoreaQC 20240216

Deep learning and remote sensing techniques have significantly advanced water surface monitoring; however, the need for annotated data remains a challenge. This is particularly problematic in wetland detection, where water extent varies over time and space, demanding multiple annotations for the same area. In this paper, we present DEEPAQUA, a deep learning model inspired by knowledge distillation (a.k.a. teacher–student model) to generate labeled data automatically and eliminate the need for manual annotations during the training phase. We utilize the Normalized Difference Water Index (NDWI) as a teacher model to train a Convolutional Neural Network (CNN) for segmenting water from Synthetic Aperture Radar (SAR) images. To train the student model, we exploit cases where optical- and radar-based water masks coincide, enabling the detection of both open and vegetated water surfaces. DEEPAQUA represents a significant advancement in computer vision techniques for water detection by effectively training semantic segmentation models without any manually annotated data. Experimental results show that DEEPAQUA outperforms other unsupervised methods by improving accuracy by 3%, Intersection Over Union by 11%, and F1-score by 6%. This approach offers a practical solution for monitoring wetland water extent changes without the need of ground truth data, making it highly adaptable and scalable for wetland monitoring., QC 20240108

We explore multi-log grasping using reinforcement learning and virtual visual servoing for automated forwarding in a simulated environment. Automation of forest processes is a major challenge, and many techniques regarding robot control pose different challenges due to the unstructured and harsh outdoor environment. Grasping multiple logs involves various problems of dynamics and path planning, where understanding the interaction between the grapple, logs, terrain, and obstacles requires visual information. To address these challenges, we separate image segmentation from crane control and utilise a virtual camera to provide an image stream from reconstructed 3D data. We use Cartesian control to simplify domain transfer to real-world applications. Because log piles are static, visual servoing using a 3D reconstruction of the pile and its surroundings is equivalent to using real camera data until the point of grasping. This relaxes the limits on computational resources and time for the challenge of image segmentation, and allows for data collection in situations where the log piles are not occluded. The disadvantage is the lack of information during grasping. We demonstrate that this problem is manageable and present an agent that is 95% successful in picking one or several logs from challenging piles of 2–5 logs., Mistra Digital Forest

Recent advances in artificial intelligence and machine learning have significantly impacted the field of robotics and led to the interdisciplinary study of robot learning. These developments have the potential to revolutionize the automation of tasks in various industries by reducing the reliance on human workers. However, fully autonomous, learning-based robotic systems are still mainly limited to controlled environments. Ideally, we are looking for methods that enable autonomous acquisition of robotic skills for any temporally extended setting with potentially complex sensor observations. Classical sampling-based planning algorithms used in robot motion planning compute feasible paths between robot states over long time horizons and even in geometrically complex environments. This thesis investigates the possibility of combining learning-based methods with these classical approaches to solve challenging problems in robot manipulation, e.g. the manipulation of deformable objects. The core idea is to leverage the best of both worlds and achieve long-horizon control through planning, while using learning to obtain useful environment models from potentially high-dimensional and complex observation data. The presented frameworks rely on recent machine learning techniques such as contrastive representation learning, generative modeling and reinforcement learning. Finally, we outline the potentials, challenges and limitations of this type of approaches and highlight future directions., De senaste framstegen inom artificiell intelligens och maskininlärning har haft en betydande inverkan på robotikområdet och lett till det tvärvetenskapliga studerandet av robotinlärning. Dessa utvecklingar har potentialen att revolutionera automatiseringen inom olika industrier genom att minska beroendet av mänskliga arbetare. Dock är helt autonoma, inlärningsbaserade robotsystem fortfarande huvudsakligen begränsade till kontrollerade miljöer. Idealt sett letar vi efter metoder som möjliggör autonom förvärvning av robotfärdigheter för situationer med långa tidshorisonter och potentiellt komplexa sensorobservationer. Klassiska sampling-baserade planeringsalgoritmer som används i robotrörelseplanering beräknar genomförbara vägar mellan robottillstånd över långa tidshorisonter och även i geometriskt komplexa miljöer. I detta arbete undersöker vi möjligheten att kombinera inlärningsbaserade tillvägagångssätt med dessa klassiska tillvägagångssätt för att lösa utmanande problem inom robotmanipulation, t.ex. hantering av formbara objekt. Kärnidén är att utnyttja det bästa av båda världarna och uppnå långsiktig kontroll genom planering, samtidigt som man använder inlärning för att erhålla användbara miljömodeller från potentiellt högdimensionella och komplexa observationsdata. De presenterade ramverken förlitar sig på senaste maskininlärningstekniker såsom kontrastiv representationsinlärning, generativ modellering och förstärkningsinlärning. Slutligen skisserar vi potentialerna, utmaningarna och begränsningarna med denna typ av tillvägagångssätt och belyser framtida riktningar., QC 20240108

Human–robot collaborative assembly (HRCA) combines the flexibility and adaptability of humans with the efficiency and reliability of robots during collaborative assembly operations, which facilitates complex product assembly in the mass personalisation paradigm. The cognitive ability of robots to recognise and predict human actions and make responses accordingly is essential but currently still limited, especially when facing highly similar human actions. To improve the cognitive ability of robots in HRCA, firstly, a two-stage skeleton-RGB integrated model focusing on human-parts interaction is proposed to recognise highly similar human actions. Specifically, it consists of a feature guidance module and a feature fusion module, which can balance the accuracy and efficiency of human action recognition. Secondly, an online prediction approach is developed to predict human actions ahead of schedule, which includes a pre-trained skeleton-RGB integrated model and a preprocessing module. Thirdly, considering the positioning accuracy of the parts to be assembled and the continuous update of human actions, a dynamic response scheme of the robot is designed. Finally, the feasibility and effectiveness of the proposed model and approach are verified by a case study of a worm-gear decelerator assembly. The experimental results demonstrate that the proposed model achieves precise human action recognition with a high accuracy of 93.75% and a lower computational cost. Specifically, only 15 frames from a skeleton stream and 5 frames (less than 16 frames in general) from an RGB video stream are adopted. Moreover, it only takes 1.026 s to achieve online human action prediction based on the proposed prediction method. The dynamic response scheme of the robot is also proven to be feasible. It is expected that the efficiency of human–robot interaction in HRCA can be improved from a closed-loop view of perception, prediction, and response., QC 20231024

Traffic fatalities remain among the leading death causes worldwide. To reduce this figure, car safety is listed as one of the most important factors. To actively support human drivers, it is essential for advanced driving assistance systems to be able to recognize the driver's actions and intentions. Prior studies have demonstrated various approaches to recognize driving actions and intentions based on in-cabin and external video footage. Given the performance of self-supervised video pre-trained (SSVP) Video Masked Autoencoders (VMAEs) on multiple action recognition datasets, we evaluate the performance of SSVP VMAEs on the Honda Research Institute Driving Dataset for driver action recognition (DAR) and on the Brain4Cars dataset for driver intention recognition (DIR). Besides the performance, the application of an artificial intelligence system in a safety-critical environment must be capable to express when it is uncertain about the produced results. Therefore, we also analyze uncertainty estimations produced by a Bayes-by-Backprop last-layer (BBB-LL) and Monte-Carlo (MC) dropout variants of an VMAE. Our experiments show that an VMAE achieves a higher overall performance for both offline DAR and end-to-end DIR compared to the state-of-the-art. The analysis of the BBB-LL and MC dropout models show higher uncertainty estimates for incorrectly classified test instances compared to correctly predicted test instances.

This thesis investigates the seasonal predictive capabilities of Neural Radiance Fields (NeRF) applied to satellite images. Focusing on the utilization of satellite data, the study explores how Sat-NeRF, a novel approach in computer vision, per- forms in predicting seasonal variations across different months. Through compre- hensive analysis and visualization, the study examines the model’s ability to cap- ture and predict seasonal changes, highlighting specific challenges and strengths. Results showcase the impact of the sun on predictions, revealing nuanced details in seasonal transitions, such as snow cover, color accuracy, and texture represen- tation in different landscapes. The research introduces modifications to the Sat- NeRF network. The implemented versions of the network include geometrically rendered shadows, a signed distance function, and a month embedding vector, where the last version mentioned resulted in Planet-NeRF. Comparative evalua- tions reveal that Planet-NeRF outperforms prior models, particularly in refining seasonal predictions. This advancement contributes to the field by presenting a more effective approach for seasonal representation in satellite imagery analysis, offering promising avenues for future research in this domain.

Since the advent of machine learning and machine vision methods, progress has been made in tackling the long-standing research question of autonomous grasping of arbitrary objects using robotic end-effectors. Building on these efforts, we focus on a subset of the general grasping problem concerning the automation of a forwarder. This forestry vehicle collects and transports felled and cut tree logs in a forest environment to a nearby roadside landing. The forwarder must safely and energy-efficiently grip logs to minimize fuel consumption and reduce loading times. In this thesis project, we develop a data-driven model for predicting the expected outcome of grasping attempts made by the forwarder's crane. For a given pile of logs, such a model can estimate the optimal horizontal location and angle for applying the claw grapple, enabling effective grasp planning. We utilize physics-based simulations to create a ground truth dataset of 12 500 000 simulated grasps distributed across 5000 randomly generated log piles. Our semi-generative, supervised model is a fully convolutional network that inputs the orthographic depth image of a pile and returns images predicting the corresponding grasps' initial grapple angle and outcome metrics as a function of position. Over five folds of cross-validation, our model predicted the number of grasped logs and the initial grapple angle with a normalized root mean squared error of 15.77(2)% and 2.64(4)%, respectively. The grasps' energy efficiency and energy waste were similarly predicted with a relative error of 14.43(2)% and 21.06(3)%., Sedan tillkomsten av maskininlärnings- och maskinseendebaserade metoder har betydande framsteg gjorts inom forskningsområdet för autonom greppning av godtyckliga objekt med en robotisk sluteffektor. Vi bygger vidare på dessa resultat och fokuserar på en del av det generella greppningsproblemet gällande automatisering av en skotare. Denna skogsmaskin samlar in och transporterar fällda och kapade trädstammar från avverkningsplats till upplag intill närliggande skogsbilväg. Skotaren måste greppa och lyfta stockarna på ett säkert och energieffektivt sätt för att minimera bränsleförbrukningen samt minska lastningstiderna. I detta examensarbete utvecklar vi en datadriven modell för att förutsäga det förväntade resultatet av gripförsök utförda av skotarens kran. För en given timmerstockshög kan en sådan modell uppskatta den optimala positionen och vinkeln för att applicera skotarens gripklo, vilket möjliggör effektiv planering av lastningen. Vi använder fysikbaserade simuleringar för att skapa ett dataset med 12 500 000 simulerade gripförsök fördelade över 5000 slumpmässigt genererade timmerhögar. Vår semi-generativa, övervakade modell är ett djupt faltningsnätverk utan helt sammankopplade neuronlager som tar in en ortografisk djupbild av en timmerhög och returnerar bilder som predikterar de motsvarande gripförsökens initiala gripvinkel och resultatmått som en funktion av position. Vid en femfaldig korsvalidering förutsåg vår modell antalet greppade stockar och den initiala gripvinkeln med ett normaliserat rotmedelkvadratfel på 15.77(2)% respektive 2.64(4)%. Gripförsökens energieffektivitet och energiförlust predikterades på liknande sätt med ett relativt fel på 14.43(2)% och 21.06(3)%.

A persisting problem with existing super-resolution (SR) models is that they cannot produce minute details of anatomical structures, pathologies, and textures critical for proper diagnosis. This is mainly because they assume specific degradations like bicubic downsampling or Gaussian noise, whereas, in practice, the degradations can be more complex and hence need to be modelled "blindly". We propose a novel attention-based GAN model for medical image super-resolution that models the degradation in a data-driven agnostic way ("blind") to achieve better fidelity of diagnostic features in medical images. We introduce a new ensemble loss in the generator that boosts performance and a spectral normalisation in the discriminator to enhance stability. Experimental results on lung CT scans demonstrate that our model, BliMSR, produces super-resolved images with enhanced details and textures and outperforms recent competing models, including a diffusion model for generating super-resolution images, thus establishing a state-of-the-art. The code is available at https://github.com/Samiran-Dey/BliMSR.

CC BY-NC-ND 4.0The present work was financially supported by the Swedish innovation agency Vinnova through the research and innovation programme Produktion2030, grant #2022-01279: Empowering Human Workers for Assembly of Wire Harnesses (EWASS), Empowering Human Workers for Assembly of Wire Harnesses (EWASS)

The development of high-brilliance X-ray sources, such as the fourth-generation diffraction-limited storage rings and X-ray free-electron lasers, have opened up new possibilities for X-ray imaging and pushed the temporal resolutions of imaging techniques to unprecedented levels. Capturing fast dynamics in two-dimensional (2D), three-dimensional (3D), and even four-dimensional (4D, 3D + time) beyond microsecond temporal resolution has become possible. To fully exploit the unique capabilities of these facilities, challenges such as the data problem must be addressed. Automated tools are needed to handle the large amount of data acquired from each experiment.As a data-driven approach, deep learning has undergone rapid development over the past decade and offers a promising solution to this problem. However, state-of-the-art deep learning methods applied to X-ray imaging ignore the physics of X-ray propagation and interaction with matter and require paired training datasets. In this thesis, we show that combining the physical principles of X-ray imaging with deep learning greatly improves the performance and robustness of the approaches, and it is possible to construct reliable unsupervised approaches, where no paired datasets are needed.Firstly, we present a theoretical background on X-ray imaging and various imaging methods. Secondly, we provide an overview of deep learning, including training strategies and common frameworks for addressing imaging tasks. Lastly, we introduce novel algorithms developed during this thesis:1. FFCGAN, a supervised approach for shot-to-shot flat-field correction at X-ray free-electron lasers.2. PhaseGAN, a phase-retrieval approach for unpaired datasets.3. ONIX, a self-supervised approach for 3D reconstruction from sparse views.4. 4D-ONIX, a self-supervised approach for reconstructing 3D movies from sparse projections.These approaches offer high-quality image reconstructions for X-ray imaging techniques, enabling further exploration and un

Health-promoting digital agents, taking on the role of an assistant, coach or companion, are expected to have knowledge about a person's medical and health aspects, yet they typically lack knowledge about the person's activities. These activities may vary daily or weekly and are contextually situated, posing challenges for the human-Agent interaction. This pilot study aimed to explore the experiences and behaviors of older adults when interacting with an initially unknowledgeable digital agent that queries them about an activity that they are simultaneously engaged in. Five older adults participated in a scenario involving preparing coffee followed by having coffee with a guest. While performing these activities, participants educated the smartwatch-embedded agent, named Virtual Occupational Therapist (VOT), about their activity performance by answering a set of activity-ontology based questions posed by the VOT. Participants' interactions with the VOT were observed, followed by a semi-structured interview focusing on their experience with the VOT. Collected data were analyzed using an activity-Theoretical framework. Results revealed participants exhibited agency and autonomy, deciding whether to adapt to the VOT's actions in three phases: Adjustment to the VOT, partial adjustment, and the exercise of agency by putting the VOT to sleep after the social conditions and activity changed. Results imply that the VOT should incorporate the ability to distinguish when humans collaborate as expected by the VOT and when they choose not to comply and instead act according to their own agenda. Future research focuses on how collaboration evolves and how the VOT needs to adapt in the process., HumanE-AI-Net

This paper presents a method for learning world models for wheel loaders performing automatic loading actions on a pile of soil. Data-driven models were learned to output the resulting pile state, loaded mass, time, and work for a single loading cycle given inputs that include a heightmap of the initial pile shape and action parameters for an automatic bucket-filling controller. Long-horizon planning of sequential loading in a dynamically changing environment is thus enabled as repeated model inference. The models, consisting of deep neural networks, were trained on data from a 3D multibody dynamics simulation of over 10,000 random loading actions in gravel piles of different shapes. The accuracy and inference time for predicting the loading performance and the resulting pile state were, on average, 95% in 1.21.2 ms and 97% in 4.54.5 ms, respectively. Long-horizon predictions were found feasible over 40 sequential loading actions.

Even with widespread adoption of automated anomaly detection in safety-critical areas, both classical and advanced machine learning models are susceptible to first-order evasion attacks that fool models at run-time (e.g. an automated firewall or an anti-virus application). Kernelized support vector machines (KSVMs) are an especially useful model because they combine a complex geometry with low run-time requirements (e.g. when compared to neural networks), acting as a run-time lower bound when compared to contemporary models (e.g. deep neural networks), to provide a cost-efficient way to measure model and attack run-time costs. To properly measure and combat adversaries, we propose a massively parallel projected gradient descent (PGD) evasion attack framework. Through theoretical examinations and experiments carried out using linearly-separable Gaussian normal data, we present (i) a massively parallel naive attack, we show that adversarial retraining is unlikely to be an effective means to combat an attacker even on linearly separable datasets, (ii) a cost effective way of evaluating models defences and attacks, and an extensible code base for doing so, (iii) an inverse relationship between adversarial robustness and benign accuracy, (iv) the lack of a general relationship between attack time and efficacy, and (v) that adversarial retraining increases compute time exponentially while failing to reliably prevent highly-confident false classifications.

The use of deep neural networks (DNNs) in medical images has enabled the development of solutions characterized by the need of leveraging information coming from multiple sources, raising the Multimodal Deep Learning. DNNs are known for their ability to provide hierarchical and high-level representations of input data. This capability has led to the introduction of methods performing data fusion at an intermediate level, preserving the distinctiveness of the heterogeneous sources in modality-specific paths, while learning the way to define an effective combination in a shared representation. However, modeling the intricate relationships between different data remains an open issue. In this paper, we aim to improve the integration of data coming from multiple sources. We introduce between layers belonging to different modality-specific paths a Transfer Module (TM) able to perform the cross-modality calibration of the extracted features, reducing the effects of the less discriminative ones. As case of study, we focus on the axillary lymph nodes metastasis evaluation in malignant breast cancer, a crucial prognostic factor, affecting patient’s survival. We propose a Multi-Input Single-Output 3D Convolutional Neural Network (CNN) that considers both images acquired with multiparametric Magnetic Resonance and clinical information. In particular, we assess the proposed methodology using four architectures, namely BasicNet and three ResNet variants, showing the improvement of the performance obtained by including the TM in the network configuration. Our results achieve up to 90% and 87% of accuracy and Area under ROC curve, respectively when the ResNet10 is considered, surpassing various fusion strategies proposed in the literature.

The influence of natural image transformations on receptive field responses is crucial for modelling visual operations in computer vision and biological vision. In this regard, covariance properties with respect to geometric image transformations in the earliest layers of the visual hierarchy are essential for expressing robust image operations, and for formulating invariant visual operations at higher levels. This paper defines and proves a set of joint covariance properties for spatio-temporal receptive fields in terms of spatio-temporal derivative operators applied to spatio-temporally smoothed image data under compositions of spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations. Specifically, the derived relations show how the parameters of the receptive fields need to be transformed, in order to match the output from spatio-temporal receptive fields under composed spatio-temporal image transformations. For this purpose, we also fundamentally extend the notion of scale-normalized derivatives to affine-normalized derivatives, that are computed based on spatial smoothing with affine Gaussian kernels, and analyze the covariance properties of the resulting affine-normalized derivatives for the affine group as well as for important subgroups thereof. We conclude with a geometric analysis, showing how the derived joint covariance properties make it possible to relate or match spatio-temporal receptive field responses, when observing, possibly moving, local surface patches from different views, under locally linearized perspective or projective transformations, as well as when observing different instances of spatio-temporal events, that may occur either faster or slower between different views of similar spatio-temporal events. We do furthermore describe how the parameters in the studied composed spatio-temporal image transformation models directly relate to geometric entities in the image forma, QC 20240709

This paper presents an analysis of properties of two hybrid discretization methods for Gaussian derivatives, based on convolutions with either the normalized sampled Gaussian kernel or the integrated Gaussian kernel followed by central differences. The motivation for studying these discretization methods is that in situations when multiple spatial derivatives of different order are needed at the same scale level, they can be computed significantly more efficiently compared to more direct derivative approximations based on explicit convolutions with either sampled Gaussian kernels or integrated Gaussian kernels. While these computational benefits do also hold for the genuinely discrete approach for computing discrete analogues of Gaussian derivatives, based on convolution with the discrete analogue of the Gaussian kernel followed by central differences, the underlying mathematical primitives for the discrete analogue of the Gaussian kernel, in terms of modified Bessel functions of integer order, may not be available in certain frameworks for image processing, such as when performing deep learning based on scale-parameterized filters in terms of Gaussian derivatives, with learning of the scale levels. The hybrid discretizations studied in this paper do, from this perspective, offer a computationally more efficient way of implementing deep networks based on Gaussian derivatives for such use cases. In this paper, we present a characterization of the properties of these hybrid discretization methods, in terms of quantitative performance measures concerning the amount of spatial smoothing that they imply, as well as the relative consistency of scale estimates obtained from scale-invariant feature detectors with automatic scale selection, with an emphasis on the behaviour for very small values of the scale parameter, which may differ significantly from corresponding results obtained from the fully continuous scale-space theory, as well as between different types of dis, QC 20240514, Covariant and invariant deep networks

The influence of natural image transformations on receptive field responses is crucial for modelling visual operations in computer vision and biological vision. In this regard, covariance properties with respect to geometric image transformations in the earliest layers of the visual hierarchy are essential for expressing robust image operations, and for formulating invariant visual operations at higher levels. This paper defines and proves a set of joint covariance properties under compositions of spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations, which make it possible to characterize how different types of image transformations interact with each other and the associated spatio-temporal receptive field responses. In this regard, we also extend the notion of scale-normalized derivatives to affine-normalized derivatives, to be able to obtain true affine-covariant properties of spatial derivatives, that are computed based on spatial smoothing with affine Gaussian kernels. The derived relations show how the parameters of the receptive fields need to be transformed, in order to match the output from spatio-temporal receptive fields under composed spatio-temporal image transformations. As a side effect, the presented proof for the joint covariance property over the integrated combination of the different geometric image transformations also provides specific proofs for the individual transformation properties, which have not previously been fully reported in the literature. We conclude with a geometric analysis, showing how the derived joint covariance properties make it possible to relate or match spatio-temporal receptive field responses, when observing, possibly moving, local surface patches from different views, under locally linearized perspective or projective transformations, as well as when observing different instances of spatio-temporal events that may occur either faster or slower be, QC 20240429, Covariant and invariant deep networks

Biological nervous systems constitute important sources of inspiration towards computers that are faster, cheaper, and more energy efficient. Neuromorphic disciplines view the brain as a coevolved system, simultaneously optimizing the hardware and the algorithms running on it. There are clear efficiency gains when bringing the computations into a physical substrate, but we presently lack theories to guide efficient implementations. Here, we present a principled computational model for neuromorphic systems in terms of spatio-temporal receptive fields, based on affine Gaussian kernels over space and leaky-integrator and leaky integrate-and-fire models over time. Our theory is provably covariant to spatial affine and temporal scaling transformations, and with close similarities to the visual processing in mammalian brains. We use these spatio-temporal receptive fields as a prior in an event-based vision task, and show that this improves the training of spiking networks, which otherwise is known as problematic for event-based vision. This work combines efforts within scale-space theory and computational neuroscience to identify theoretically well-founded ways to process spatio-temporal signals in neuromorphic systems. Our contributions are immediately relevant for signal processing and event-based vision, and can be extended to other processing tasks over space and time, such as memory and control., QC 20240508

Simultaneous localization and mapping (SLAM) is the task of building a map representation of an unknown environment while at the same time using it for positioning. A probabilistic interpretation of the SLAM task allows for incorporating prior knowledge and for operation under uncertainty. Contrary to the common practice of computing point estimates of the system states, we capture the full posterior density through approximate Bayesian inference. This dynamic learning task falls under state estimation, where the state-of-the-art is in sequential Monte Carlo methods that tackle the forward filtering problem. In this paper, we introduce a framework for probabilistic SLAM using particle smoothing that does not only incorporate observed data in current state estimates, but it also backtracks the updated knowledge to correct for past drift and ambiguities in both the map and in the states. Our solution can efficiently handle both dense and sparse map representations by Rao-Blackwellization of conditionally linear and conditionally linearized models. We show through simulations and real-world experiments how the principles apply to radio (Bluetooth low-energy/Wi-Fi), magnetic field, and visual SLAM. The proposed solution is general, efficient, and works well under confounding noise.

Driver intention recognition (DIR) methods mostly rely on deep neural networks (DNNs). To use DNNs in asafety-critical real-world environment it is essential to quantify how confident the model is about the producedpredictions. Therefore, this study evaluates the performance and calibration of a temporal convolutionalnetwork (TCN) for multiple probabilistic deep learning (PDL) methods (Bayes-by-Backprop, Monte-Carlodropout, Deep ensembles, Stochastic Weight averaging - Gaussian, Multi SWA-G, cyclic Stochastic GradientHamiltonian Monte Carlo). Notably, we formalize an approach that combines optimization-based pre-trainingwith Hamiltonian Monte-Carlo (PT-HMC) sampling, aiming to leverage the strengths of both techniques. Ouranalysis, conducted on two pre-processed open-source DIR datasets, reveals that PT-HMC not only matchesbut occasionally surpasses the performance of existing PDL methods. One of the remaining challenges thatprohibits the integration of a PDL-based DIR system into an actual car is the computational requirements toperform inference. Therefore, future work could focus on optimizing PDL methods to be more computationallyefficient without sacrificing performance or the ability to estimate uncertainties.

A crucial part of traditional reinforcement learning (RL) is the initial exploration phase, in which trying available actions randomly is a critical element. As random behavior might be detrimental to a social interaction, this work proposes a novel paradigm for learning social robot behavior--the use of shielding to ensure socially appropriate behavior during exploration and learning. We explore how a data-driven approach for shielding could be used to generate listening behavior. In a video-based user study (N=110), we compare shielded exploration to two other exploration methods. We show that the shielded exploration is perceived as more comforting and appropriate than a straightforward random approach. Based on our findings, we discuss the potential for future work using shielded and socially guided approaches for learning idiosyncratic social robot behaviors through RL., Paper will be published later this year (accepted camera-ready version available).QC 20240717

This paper develops an in-depth treatment concerning the problem of approximating the Gaussian smoothing and Gaussian derivative computations in scale-space theory for application on discrete data. With close connections to previous axiomatic treatments of continuous and discrete scale-space theory, we consider three main ways discretizing these scale-space operations in terms of explicit discrete convolutions, based on either (i) sampling the Gaussian kernels and the Gaussian derivative kernels, (ii) locally integrating the Gaussian kernels and the Gaussian derivative kernels over each pixel support region and (iii) basing the scale-space analysis on the discrete analogue of the Gaussian kernel, and then computing derivative approximations by applying small-support central difference operators to the spatially smoothed image data. We study the properties of these three main discretization methods both theoretically and experimentally, and characterize their performance by quantitative measures, including the results they give rise to with respect to the task of scale selection, investigated for four different use cases, and with emphasis on the behaviour at fine scales. The results show that the sampled Gaussian kernels and derivatives as well as the integrated Gaussian kernels and derivatives perform very poorly at very fine scales. At very fine scales, the discrete analogue of the Gaussian kernel with its corresponding discrete derivative approximations performs substantially better. The sampled Gaussian kernel and the sampled Gaussian derivatives do, on the other hand, lead to numerically very good approximations of the corresponding continuous results, when the scale parameter is sufficiently large, in the experiments presented in the paper, when the scale parameter is greater than a value of about 1, in units of the grid spacing. Below a standard deviation of about 0.75, the results from the sampled Gaussian kernel or the sampled Gaussian derivatives are, howe, QC 20240430, Covariant and invariant deep networks

The appearance of the log cross-section provides important information when assessing the quality of the log, where properties to consider include pith location and density of annual rings. This makes tasks like estimation of pith location and annual ring detection of great interest. However, creating labeled training data for these tasks can be time-consuming and subject to misjudgments. For this reason, we aim to create generated training data with controlled properties of pith location and amount of annual rings. We propose a two-step generator based on generative adversarial networks in which we can completely avoid manual labeling, not only when generating training data but also during training of the generator itself. This opens up the possibility to train the generator on other types of log end data without the need to manually label new training data. The same method is used to create two generated training datasets; one of entire log ends and one of patches of log ends. To evaluate how the generated data compares to real data, we train two deep learning models to perform estimation of pith location and ring counting, respectively. The models are trained separately on real and generated data and evaluated on real data only. The results show that the performance of both estimation of pith location and ring counting can be improved by replacing real training data with larger sets of generated training data.

Deep learning-based image segmentation methods require densely annotated and massive datasets to produce effective results. On the other hand, active contours-based methods are excellent alternatives to the situation, producing acceptable segmentation results. Earlier active contour models, including local and global region information, struggle with their limitations, such as spurious contours appearing in inhomogeneous images. Bias correction is utilized to solve the bias field's energy, considering the intensity inhomogeneity and the level set functions that suggest an image domain division. In our approach, we combine the advantages of local and global information in the image level set function, resulting in a combined energy function that aids in the efficient evolution of contours on images and can judge the relevance of the item and its surroundings. The proposed model computes data force by extracting local information from an in-homogeneous image using image-fitting energy and then computing all pixel values simultaneously. Objects with high differences between grey levels or more in-homogeneity can be segmented. The outcome demonstrates that our method is more dependable and computationally efficient than previous methods.

This thesis evaluates deep learning architectures for rats pose estimation through the presented six-camera system, focusing on ResNet and EfficientNet models via different depth and augmentation techniques. Among the configurations tested, ResNet 152 with default augmentation proved the best performance in the controlled experimental setup, particularly when employing a multi-perspective network approach. It reached a Root Mean Squared Error (RMSE) of 8.74, 8.78, and 9.72 pixels for the different angles. This configuration not only minimized test errors but also demonstrated its ability to track animal movement consistently. The utilization of data augmentation revealed that less altering yields better performance given the experimental conditions. The thesis suggests potential areas for future directions, including extended refinement of model configurations, further investigation of inference speed, and utilization of the network weights to transfer the approach to other species, such as mice. The findings underscore the potential for further development of deep learning solutions advancing pre-clinical research in behavioral neuroscience.

This study investigates the integration of monocular CAD and RGB-based 6D pose estimation into robotic lawnmowers, focusing on charging station docking. A systematic literature review establishes the foundation by exploring methodologies, models, architectures, datasets, and relevant topics from existing literature on 6D pose estimation with CAD and RGB data. An approach for system development was developed integrating YOLOv7 and MegaPose, both deep learning-based, as the models for object detection and 6D pose prediction respectively. The detector model’s and 6D pose predictor’s performance across diverse datasets and distances highlights its potential, though specific challenges suggest the need for further refinement. A methodology for implementing an intelligent 6D pose estimation system for robotic lawnmower operation is developed through the Iterative Development Process where the results demonstrate promising outcomes. Notably, the detector model, trained exclusively on synthetically generated data, exhibits robust performance in detecting charging stations. The subsequent 6D pose predictor, MegaPose, demonstrates robust performance during neutral robotic lawnmower operation and scenarios with artificial degradation such as occlusion and water on the camera lens. Challenges encountered in real-world scenarios, such as low camera height, occlusions at close distances, and unclear imagery underscore the need for future improvements. This study bridges theoretical concepts with real-world implementation, laying a foundation for further advancements. The findings provide valuable insights for the robotic lawnmower domain and potentially for other robotic systems and domains by introducing them to or improving their capabilities via advanced autonomous navigation and interaction.

Sorting second-hand clothing accurately and efficiently has a significant environmental benefit, as it reduces the need for manufacturing new garments. The EU has put forward a new directive stating that all member states must have a separate collection of textiles by January 2025. This will drastically increase the volume of second-hand garments, which demands effective sorting techniques. Sorting and evaluating clothing is a manual process today. In this thesis, a Faster R-CNN object detection model was trained to detect damages from images of second-hand garments. Given an image, the model predicts bounding boxes and classifies damages such as stains, holes, and other small damages. The model reached an mAP@.25 of 27.54 (0-100 scale) by leveraging data augmentation and transfer learning techniques. After removing unconfident and overlapping boxes it reached an F1 score of 0.26. Based on visual inspection, the model typically has a rationale for its predictions. The model sometimes falsely detects shadows and wrinkles as damages. Some false positives look very convincing and is even hard for the human eye to tell whether it a damage or not. Chi-square (X2) tests of independence were conducted to determine whether the model performs differently depending on the size and type of damage. The tests showed that with 99% confidence, the model performs better for damages with a box smaller than 322 pixels. It performs worse for stains than for holes and TUDs (Tiny Unidentified Damage). Model performance could potentially improve with a larger, more robust labeled dataset, and higher resolution images.

Point cloud registration is an extensively studied field in computer vision, featuring a variety of existing methods, all aimed at achieving the common objective of determining a transformation that aligns two point clouds. Methods like the Iterative Closet Point (ICP) and Fast Global Registration (FGR) have shown to work well for many years, but recent work has explored different learning-based approaches, showing promising results. This work compares the performance of two learning-based methods GeoTransformer and RegFormer against three baseline methods ICP point-to-point, ICP point-to-plane, and FGR. The comparison was conducted on data provided by the Swedish Defence Research Agency (FOI), where the data was captured with a photon-counting LiDAR sensor. Findings suggest that while ICP point-to-point and ICP point-to-plane exhibit solid performance, the GeoTransformer demonstrates the potential for superior outcomes. Additionally, the RegFormer and FGR perform worse than the ICP variants and the GeoTransformer.

This master’s thesis investigates the potential of enhancing cross-view localization (CVL) of UAV imagery using video input. The study, conducted in collaboration with Maxar Intelligence and Linköping University, aims to improve localization performance by leveraging sequential image data captured by UAVs and with satellite imagery as a reference. The thesis explores three main questions; the potential of modifying single-image CVL models to process image sequences, the adaptability of existing ground-to-satellite video CVL models for UAV-to-satellite localization, and the performance comparison of aerial views and satellite imagery extracted from Maxar’s 3D reconstructed models with public datasets. To address these questions, synthetic datasets mimicking University-1652 and SUES-200 were created using Maxar’s 3D data. The results demonstrated that while single-image models like Sample4Geo achieved high accuracy on synthetic datasets, their performance diminished with data that more closely represented the real world due to overfitting to the uniformity of synthetic environments. Existing ground-to-satellite video CVL models faced practical challenges when adapted to UAV perspectives, highlighting the need for tailored approaches. The synthetic datasets from Maxar outperformed public datasets but lacked the variability and realism of real-world conditions. Furthermore, the study introduced a novel multiple-image CVL model that leverages a post-processing technique using density-based clustering (OPTICS) to enhance localization accuracy by filtering noisy predictions and identifying high confidence clusters. The findings indicate that while this approach improves localization performance, significant advancements are required to fully utilize video input for UAV-to-satellite localization in dynamic environments.

Similar to how we humans use prior knowledge from one skill when learning new ones, multi-task networks can leverage similarities between different tasks to enhance learning speed. By acknowledging and leveraging common features across tasks, multi-task networks can be more efficient when learning and are able to generalize better. However, due to the technological advancements in deep learning, the modern architecture of such networks can be highly complex and vary significantly, so finding an optimal architecture is far from trivial. To handle this problem, we have utilized different methods to limit the search space and find the suitable architectural designs given a varying number of tasks. Through a comprehensive analysis of neural network structures and loss weighting strategies, the thesis contributes valuable insights into achieving a suitable trade-off between resource usage and task performance, paving the way for more effective and resource-conscious multi-task network designs.

This thesis investigates the application of knowledge transfer techniques to process event-based data forperson detection in area surveillance. A teacher-student model setup is employed, where both modelsare pretrained on conventional visual data. The teacher model processes visual images to generate targetlabels for the student model trained on event-based data, forming the baseline model. Building onthis, the project incorporates feature-based knowledge transfer, specifically transferring features fromthe Feature Pyramid Network (FPN) component of the Faster R-CNN ResNet-50 FPN network. Resultsindicate that response-based knowledge transfer can effectively finetune models for event-based data.However, feature-based knowledge transfer yields mixed results, requiring more refined techniques forconsistent improvement. The study identifies limitations, including the need for a more diverse dataset,improved preprocessing methods, labeling techniques, and refined feature-based knowledge transfermethods. This research bridges the gap between conventional object detection methods and event-baseddata, enhancing the applicability of event cameras in surveillance applications.

Semantic segmentation is the process of assigning a specific class label to each pixel in an image. There are multiple areas of use for semantic segmentation of remote sensing images, including climate change studies and urban planning and development. When training a network to perform semantic segmentation in a supervised manner, annotated data is crucial, and annotating satellite images is an expensive and time-consuming task. A resolution to this issue might be self-supervised learning. Training a pretext task on a large unlabeled dataset, and a downstream task on a smaller labeled dataset, could mitigate the need for large amounts of labeled data. In this thesis, the use of self-supervised learning for semantic segmentation of remote sensing data is investigated and compared to the traditional use of supervised pre-training using ImageNet. Two different methods of self-supervised learning are evaluated, a reconstructive method and a contrastive method. Furthermore, whether including modalities unique to remote sensing data yields greater performance for semantic segmentation is investigated. The findings indicate that self-supervised learning with in-domain data shows significant potential. While the performance of models pre-trained using self-supervised learning on remote sensing data, does not surpass that of pre-trained models using supervised learning on ImageNet, it achieves a comparable level. This is notable given the substantially smaller training data used. However, in cases where the in-domain dataset is small — as in this thesis with approximately 20,000 images — leveraging ImageNet for pre-training is preferable. Furthermore, self-supervised learning demonstrates promise as a more effective pre-training approach compared to supervised learning, when both methods are trained on ImageNet. The reconstructive method proves more suitable for semantic segmentation of remote sensing data compared to the contrastive method, and incorporating modalities uni

This thesis explores and evaluates solutions for the inspection of cutting tool containers at Sandvik Coromant, focusing on the transition from current vision systems utilizing infrared (IR) light to new methods compatible with recycled polypropylene (PP) plastic containers. The primary goal is to evaluate the effectiveness of stereo vision and artificial intelligence (AI) for depth estimation, ensuring that the containers are properly populated with cutting tools. Various methods and algorithms are tested to determine their accuracy and speed, to meet the time requirements of the production line at Sandvik Coromant. The results indicate that, while traditional IR-based systems excel in processing speed and robustness, monocular artificial intelligence methods offer adaptability that could be utilized with the new container material. Future work will involve further optimization and real-world testing to confirm these findings.

Gene editing techniques have been emerging rapidly through the years, and with this trend comes the great responsibility of making sure the edits are correct. One way to safeguard against mistakes in the edits is to measure gene editing efficiency. Countagen’s GeneAbacus does just that, it calculates the gene editing efficiency of CRISPR edits. A key aspect of the GeneAbacus workflow involves quantifying DNA microballs captured in fluorescence microscopy images. This thesis delves into novel image analysis pipelines aimed at optimizing this task. Six image processing techniques (Maximum Intensity Projection (MIP), white top hat transform, Contrast Limited Adaptive Histogram Equalisation (CLAHE), edge enhancement filter, Gaussian Blur, and unsharp masking) along with two object segmentation models (Segment Anything (SAM) and SAM for Microscopy (MicroSAM)) were implemented. They underwent evaluation in two stages: first, through an ablation study of the preprocessing techniques, and then by computing R2 values and log-log plot slopes on different datasets. The evaluation resulted in the selection of MicroSAM with white top hat transform, Gaussian blur and unsharp masking, yielding an average slope value of 0.698 and an average R2 value of 0.8724., Genredigeringstekniker har vuxit fram snabbt genom åren, och med denna trend följer det stora ansvaret att se till att redigeringarna är korrekta. Ett sätt att skydda sig mot misstag i redigeringarna är att mäta effektiviteten i genredigering. Countagens GeneAbacus gör just det, den beräknar genredigeringseffektiviteten för CRISPR-redigeringar. En nyckelaspekt av GeneAbacus arbetsflöde involverar kvantifiering av DNA-mikrobollar som fångats i fluorescensmikroskopibilder. Detta examensarbete fördjupar sig i nya bildanalyspipelines som syftar till att optimera denna uppgift. Sex bildbehandlingstekniker (Maximum Intensity Projection (MIP), white top hat transform, CLAHE, edge enhancement filter, Gaussian Blur och osharp maskning) tillsammans med två objektsegmenteringsmodeller (Segment Anything (SAM) och SAM for Microscopy (MicroSAM)) implementerades. De genomgick utvärdering i två steg: först genom en ablationsstudie av förbehandlingsteknikerna och sedan genom att beräkna R2 värden och log-log-plottlutningar på olika datamängder. Utvärderingen resulterade i valet av MicroSAM med en white top hat transform, Gaussian Blur och osharp maskning, vilket gav ett genomsnittligt lutningvärde på 0,698 och ett genomsnittligt värde på R2 på 0,8724.

In the field of robotics, Artificial Intelligence based on Machine Learning and Deep Learning is a key enabling technology for robot navigation, interaction and task execution. Despite significant advances in AI, there remain notable hurdles in deploying AI algorithms in real-time safety-critical systems such as robotic systems, to achieve high levels of safety assurance in the presence of stringent hardware resource constraints. For Deep Learning-based perception based on computer vision, adversarial patch attacks have emerged as a potent technique for fooling classifiers by placing a patch on the input image, and defence techniques against such attacks is an active research topic. In this thesis, we address two research questions: RQ1: How do adversarial patch defence algorithms perform on different hardware platforms with varying computing capabilities? RQ2: How do heuristics-based adversarial defence algorithms perform with increasing patch sizes? To address RQ1, this thesis measures and compares among six well-known adversarial patch defence algorithms, including 14 models, on three different hardware platforms. Their performance in accuracy and processing time are compared and trade-offs are presented. To address RQ2, this thesis measures and compares accuracy and timing performance of a representative heuristics-based algorithm with increasing patch sizes, and compares the performance of masking-alone mitigation and Generative Adversarial Network (GAN)-based mitigation. The research results of this thesis aim to serve as a useful reference for the practical deployment of adversarial patch defence algorithms in robotic systems.

Traditional cameras are most commonly used in visual SLAM to provide visual information about the scene and positional information about the camera motion. However, in the presence of varying illumination and rapid camera movement, the visual quality captured by traditional cameras diminishes. This limits the applicability of visual SLAM in challenging environments such as search and rescue situations. The emerging event camera has been shown to overcome the limitations of the traditional camera with the event camera's superior temporal resolution and wider dynamic range, opening up new areas of applications and research for event-based SLAM. In this thesis, several asynchronous feature detectors and trackers will be used to initialize SLAM using event camera data. To assess the pose estimation accuracy between the different feature detectors and trackers, the initialization performance was evaluated from datasets captured from various environments. Furthermore, two different methods to align corner-events were evaluated on the datasets to assess the difference. Results show that besides some slight variation in the number of accepted initializations, the alignment methods show no overall difference in any metric. Overall highest performance among the event-based trackers for initialization is HASTE with mostly high pose accuracy and a high number of accepted initializations. However, the performance degrades in featureless scenes. CET on the other hand shows mostly lower performance compared to HASTE.

Efficient management of agricultural resources de-mands a profound comprehension of plant growth dynamics, focusing on sustainable methodologies. This study delves into the forestry sector in Sweden, explicitly addressing the critical early phases of pine tree development within controlled en-vironments, predominantly nurseries and specialised facilities. To tackle the complexities of temporal classification in pine tree growth, we propose a novel measurement system that uses image data to leverage the power of different deep transfer learning-based convolutional neural networks. To classify pine trees over time, our approach integrates different state-of-the-art transfer learning models based on the features extracted by SqueezeNet, MobileNetV2, GoogLeNet, ShuffieNet, and ResNetXt into our measurement system. The primary challenge in tem-porally sorting plant growth is that instances within the same class exhibit variations, whereas instances belonging to different classes may share similarities. We also addressed a pivotal question: whether focusing on the region of interest (ROI) as the input improves the final results. To explore this, we defined two distinct scenarios and conducted a comparative analysis to elucidate the impact of varied input images on the efficacy of our measurement systems. Our experimental results highlight the superior performance of deep SqueezeNet transfer learning over other models, with an error reduction of 15% compared to the second-best approach. Additionally, we provide a sensitivity analysis of the hyperparameters.