Search

Advances in lightweight neural networks have revolutionized computer vision in a broad range of IoT applications, encompassing remote monitoring and process automation. However, the detection of small objects, which is crucial for many of these applications, remains an underexplored area in current computer vision research, particularly for low-power embedded devices that host resource-constrained processors. To address said gap, this paper proposes an adaptive tiling method for lightweight and energy-efficient object detection networks, including YOLO-based models and the popular FOMO network. The proposed tiling enables object detection on low-power MCUs with no compromise on accuracy compared to large-scale detection models. The benefit of the proposed method is demonstrated by applying it to FOMO and TinyissimoYOLO networks on a novel RISC-V-based MCU with built-in ML accelerators. Extensive experimental results show that the proposed tiling method boosts the F1-score by up to 225% for both FOMO and TinyissimoYOLO networks while reducing the average object count error by up to 76% with FOMO and up to 89% for TinyissimoYOLO. Furthermore, the findings of this work indicate that using a soft F1 loss over the popular binary cross-entropy loss can serve as an implicit non-maximum suppression for the FOMO network. To evaluate the real-world performance, the networks are deployed on the RISC-V based GAP9 microcontroller from GreenWaves Technologies, showcasing the proposed method's ability to strike a balance between detection performance ($58% - 95%$ F1 score), low latency (0.6 ms/Inference - 16.2 ms/Inference}), and energy efficiency (31 uJ/Inference} - 1.27 mJ/Inference) while performing multiple predictions using high-resolution images on a MCU., Comment: arXiv admin note: text overlap with arXiv:2311.07163

Electroencephalogram (EEG)-based Brain-Computer Interfaces (BCIs) have garnered significant interest across various domains, including rehabilitation and robotics. Despite advancements in neural network-based EEG decoding, maintaining performance across diverse user populations remains challenging due to feature distribution drift. This paper presents an effective approach to address this challenge by implementing a lightweight and efficient on-device learning engine for wearable motor imagery recognition. The proposed approach, applied to the well-established EEGNet architecture, enables real-time and accurate adaptation to EEG signals from unregistered users. Leveraging the newly released low-power parallel RISC-V-based processor, GAP9 from Greeenwaves, and the Physionet EEG Motor Imagery dataset, we demonstrate a remarkable accuracy gain of up to 7.31\% with respect to the baseline with a memory footprint of 15.6 KByte. Furthermore, by optimizing the input stream, we achieve enhanced real-time performance without compromising inference accuracy. Our tailored approach exhibits inference time of 14.9 ms and 0.76 mJ per single inference and 20 us and 0.83 uJ per single update during online training. These findings highlight the feasibility of our method for edge EEG devices as well as other battery-powered wearable AI systems suffering from subject-dependant feature distribution drift.

Recent advancements in propagation-based phase-contrast imaging, such as hierarchical imaging, have enabled the visualization of internal structures in large biological specimens and material samples. However, wavefront marker-based techniques, which provide quantitative electron density information, face challenges when imaging larger objects due to stringent beam stability requirements and potential structural changes in objects during longer measurements. Extending the fields-of-view of these methods is crucial for obtaining comparable quantitative results across beamlines and adapting to the smaller beam profiles of fourth-generation synchrotron sources. We introduce a novel technique combining an adapted eigenflat optimization with deformable image registration to address the challenges and enable quantitative high-resolution scans of centimeter-sized objects with micrometre resolution. We demonstrate the potential of the method by obtaining an electron density map of a rat brain sample 15 mm in diameter using speckle-based imaging, despite the limited horizontal field-of-view of 6 mm of the beamline (PETRA III, P05, operated by Hereon at DESY). This showcases the ability of the technique to significantly widen the range of application of wavefront marker-based techniques in both biological and materials science research., Comment: *The authors contributed equally to this work

Human Activity Recognition (HAR) is an attractive topic to perceive human behavior and supplying assistive services. Besides the classical inertial unit and vision-based HAR methods, new sensing technologies, such as ultrasound and body-area electric fields, have emerged in HAR to enhance user experience and accommodate new application scenarios. As those sensors are often paired with AI for HAR, they frequently encounter challenges due to limited training data compared to the more widely IMU or vision-based HAR solutions. Additionally, user-induced concept drift (UICD) is common in such HAR scenarios. UICD is characterized by deviations in the sample distribution of new users from that of the training participants, leading to deteriorated recognition performance. This paper proposes an on-device transfer learning (ODTL) scheme tailored for energy- and resource-constrained IoT edge devices. Optimized on-device training engines are developed for two representative MCU-level edge computing platforms: STM32F756ZG and GAP9. Based on this, we evaluated the ODTL benefits in three HAR scenarios: body capacitance-based gym activity recognition, QVAR- and ultrasonic-based hand gesture recognition. We demonstrated an improvement of 3.73%, 17.38%, and 3.70% in the activity recognition accuracy, respectively. Besides this, we observed that the RISC-V-based GAP9 achieves 20x and 280x less latency and power consumption than STM32F7 MCU during the ODTL deployment, demonstrating the advantages of employing the latest low-power parallel computing devices for edge tasks.

Event-based cameras, also called silicon retinas, potentially revolutionize computer vision by detecting and reporting significant changes in intensity asynchronous events, offering extended dynamic range, low latency, and low power consumption, enabling a wide range of applications from autonomous driving to longtime surveillance. As an emerging technology, there is a notable scarcity of publicly available datasets for event-based systems that also feature frame-based cameras, in order to exploit the benefits of both technologies. This work quantitatively evaluates a multi-modal camera setup for fusing high-resolution DVS data with RGB image data by static camera alignment. The proposed setup, which is intended for semi-automatic DVS data labeling, combines two recently released Prophesee EVK4 DVS cameras and one global shutter XIMEA MQ022CG-CM RGB camera. After alignment, state-of-the-art object detection or segmentation networks label the image data by mapping boundary boxes or labeled pixels directly to the aligned events. To facilitate this process, various time-based synchronization methods for DVS data are analyzed, and calibration accuracy, camera alignment, and lens impact are evaluated. Experimental results demonstrate the benefits of the proposed system: the best synchronization method yields an image calibration error of less than 0.90px and a pixel cross-correlation deviation of1.6px, while a lens with 8mm focal length enables detection of objects with size 30cm at a distance of 350m against homogeneous background.

Smart glasses are rapidly gaining advanced functionality thanks to cutting-edge computing technologies, accelerated hardware architectures, and tiny AI algorithms. Integrating AI into smart glasses featuring a small form factor and limited battery capacity is still challenging when targeting full-day usage for a satisfactory user experience. This paper illustrates the design and implementation of tiny machine-learning algorithms exploiting novel low-power processors to enable prolonged continuous operation in smart glasses. We explore the energy- and latency-efficient of smart glasses in the case of real-time object detection. To this goal, we designed a smart glasses prototype as a research platform featuring two microcontrollers, including a novel milliwatt-power RISC-V parallel processor with a hardware accelerator for visual AI, and a Bluetooth low-power module for communication. The smart glasses integrate power cycling mechanisms, including image and audio sensing interfaces. Furthermore, we developed a family of novel tiny deep-learning models based on YOLO with sub-million parameters customized for microcontroller-based inference dubbed TinyissimoYOLO v1.3, v5, and v8, aiming at benchmarking object detection with smart glasses for energy and latency. Evaluations on the prototype of the smart glasses demonstrate TinyissimoYOLO's 17ms inference latency and 1.59mJ energy consumption per inference while ensuring acceptable detection accuracy. Further evaluation reveals an end-to-end latency from image capturing to the algorithm's prediction of 56ms or equivalently 18 fps, with a total power consumption of 62.9mW, equivalent to a 9.3 hours of continuous run time on a 154mAh battery. These results outperform MCUNet (TinyNAS+TinyEngine), which runs a simpler task (image classification) at just 7.3 fps per second.

This paper deploys and explores variants of TinyissimoYOLO, a highly flexible and fully quantized ultra-lightweight object detection network designed for edge systems with a power envelope of a few milliwatts. With experimental measurements, we present a comprehensive characterization of the network's detection performance, exploring the impact of various parameters, including input resolution, number of object classes, and hidden layer adjustments. We deploy variants of TinyissimoYOLO on state-of-the-art ultra-low-power extreme edge platforms, presenting an in-depth a comparison on latency, energy efficiency, and their ability to efficiently parallelize the workload. In particular, the paper presents a comparison between a novel parallel RISC-V processor (GAP9 from Greenwaves) with and without use of its on-chip hardware accelerator, an ARM Cortex-M7 core (STM32H7 from ST Microelectronics), two ARM Cortex-M4 cores (STM32L4 from STM and Apollo4b from Ambiq), and a multi-core platform with a CNN hardware accelerator (Analog Devices MAX78000). Experimental results show that the GAP9's hardware accelerator achieves the lowest inference latency and energy at 2.12ms and 150uJ respectively, which is around 2x faster and 20% more efficient than the next best platform, the MAX78000. The hardware accelerator of GAP9 can even run an increased resolution version of TinyissimoYOLO with 112x112 pixels and 10 detection classes within 3.2ms, consuming 245uJ. To showcase the competitiveness of a versatile general-purpose system we also deployed and profiled a multi-core implementation on GAP9 at different operating points, achieving 11.3ms with the lowest-latency and 490uJ with the most energy-efficient configuration. With this paper, we demonstrate the suitability and flexibility of TinyissimoYOLO on state-of-the-art detection datasets for real-time ultra-low-power edge inference., Comment: * The first three authors contributed equally to this research

This paper introduces a highly flexible, quantized, memory-efficient, and ultra-lightweight object detection network, called TinyissimoYOLO. It aims to enable object detection on microcontrollers in the power domain of milliwatts, with less than 0.5MB memory available for storing convolutional neural network (CNN) weights. The proposed quantized network architecture with 422k parameters, enables real-time object detection on embedded microcontrollers, and it has been evaluated to exploit CNN accelerators. In particular, the proposed network has been deployed on the MAX78000 microcontroller achieving high frame-rate of up to 180fps and an ultra-low energy consumption of only 196{\mu}J per inference with an inference efficiency of more than 106 MAC/Cycle. TinyissimoYOLO can be trained for any multi-object detection. However, considering the small network size, adding object detection classes will increase the size and memory consumption of the network, thus object detection with up to 3 classes is demonstrated. Furthermore, the network is trained using quantization-aware training and deployed with 8-bit quantization on different microcontrollers, such as STM32H7A3, STM32L4R9, Apollo4b and on the MAX78000's CNN accelerator. Performance evaluations are presented in this paper., Comment: Published In: 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS)

In this paper, we introduce a novel class of projectors for 3D cone beam tomographic reconstruction. Analytical formulas are derived to compute the relationship between the volume of a voxel projected onto a detector pixel and its contribution to the line integral of attenuation recorded by that pixel. Based on these formulas, we construct a near-exact projector and backprojector, particularly suited for algebraic reconstruction techniques and hierarchical reconstruction approaches with nonuniform voxel grids. Unlike traditional projectors, which assume a uniform grid with fixed voxel sizes, our method enables local refinement of voxels, allowing for adaptive grid resolution and improved reconstruction quality in regions of interest. We have implemented this cutting voxel projector along with a relaxed, speed-optimized version and compared them to two established projectors: a ray-tracing projector based on Siddon's algorithm and a TT footprint projector. Our results demonstrate that the cutting voxel projector achieves higher accuracy than the TT projector, especially for large cone beam angles. Furthermore, the relaxed version of the cutting voxel projector offers a significant speed advantage over current footprint projector implementations, while maintaining comparable accuracy. In contrast, Siddon's algorithm, when achieving similar accuracy, is considerably slower than the cutting voxel projector. All algorithms are implemented in an open-source framework for algebraic reconstruction using OpenCL and C++, optimized for efficient GPU computation. GitHub repository of the project https://github.com/kulvait/KCT_cbct., Comment: 18 pages, 4 figures

Two-dimensional Talbot array illuminators (TAIs) were designed, fabricated, and evaluated for high-resolution high-contrast x-ray phase imaging of soft tissue at 10-20keV. The TAIs create intensity modulations with a high compression ratio on the micrometer scale at short propagation distances. Their performance was compared with various other wavefront markers in terms of period, visibility, flux efficiency and flexibility to be adapted for limited beam coherence and detector resolution. Differential x-ray phase contrast and dark-field imaging were demonstrated with a one-dimensional, linear phase stepping approach yielding two-dimensional phase sensitivity using Unified Modulated Pattern Analysis (UMPA) for phase retrieval. The method was employed for x-ray phase computed tomography reaching a resolution of 3$\mu$m on an unstained murine artery. It opens new possibilities for three-dimensional, non-destructive, and quantitative imaging of soft matter such as virtual histology. The phase modulators can also be used for various other x-ray applications such as dynamic phase imaging, super-resolution structured illumination microscopy, or wavefront sensing., Comment: 17 pages, 10 figures

Currently, most bone implants used in orthopedics and traumatology are non-degradable and may need to be surgically removed later on e.g. in the case of children. This removal is associated with health risks which could be minimized by using biodegradable implants. Therefore, research on magnesium-based implants is ongoing, which can be objectively quantified through synchrotron radiation microtomography and subsequent image analysis. In order to evaluate the suitability of these materials, e.g. their stability over time, accurate pixelwise segmentations of these high-resolution scans are necessary. The fully-convolutional U-Net architecture achieves a Dice coefficient of 0.750 +/- 0.102 when trained with a small dataset with dense expert annotations. However, extending the learning to larger databases would require prohibitive annotation efforts. Hence, in this work we implemented and compared new training methods that require only a small fraction of manually annotated pixels. While directly training on these scribble annotation deteriorates the segmentation quality by 26.8 percentage points, our new random walk-based semi-automatic target achieves the same Dice overlap as a dense supervision, and thus offers a more promising approach for sparse annotations., Comment: MIDL 2019 [arXiv:1907.08612] Extended Abstract

Spectra for the emission of charged leptons from evaporating hot-spots of preconfining phase in SU(2) Yang-Mills thermodynamics are computed. Specifically, we consider charged single and dileptons with their spectra being functions of energy and invariant mass, respectively. In the former case, our results relate to narrow and correlated electron and positron peaks measured in supercritical heavy-ion collisions performed at GSI in the 1980ies. In the latter case, we point out how strongly the spectra depend on typical kinematic cuts (CDF analysis of Tevatron Run II data). We also propose a scenario on how muon events of anomalously high multiplicity and large impact-parameter modulus arise in the Tevatron data., Comment: 13 pages, 2 figures

We consider spatial coarse-graining in statistical ensembles of non-selfintersecting and one-fold selfintersecting center-vortex loops as they emerge in the confining phase of SU(2) Yang-Mills thermodynamics. This coarse-graining is due to a noisy environment and described by a curve shrinking flow of center-vortex loops locally embedded in a two-dimensional flat plane. The renormalization-group flow of an effective `action', which is defined in purely geometric terms, is driven by the curve shrinking evolution. In the case of non-selfintersecting center-vortex loops, we observe critical behavior of the effective `action' as soon as the center-vortex loops vanish from the spectrum of the confining phase due to curve shrinking. This suggest the existence of an asymptotic mass gap. An entirely unexpected behavior in the ensemble of one-fold selfintersecting center-vortex loops is connected with the spontaneous emergence of order. We speculate that the physics of planar, one-fold selfintersecting center-vortex loops to be relevant for two-dimensional systems exhibiting high-temperature superconductivity., Comment: Diploma thesis, 60 pages,26 figures

We consider coarse-graining applied to nonselfintersecting planar center-vortex loops as they emerge in the confining phase of an SU(2) Yang-Mills theory. Well-established properties of planar curve-shrinking predict that a suitably defined, geometric effective action exhibits (mean-field) critical behavior when the conformal limit of circular points is reached. This suggests the existence of an asymptotic mass gap. We demonstrate that the initially sharp mean center-of-mass position in a given ensemble of curves develops a variance under the flow as is the case for a position eigenstate in free-particle quantum mechanics under unitary time evolution. A possible application of these concepts is an approach to high-$T_c$ superconductivity based (a) on the nonlocal nature of the electron (1-fold selfintersecting center-vortex loop) and (b) on planar curve-shrinking flow representing the decrease in thermal noise in a cooling cuprate., Comment: 15 pages, 8 figures