Search

Automated sensors have potential to standardize and expand the monitoring of insects across the globe. As one of the most scalable and fastest developing sensor technologies, we describe a framework for automated, image-based monitoring of nocturnal insects—from sensor development and field deployment to workflows for data processing and publishing. Sensors comprise a light to attract insects, a camera for collecting images and a computer for scheduling, data storage and processing. Metadata is important to describe sampling schedules that balance the capture of relevant ecological information against power and data storage limitations. Large data volumes of images from automated systems necessitate scalable and effective data processing. We describe computer vision approaches for the detection, tracking and classification of insects, including models built from existing aggregations of labelled insect images. Data from automated camera systems necessitate approaches that account for inherent biases. We advocate models that explicitly correct for bias in species occurrence or abundance estimates resulting from the imperfect detection of species or individuals present during sampling occasions. We propose ten priorities towards a step-change in automated monitoring of nocturnal insects, a vital task in the face of rapid biodiversity loss from global threats. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'. [ABSTRACT FROM AUTHOR]

Due to rapid technological innovations, the automated monitoring of insect assemblages comes within reach. However, this continuous innovation endangers the methodological continuity needed for calculating reliable biodiversity trends in the future. Maintaining methodological continuity over prolonged periods of time is not trivial, since technology improves, reference libraries grow and both the hard- and software used now may no longer be available in the future. Moreover, because data on many species are collected at the same time, there will be no simple way of calibrating the outputs of old and new devices. To ensure that reliable long-term biodiversity trends can be calculated using the collected data, I make four recommendations: (1) Construct devices to last for decades, and have a five-year overlap period when devices are replaced. (2) Construct new devices to resemble the old ones, especially when some kind of attractant (e.g. light) is used. Keep extremely detailed metadata on collection, detection and identification methods, including attractants, to enable this. (3) Store the raw data (sounds, images, DNA extracts, radar/lidar detections) for future reprocessing with updated classification systems. (4) Enable forward and backward compatibility of the processed data, for example by in-silico data 'degradation' to match the older data quality. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'. [ABSTRACT FROM AUTHOR]

None of the global targets for protecting nature are currently met, although humanity is critically dependent on biodiversity. A significant issue is the lack of data for most biodiverse regions of the planet where the use of frugal methods for biomonitoring would be particularly important because the available funding for monitoring is insufficient, especially in low-income countries. We here discuss how three approaches to insect biomonitoring (computer vision, lidar, DNA sequences) could be made more frugal and urge that all biomonitoring techniques should be evaluated for global suitability before becoming the default in high-income countries. This requires that techniques popular in high-income countries should undergo a phase of 'innovation through simplification' before they are implemented more broadly. We predict that techniques that acquire raw data at low cost and are suitable for analysis with AI (e.g. images, lidar-signals) will be particularly suitable for global biomonitoring, while techniques that rely heavily on patented technologies may be less promising (e.g. DNA sequences). We conclude the opinion piece by pointing out that the widespread use of AI for data analysis will require a global strategy for providing the necessary computational resources and training. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'. [ABSTRACT FROM AUTHOR]

Insects are the most diverse group of animals on Earth, yet our knowledge of their diversity, ecology and population trends remains abysmally poor. Four major technological approaches are coming to fruition for use in insect monitoring and ecological research—molecular methods, computer vision, autonomous acoustic monitoring and radar-based remote sensing—each of which has seen major advances over the past years. Together, they have the potential to revolutionize insect ecology, and to make all-taxa, fine-grained insect monitoring feasible across the globe. So far, advances within and among technologies have largely taken place in isolation, and parallel efforts among projects have led to redundancy and a methodological sprawl; yet, given the commonalities in their goals and approaches, increased collaboration among projects and integration across technologies could provide unprecedented improvements in taxonomic and spatio-temporal resolution and coverage. This theme issue showcases recent developments and state-of-the-art applications of these technologies, and outlines the way forward regarding data processing, cost-effectiveness, meaningful trend analysis, technological integration and open data requirements. Together, these papers set the stage for the future of automated insect monitoring. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'. [ABSTRACT FROM AUTHOR]

Multi-Object Tracking, also known as Multi-Target Tracking, is an important area of computer vision with various applications in different domains. The advent of deep learning has had a profound impact on this field, forcing researchers to explore innovative avenues. Deep learning methods have become the cornerstone of today's state-of-the-art solutions, consistently delivering exceptional tracking results. However, the significant computational demands of deep learning models require powerful hardware resources that do not always match real-time tracking requirements, limiting their practical applicability in real-world scenarios. Thus, there is an imperative to strike a balance by merging robust deep learning strategies with conventional approaches to enable more accessible, cost-effective solutions that meet real-time requirements. This paper embarks on this endeavor by presenting a hybrid strategy for real-time multi-target tracking. It effectively combines a classical optical flow algorithm with a deep learning architecture tailored for human crowd tracking systems. This hybrid approach achieves a commendable balance between tracking accuracy and computational efficiency. The proposed architecture, subjected to extensive experimentation in various settings, demonstrated notable results, achieving a Mean Object Tracking Accuracy (MOTA) of 0.608. This level of performance placed it as the highest ranking solution on the MOT15 benchmark, surpassing the state-of-the-art benchmark of 0.549, and consistently ranked among the superior models on the MOT17 and MOT20 benchmarks. Additionally, the incorporation of the optical flow phase resulted in a substantial reduction in processing time, nearly halving the duration, while simultaneously maintaining accuracy levels comparable to established techniques. [ABSTRACT FROM AUTHOR]

Once realized, autonomous aerial refueling will revolutionize unmanned aviation by removing current range and endurance limitations. Previous attempts at establishing vision-based solutions have come close but rely heavily on near perfect extrinsic camera calibrations that often change midflight. In this paper, we propose dual object detection, a technique that overcomes such requirement by transforming aerial refueling imagery directly into receiver aircraft reference frame probe-to-drogue vectors regardless of camera position and orientation. These vectors are precisely what autonomous agents need to successfully maneuver the tanker and receiver aircraft in synchronous flight during refueling operations. Our method follows a common 4-stage process of capturing an image, finding 2D points in the image, matching those points to 3D object features, and analytically solving for the object pose. However, we extend this pipeline by simultaneously performing these operations across two objects instead of one using machine learning and add a fifth stage that transforms the two pose estimates into a relative vector. Furthermore, we propose a novel supervised learning method using bounding box corrections such that our trained artificial neural networks can accurately predict 2D image points corresponding to known 3D object points. Simulation results show that this method is reliable, accurate (within 3 cm at contact), and fast (45.5 fps). [ABSTRACT FROM AUTHOR]

Electrical equivalent circuits are a widely applied tool with which electrical processes can be rationalized. There is a wide‐ranging selection of fields from bioelectrochemistry to batteries to fuel cells making use of this tool. Enabling meta‐analysis on the similarities and differences in the used circuits will help to identify commonly used circuits and aid in evaluating the underlying physics. We present a method and an implementation that enables the conversion of circuits included in scientific publications into a machine‐readable form for generating machine learning datasets or circuit simulations. [ABSTRACT FROM AUTHOR]

Opto-neuromorphic operation is critical for biological system to recognize the visual objects and mimicking such operation is important for artificial prosthesis as well as machine vision system for industrial applications. To sophisticatedly mimic biological system, regulation of learning and memorizing efficiency is needed, however engineered synthetic platform has been lack of controllability, which makes huge gap between biological system and synthetic platform. Here we demonstrated controllable learning and memorizing opto-neuromorphic operation at plasmonic hot electron transistor. Especially, circularly polarized light (CPL) sensitive synaptic characteristics and learning experience capability are enabled by incorporating chiral plasmonic nanoparticle. Furthermore, gate voltage gives rise to controllable neuromorphic operation due to hot electron injection and trapping effect, resulting in high remaining synaptic weight of ∼70% at negative gate voltage under CPL excitation. We believe that this discovery makes significant leap toward on-demand in-sensor computing as well as toward bio-realistic device. [ABSTRACT FROM AUTHOR]

Medical image segmentation is a crucial field of computer vision. Obtaining correct pathological areas can help clinicians analyze patient conditions more precisely. We have observed that both CNN-based and attention-based neural networks often produce rough segmentation results around the edges of the regions of interest. This significantly impacts the accuracy of obtaining the pathological areas. Without altering the original data and model architecture, further refining the initial segmentation outcomes can effectively address this issue and lead to more satisfactory results. Recently, diffusion models have demonstrated outstanding results in image generation, showcasing their powerful ability to model distributions. We believe that this ability can greatly enhance the accuracy of the reshaping results. This research proposes ERSegDiff, a neural network based on the diffusion model for reshaping segmentation borders. The diffusion model is trained to fit the distribution of the target edge area and is then used to modify the segmentation edge to produce more accurate segmentation results. By incorporating prior knowledge into the diffusion model, we can help it more accurately simulate the edge probability distribution of the samples. Moreover, we introduce the edge concern module, which leverages attention mechanisms to produce feature weights and further refine the segmentation outcomes. To validate our approach, we employed the COVID-19 and ISIC-2018 datasets for lung segmentation and skin cancer segmentation tasks, respectively. Compared with the baseline model, ERSegDiff improved the dice score by 3%–4% and 2%–4%, respectively, and achieved state-of-the-art scores compared to several mainstream neural networks, such as swinUNETR. [ABSTRACT FROM AUTHOR]

Soft-shell crab production is a labor-intensive process that necessitates constant inspections by aquacultural farmers to determine the completion of each crab's molting. The need for continuous visual inspections throughout the harvest days is imperative to prevent the new shell from hardening, a process that commences within 3 h after molting. This study proposes a vision-based deep learning approach to automate this repetitive visual inspection task, specifically targeting the challenge of locating and detecting crabs concealed within box baskets, which obstruct the camera's view. Initially, we explored the well-established technique of image classification through transfer learning and fine-tuning a pre-trained VGG-16 model, achieving an accuracy of 99%. However, our investigation using Gradient-weighted Class Activation Mapping (Grad-CAM) uncovered the sensitivity of this mature classification method to image bias, leading to the inadvertent learning of false features. In pursuit of a more robust alternative, we turned to object detection techniques, particularly Faster R-CNN, which proved to be better suited to our problem and dataset. Through experimentation with various backbones and bounding box confidence thresholds, our resulting model demonstrated the capability to detect crabs with an average precision (AP 0.5) ranging from 83 to 91%. By translating crab counting results into image classification, the accuracy remained consistently high at 98–99%. This innovative approach offers a promising solution to streamline soft-shell crab production without compromising accuracy. [ABSTRACT FROM AUTHOR]

Background: Application of artificial intelligence (AI) in general surgery is evolving. Real-world implementation of an AI-based computer-vision model in laparoscopic appendectomy (LA) is presented. We aimed to evaluate (1) its accuracy in complexity grading and safety adherence, (2) clinical correlation to outcomes. Methods: A retrospective single-center study of 499 consecutive LA videos, captured and analyzed by 'Surgical Intelligence Platform,' Theator Inc. (9/2020–5/2022). Two expert surgeons viewed all videos and manually graded complexity and safety adherence. Automated annotations were compared to surgeons' assessments. Inter-surgeons' agreements were measured. Since 7/2021 videos were linked to patients' admission numbers. Data retrieval from medical records was performed (n = 365). Outcomes were compared between high and low complexity grades. Results: Low and high complexity grades comprised 74.8 and 25.2% of 499 videos. Surgeons' agreements were high (76.9–94.4%, kappa 0.77/0.91; p < 0.001) for all annotated complexity grades. Surgeons' agreements were also high (96.0–99.8%, kappa 0.78/0.87; p < 0.001) for full safety adherence, whereas agreement was moderate in partial safety adherence and none (32.8–58.8%). Inter-surgeons' agreements were high for complexity grading (kappa 0.86, p < 0.001) and safety adherence (kappa 0.88, p < 0.001). Comparing high to low grade complexity, preoperative clinical features were similar, except larger appendix diameter on imaging (13.4 ± 4.4 vs. 10.5 ± 3.0 mm, p < 0.001). Intraoperative outcomes were significantly higher (p < 0.001), including time to achieve critical view of safety (29.6, IQR 19.1–41.6 vs. 13.7, IQR 8.5–21.1 min), operative duration (45.3, IQR 37.7–65.2 vs. 25.0, IQR 18.3–32.7 min), and intraoperative events (39.4% vs. 5.9%). Postoperative outcomes (7.4% vs. 9.2%) including surgical complications, mortality, and readmissions were comparable (p = 0.6), except length of stay (4, IQR 2–5.5 vs. 1, IQR 1-2 days; p < 0.001). Conclusion: The model accurately assesses complexity grading and full safety achievement. It can serve to predict operative time and intraoperative course, whereas no clinical correlation was found regarding postoperative outcomes. Further studies are needed. [ABSTRACT FROM AUTHOR]

With the increasing popularity of machine learning (ML) in image processing, privacy concerns have emerged as a significant issue in deploying and using ML services. However, current privacy protection approaches often require computationally expensive training from scratch or extensive fine-tuning of models, posing significant barriers to the development of privacy-conscious models, particularly for smaller organizations seeking to comply with data privacy laws. In this paper, we address the privacy challenges in computer vision by investigating the effectiveness of two recent fine-tuning methods, Model Reprogramming and Low-Rank Adaptation. We adapt these techniques to provide attribute protection for pre-trained models, minimizing computational overhead and training time. Specifically, we modify the models to produce privacy-preserving latent representations of images that cannot be used to identify unintended attributes. We integrate these methods into an adversarial min–max framework, allowing us to conceal sensitive information from feature outputs without extensive modifications to the pre-trained model, but rather focusing on a small set of new parameters. We demonstrate the effectiveness of our methods by conducting experiments on the CelebA dataset, achieving state-of-the-art performance while significantly reducing computational complexity and cost. Our research provides a valuable contribution to the field of computer vision and privacy, offering practical solutions to enhance the privacy of machine learning services without compromising efficiency. [ABSTRACT FROM AUTHOR]

The use of street view imagery (SVI) and advanced urban visual intelligence technologies has revolutionized built environment auditing (BEA) practice, by enabling high-resolution BEA at large scales. This study reviewed 96 studies of BEA published before October 2023. The Google SVI was employed in 92.7% of the included studies. Manual processing of SVI was used in BEA in most studies (81.3%), while deep learning methods were mostly used in the remaining studies. Validated auditing tools were used in 71% of the studies. Streets were the most frequently audited objects (54.2%), followed by sidewalk (51%), traffic (49%), and land use (34.4%). Objective attributes exhibited higher reliability in BEA, compared to subjective attributes (e.g. neighborhood environmental perception). The Active Neighborhood Checklist and Microscale Audit of Pedestrian Streetscapes were the two most widely used SVI-based BEA tools. Several key areas for improving the accuracy and reliability of SVI-based BEA were identified: building standardized datasets of built environment features for more accurate auditing, combining multi-source SVI for more comprehensive assessments, and adapting auditing tools to the contexts in developing countries. This study would contribute to a deeper understanding of built environmental influences on health, and facilitate informed decision-making in urban planning and public health efforts. [ABSTRACT FROM AUTHOR]

Current traffic simulation approaches analyze vehicle loads and load effects from a statistical perspective; however, they fail to reproduce the spatiotemporal distribution of bridge loads and resultant load effects at every moment, hindering real‐time bridge health management. This paper proposes a two‐step hybrid virtual–real traffic simulation (HvrTS) approach to reproduce the spatiotemporal distribution of bridge loads. Vehicle load sequences are first identified using computer vision based on traffic video from two surveillance cameras installed along the bridge. Next, traffic microsimulation is optimized with the identified vehicle load sequences from the two cameras serving as the known input and validated output. Using a cable‐stayed bridge under free‐flowing traffic, the HvrTS approach achieved a weighted mean square matching error of ≤0.3 m for the vehicle longitudinal location and a matching error of ≤7% for the vehicle lane position, whereas with congested traffic, the matching errors were much higher due to the inherent complexities and challenges associated with reproducing vehicle behavior in heavily congested situations. The traffic load effects calculated via HvrTS presented excellent spatiotemporal matching with those measured by a structural health monitoring system, especially in free‐flow traffic conditions. Applications on a continuous rigid frame girder bridge further validate these findings. Hence, the proposed HvrTS approach can overcome the challenge of spatiotemporal matching between vehicle loads and load effects in field monitoring. [ABSTRACT FROM AUTHOR]

In mining and construction, excavators are integral to earth-moving operations. Accurate knowledge of excavator activities may be used in productivity analysis to streamline delivery. This paper presents a computer vision-based method for excavator action detection which can automatically inference the occurrence and time duration of excavator actions from untrimmed video captured from the excavator cab. The model uses a three-stage architecture consisting of a VGG16 feature extractor, a four-stage Transformer Encoder-Long Short-Term Memory (LSTM) module, and a post-processing component. The model's predictive performance has been validated on the largest dataset among similar studies, comprising 567,000 frames filmed on-site at day and night. When tested on night and daytime videos, the model achieves accuracies of 90% and 70%, respectively, highlighting strong potential for practical implementation of the Transformer-LSTM network in excavator action detection. This study presents the first application of the combined Transformer-LSTM network for action detection in computer vision. [ABSTRACT FROM AUTHOR]

High-speed video recordings of slab burner experiments were analyzed using a machine learning approach with convolutional neural networks in order to compute the regression rate of hybrid rocket fuels over time. Combustion tests of paraffin-based fuel grains performed in two different hybrid rocket slab burners were recorded with high-speed video cameras and the resulting image data are analyzed in order to determine the height of the fuel in each frame. To this end, a deep neural network with U-net architecture is trained in a supervised fashion to segment the shape of the fuel slab. It is demonstrated that this approach is more capable to segment combustion images in unsteady flow conditions than classical computer vision methods based on thresholding or edge detection. Furthermore, methods in the area of uncertainty quantification of neural networks are applied to estimate the errors in the neural network prediction to new previously unseen data. Finally, the regression rate of the fuel is computed as the rate of change of this height. This method enables automatic analysis of a large amount of video data, taking full advantage of the optical access capabilities of slab burners. Additionally, the method delivers not only the time and space average values of the fuel regression rate, but also quantifies its variation over time and over the length of the slab, providing deeper insights into the combustion mechanics of hybrid rockets. • Shape of solid fuel detected in images from combustion test using a U-Net. • U-Net applied to a high-speed video enables measurement of the regression rate. • Regression rate can be resolved as time- and space-dependent quantity. • Method is applied to combustion tests performed at two different test facilities. • Tests under both atmospheric and super-critical pressure conditions were evaluated. [ABSTRACT FROM AUTHOR]

This study compares the performance of Transformers with LSTM for the classification of the behavioural time budget in horses based on video data. The behavioural time budget of a horse consists of amount of time of the activities such as feeding, resting, lying, and moving, which are important indicators of welfare and can be a basis of pain detection. Video technology offers a non-invasive and continuous monitoring approach for automated detection of horse behaviours. Computer vision and deep learning methods have been used for automated monitoring of animal behaviours, but accurate behaviour recognition remains a challenge. Previous studies have employed Convolutional LSTM models for behaviour classification, and more recently, Transformer-based models have shown superior performance in various tasks. This study proposes a multi-input, multi-output classification methodology to address the challenges of accurately detecting and classifying horse behaviours. The results demonstrate that the multi-input and multi-output Transformer model achieves the best performance in behaviour classification compared with single input and single output strategy. The proposed methodology provides a basis for detecting changes in behaviour time budgets related to pain and discomfort in horses, which can be valuable for monitoring and treating horse health problems. • Changes in behaviour time budgets has potential to detect health issues in horses. • A comparison of LSTM and Transformer models for equine behaviour classification. • A novel multi-input, multi-output methodology for equine behaviour classification. [ABSTRACT FROM AUTHOR]

Summary: Wide variation in amenability to transformation and regeneration (TR) among many plant species and genotypes presents a challenge to the use of genetic engineering in research and breeding. To help understand the causes of this variation, we performed association mapping and network analysis using a population of 1204 wild trees of Populus trichocarpa (black cottonwood).To enable precise and high‐throughput phenotyping of callus and shoot TR, we developed a computer vision system that cross‐referenced complementary red, green, and blue (RGB) and fluorescent‐hyperspectral images. We performed association mapping using single‐marker and combined variant methods, followed by statistical tests for epistasis and integration of published multi‐omic datasets to identify likely regulatory hubs.We report 409 candidate genes implicated by associations within 5 kb of coding sequences, and epistasis tests implicated 81 of these candidate genes as regulators of one another. Gene ontology terms related to protein–protein interactions and transcriptional regulation are overrepresented, among others.In addition to auxin and cytokinin pathways long established as critical to TR, our results highlight the importance of stress and wounding pathways. Potential regulatory hubs of signaling within and across these pathways include GROWTH REGULATORY FACTOR 1 (GRF1), PHOSPHATIDYLINOSITOL 4‐KINASE β1 (PI‐4Kβ1), and OBF‐BINDING PROTEIN 1 (OBP1). [ABSTRACT FROM AUTHOR]

Dentistry frequently makes use of intraoral scanning technologies to digitally acquire the three‐dimensional (3D) geometry of teeth. In recent times, dental clinics over the globe utilize used computer aided diagnosis (CAD) models to make treatment plans, for example, orthodontics. Orthodontic CAD system acts as a vital part of the advanced dentistry field. A 3D dental model, computed by patient impression, as input and aids dentist in the extraction, moving, deletion, and rearranging of teeth to simulate treatment output. Tooth segmentation and labelling is the basic and foremost element of the CAD model which needs to be addressed. Automated segmentation and classification of 3D dental images using advanced machine learning and deep learning (DL) models become essential. This article introduces a new 3D dental image segmentation and classification using DL with tunicate swarm algorithm (3DDISC‐DLTSA) model. The major intention of the 3DDISC‐DLTSA system is to segment the tooth model and identify seven distinct tooth types. To accomplish this, the presented 3DDISC‐DLTSA model performs image pre‐processing in two stages namely image filtering and U‐Net segmentation. In addition, the 3DDISC‐DLTSA model derives DenseNet‐169 model for feature extraction purposes. For the recognition and classification of tooth type, the TSA based hyperparameter tuning process is carried out which helps to accomplish maximum classification performance. A wide range of experimental analyses is performed and the outcomes are inspected under many aspects. On dataset‐1, 3DDISC‐DLTSA model accuracy rose by 96.67%. On dataset‐3, 3DDISC‐DLTSA model accuracy rose by 97.48% and algorithm accuracy by 97.35%. The 3DDISC‐DLTSA model outperformed more modern models, according to the comparative investigation. [ABSTRACT FROM AUTHOR]

This work presents Spacecraft Pose Network v2 (SPNv2), a Convolutional Neural Network (CNN) for pose estimation of noncooperative spacecraft across domain gap. SPNv2 is a multi-scale, multi-task CNN which consists of a shared multi-scale feature encoder and multiple prediction heads that perform different tasks on a shared feature output. These tasks are all related to detection and pose estimation of a target spacecraft from an image, such as prediction of pre-defined satellite keypoints, direct pose regression, and binary segmentation of the satellite foreground. It is shown that by jointly training on different yet related tasks with extensive data augmentations on synthetic images only, the shared encoder learns features that are common across image domains that have fundamentally different visual characteristics compared to synthetic images. This work also introduces Online Domain Refinement (ODR) which refines the parameters of the normalization layers of SPNv2 on the target domain images online at deployment. Specifically, ODR performs self-supervised entropy minimization of the predicted satellite foreground, thereby improving the CNN's performance on the target domain images without their pose labels and with minimal computational efforts. The GitHub repository for SPNv2 is available at https://github.com/tpark94/spnv2. [ABSTRACT FROM AUTHOR]

Accurate and efficient segmenting of vertebral bodies, muscles, and discs is crucial for analyzing various spinal diseases. However, traditional methods are either laborious and time-consuming (manual segmentation) or require extensive training data (fully automatic segmentation). FastCleverSeg, our proposed semi-automatic segmentation approach, addresses those limitations by significantly reducing user interaction while maintaining high accuracy. First, we reduce user interaction by requiring the manual annotation of only two or three slices. Next, we automatically Estimate the Annotation on Intermediary Slices (EANIS) using traditional computer vision/graphics concepts. Finally, our proposed method leverages improved voxel weight balancing to achieve fast and precise volumetric segmentation in the segmentation process. Experimental evaluations on our assembled diverse MRI databases comprising 179 patients (60 male, 119 female), demonstrate a remarkable 25 ms (30 ms standard deviation) processing time and a significant reduction in user interaction compared to existing approaches. Importantly, FastCleverSeg maintains or surpasses the segmentation quality of competing methods, achieving a Dice score of 94%. This invaluable tool empowers physicians to efficiently generate reliable ground truths, expediting the segmentation process and paving the way for future integration with deep learning approaches. In turn, this opens exciting possibilities for future fully automated spine segmentation. [ABSTRACT FROM AUTHOR]

The classification of hyperspectral images (HSI) into categories that correlate to various land cover sorts such as water bodies, agriculture and urban areas, has gained significant attention in research due to its wide range of applications in fields, such as remote sensing, computer vision, and more. Supervised deep learning networks have demonstrated exceptional performance in HSI classification, capitalizing on their capacity for end-to-end optimization and leveraging their strong potential for nonlinear modeling. However, labelling HSIs, on the other hand, necessitates extensive domain knowledge and is a time-consuming and labour-intensive exercise. To address this issue, the proposed work introduces a novel semi-supervised network constructed with an autoencoder, Siamese action, and attention layers that achieves excellent classification accuracy with labelled limited samples. The proposed convolutional autoencoder is trained using the mass amount of unlabelled data to learn the refinement representation referred to as 3D-CAE. The added Siamese network improves the feature separability between different categories and attention layers improve classification by focusing on discriminative information and neglecting the unimportant bands. The efficacy of the proposed model's performance was assessed by training and testing on both same-domain as well as cross-domain data and found to achieve 91.3 and 93.6 for Indian Pines and Salinas, respectively. [ABSTRACT FROM AUTHOR]

Transportation networks frequently employ hub-and-spoke network architectures to route flows between many origin and destination pairs. Hub facilities work as switching points for flows in large networks. In this study, we focus on a problem, called the single allocation hub-and-spoke network design problem. In the problem, the goal is to allocate each non-hub node to exactly one of the given hub nodes so as to minimize the total transportation cost. The problem is essentially equivalent to another combinatorial optimization problem, called the metric labeling problem. The metric labeling problem was first introduced by Kleinberg and Tardos in 2002, motivated by application to segmentation problems in computer vision and related areas. In this study, we deal with a case where the set of hubs forms a star, which is encountered especially in telecommunication networks. We propose a polynomial-time randomized approximation algorithm for the problem, whose approximation ratio is less than 5.281. Our algorithms solve a linear relaxation problem and apply dependent rounding procedures. [ABSTRACT FROM AUTHOR]

Human visual recognition is remarkably robust to chromatic changes. In this work, we provide a potential account of the roots of this resilience based on observations with 10 congenitally blind children who gained sight late in life. Several months or years following their sight-restoring surgeries, the removal of color cues markedly reduced their recognition performance, whereas age-matched normally sighted children showed no such decrement. This finding may be explained by the greater-than-neonatal maturity of the late-sighted childrenÕs color system at sight onset, inducing overly strong reliance on chromatic cues. Simulations with deep neural networks corroborate this hypothesis. These findings highlight the adaptive significance of typical developmental trajectories and provide guidelines for enhancing machine vision systems. [ABSTRACT FROM AUTHOR]

Visualization of sports has a lot of potential for future development in data sports because of how quickly things are changing and how much sports depend on data. Presently, conventional systems fail to accurately address sports persons’ dynamic health data change with less error rate. Further, those systems are unable to distinguish players’ health data and their visualization in a precise manner. An excellent starting point for building fitness solutions based on computer vision technology is the data visualization technology that arose in the age of big data analytics. This research presents a Big Data Analytic assisted Computer Vision Model (BD-CVM) for effective sports persons healthcare data management with improved accuracy and precision. The fitness and health of professional athletes are analyzed using information from a publicly available sports visualization dataset. Machine learning-assisted computer vision dynamic algorithm has been used for an effective image featuring and classification by categorizing sports videos through temporal and geographical data. The significance of big data’s great potential in screening data during a sporting event can be reasonably analyzed and processed effectively with less error rate. The proposed BD-CVM utilized an error analysis module which can be embedded in the design further to ensure the accuracy requirements in the data processing from sports videos. The research findings of this paper demonstrate that the strategy presented here can potentially improve accuracy and precision and optimize mean square error in sports data classification and visualization. [ABSTRACT FROM AUTHOR]

In the field of traffic management and control systems, we are witnessing a symbiotic evolution, where intelligent infrastructure is progressively collaborating with smart vehicles to produce benefits for traffic monitoring and security, by rapidly identifying hazardous behaviours. This exponential growth is due to the rapid development of deep learning in recent years, as well as the improvements in computer vision models. These technologies allow for monitoring tasks without the need to install numerous sensors or stop the traffic, using the extensive camera network of surveillance cameras already present in worldwide roads. This study proposes a computer vision-based solution that allows for real-time processing of video streams through edge computing devices, eliminating the need for Internet connectivity or dedicated sensors. The proposed system employs deep learning algorithms and vision techniques that perform vehicle detection, classification, tracking, speed estimation, and vehicle geolocation. [ABSTRACT FROM AUTHOR]

Responses to natural stimuli in area V4—a mid-level area of the visual ventral stream—are well predicted by features from convolutional neural networks (CNNs) trained on image classification. This result has been taken as evidence for the functional role of V4 in object classification. However, we currently do not know if and to what extent V4 plays a role in solving other computational objectives. Here, we investigated normative accounts of V4 (and V1 for comparison) by predicting macaque single-neuron responses to natural images from the representations extracted by 23 CNNs trained on different computer vision tasks including semantic, geometric, 2D, and 3D types of tasks. We found that V4 was best predicted by semantic classification features and exhibited high task selectivity, while the choice of task was less consequential to V1 performance. Consistent with traditional characterizations of V4 function that show its high-dimensional tuning to various 2D and 3D stimulus directions, we found that diverse non-semantic tasks explained aspects of V4 function that are not captured by individual semantic tasks. Nevertheless, jointly considering the features of a pair of semantic classification tasks was sufficient to yield one of our top V4 models, solidifying V4's main functional role in semantic processing and suggesting that V4's selectivity to 2D or 3D stimulus properties found by electrophysiologists can result from semantic functional goals. Author summary: The functional role of area V4 in the primate visual cortex has been traditionally studied by measuring tuning properties to simple, parametric stimuli, potentially overlooking other important aspects that could be revealed with richer, natural stimuli. Here, we combine single-cell recordings of macaque V1 and V4 responses to natural images, and deep learning models trained on multiple computer vision tasks. We found that V4 responses are best predicted by representations that are critical to solve semantic tasks like object and scene classification. Moreover, our results suggest that that V4's affinity to different 2D and 3D stimulus properties likely stems from its involvement in semantic processing. Overall, our diverse task-driven modeling approach enriches our understanding of the functional role of visual areas in the brain. [ABSTRACT FROM AUTHOR]

The unsatisfactory fatigue performance of Al foams significantly limits their engineering applications. Meanwhile, the unique pore structure of metal foams makes it still challenging to assess its fatigue. Herein, we report an innovative strategy for preparing carbon nanotubes reinforced Al‐Cu‐Mg (CNTs/Al‐Cu‐Mg) composite foams, which exhibit superior compressive strength (16.17 MPa) and fatigue strength (11.17 MPa). Statistics analysis confirms that the fatigue life (Nf) follows Weibull distribution, and P‐S‐N curves were obtained to calculate the theoretical stress level required to achieve the expected Nf. Moreover, computer vision is utilized to monitor the fatigue deformation process and effectively detect fatigue crack initiation sites. Notably, the accuracy of predicted Nf obtained by this method is exceptionally high (98.32%). This work provides a theoretical and practical basis for designing metal foams with superior fatigue properties and offers a promising bottom‐up tactic for predicting the Nf of metal foams to prevent fatigue failure. [ABSTRACT FROM AUTHOR]

An analytical review of existing methods for positioning by fiducial markers using technical vision is presented in the article. Their advantages and disadvantages are shown. A new label based on standard ArUco with additional elements and effective algorithms for calculating the coordinates of objects in space obtained from a group of fiducial labels with different orientations and positions is proposed. This allows one to reduce the average positioning error. It was shown using a group of synthetic tests and confirmed by a full-scale experiment using the KUKA KR3 R540 robotic manipulator, which allows one to accurately change the orientation of the marker template and automate the photography process. In comparison with standard methods for determining the position of objects in space (based on generally accepted ArUco marks), the new algorithm together with the proposed marker can reduce the average error in determining positions in space by 35–67% when using markers with a side of 32 mm. [ABSTRACT FROM AUTHOR]

This study aims to enhance the safety and efficiency of port navigation by reducing ship collision accidents, minimizing environmental risks, and optimizing waterways to increase port throughput. Initially, a three-dimensional map of the port's waterway, including data on water depth, rocks, and obstacles, is generated through laser radar scanning. Visual perception technology is adopted to process and identify the data for environmental awareness. Single Shot MultiBox Detector (SSD) is utilized to position ships and obstacles, while point cloud data create a comprehensive three-dimensional map. In order to improve the optimal navigation approach of the Rapidly-Exploring Random Tree (RRT), an artificial potential field method is employed. Additionally, the collision prediction model utilizes K-Means clustering to enhance the Faster R-CNN algorithm for predicting the paths of other ships and obstacles. The results indicate that the RRT enhanced by the artificial potential field method reduces the average path length (from 500 to 430 m), average time consumption (from 30 to 22 s), and maximum collision risk (from 15 to 8%). Moreover, the accuracy, recall rate, and F1 score of the K-Means + Faster R-CNN collision prediction model reach 92%, 88%, and 90%, respectively, outperforming other models. Overall, these findings underscore the substantial advantages of the proposed enhanced algorithm in autonomous navigation and collision prediction in port waterways. [ABSTRACT FROM AUTHOR]

Segmenting autophagic bodies in yeast TEM images is a key technique for measuring changes in autophagosome size and number in order to better understand macroautophagy/autophagy. Manual segmentation of these images can be very time consuming, particularly because hundreds of images are needed for accurate measurements. Here we describe a validated Cellpose 2.0 model that can segment these images with accuracy comparable to that of human experts. This model can be used for fully automated segmentation, eliminating the need for manual body outlining, or for model-assisted segmentation, which allows human oversight but is still five times as fast as the current manual method. The model is specific to segmentation of autophagic bodies in yeast TEM images, but researchers working in other systems can use a similar process to generate their own Cellpose 2.0 models to attempt automated segmentations. Our model and instructions for its use are presented here for the autophagy community.Abbreviations: AB, autophagic body; AvP, average precision; GUI, graphical user interface; IoU, intersection over union; MVB, multivesicular body; ROI, region of interest; TEM, transmission electron microscopy; WT,wild type. [ABSTRACT FROM AUTHOR]

Simple Summary: The poultry industry is advancing through a focus on nutrition, management practices, and technology to enhance productivity by improving feed conversion ratios, disease control, lighting management, and exploring antibiotic; alternatives. Infrared (IR) radiation, with wavelengths ranging from 760 to 10,000 nm, is being used to improve the well-being of humans, animals, and poultry. IR applications like heating, spectroscopy, beak trimming, and other therapies offer health benefits such as improved skin health, wound healing, and enhanced immune responses. In poultry production, IR radiation positively impacts growth performance, gut health, blood profiles, and food safety. Despite its benefits, applications of IR in poultry are limited but show promise for wider adoption. This innovative approach has the potential to revolutionize traditional practices in the industry by optimizing productivity and efficiency while improving animal welfare. The poultry industry is dynamically advancing production by focusing on nutrition, management practices, and technology to enhance productivity by improving feed conversion ratios, disease control, lighting management, and exploring antibiotic alternatives. Infrared (IR) radiation is utilized to improve the well-being of humans, animals, and poultry through various operations. IR radiation occurs via electromagnetic waves with wavelengths ranging from 760 to 10,000 nm. The biological applications of IR radiation are gaining significant attention and its utilization is expanding rapidly across multiple sectors. Various IR applications, such as IR heating, IR spectroscopy, IR thermography, IR beak trimming, and IR in computer vision, have proven to be beneficial in enhancing the well-being of humans, animals, and birds within mechanical systems. IR radiation offers a wide array of health benefits, including improved skin health, therapeutic effects, anticancer properties, wound healing capabilities, enhanced digestive and endothelial function, and improved mitochondrial function and gene expression. In the realm of poultry production, IR radiation has demonstrated numerous positive impacts, including enhanced growth performance, gut health, blood profiles, immunological response, food safety measures, economic advantages, the mitigation of hazardous gases, and improved heating systems. Despite the exceptional benefits of IR radiation, its applications in poultry production are still limited. This comprehensive review provides compelling evidence supporting the advantages of IR radiation and advocates for its wider adoption in poultry production practices. [ABSTRACT FROM AUTHOR]

Simple Summary: This study shows that automated computer vision techniques are highly effective when used to analyze images and to extract valuable information from them. We trained an algorithm with a set of 1600 images obtained from a study where wildlife approaching wild boar carcasses were monitored. This enabled the model to detect different classes of animals automatically in their natural environment with a mean average precision of 98.11%. Camera traps are becoming widely used for wildlife monitoring and management. However, manual analysis of the resulting image sets is labor-intensive, time-consuming and costly. This study shows that automated computer vision techniques can be extremely helpful in this regard, as they can rapidly and automatically extract valuable information from the images. Specific training with a set of 1600 images obtained from a study where wild animals approaching wild boar carcasses were monitored enabled the model to detect five different classes of animals automatically in their natural environment with a mean average precision of 98.11%, namely 'wild boar', 'fox', 'raccoon dog', 'deer' and 'bird'. In addition, sequences of images were automatically analyzed and the number of wild boar visits and respective group sizes were determined. This study may help to improve and speed up the monitoring of the potential spread of African swine fever virus in areas where wild boar are affected. [ABSTRACT FROM AUTHOR]

Simple Summary: Ensuring precise needle placement during biopsy procedures is essential for both successful outcomes and minimizing patient risk. Our study presents a new mixed reality system that helps doctors accurately navigate needles to their intended targets without needing physical markers. By using advanced imaging and computer vision techniques, the system overlays anatomical data directly onto the patient, guiding the needle along a pre-planned path. We tested this system in various ways, including its accuracy and efficiency in needle placement. Our findings showcased a significant improvement, with a reduction in number of punctures needed to reach the target location. The test was successfully completed on the first attempt in 70% of cases, as opposed to only 20% without the system. Additionally, there was a 53% reduction in procedure time, validating the effectiveness of the system. Exact biopsy planning and careful execution of needle injection is crucial to ensure successful procedure completion as initially intended while minimizing the risk of complications. This study introduces a solution aimed at helping the operator navigate to precisely position the needle in a previously planned trajectory utilizing a mixed reality headset. A markerless needle tracking method was developed by integrating deep learning and deterministic computer vision techniques. The system is based on superimposing imaging data onto the patient's body in order to directly perceive the anatomy and determine a path from the selected injection site to the target location. Four types of tests were conducted to assess the system's performance: measuring the accuracy of needle pose estimation, determining the distance between injection sites and designated targets, evaluating the efficiency of material collection, and comparing procedure time and number of punctures required with and without the system. These tests, involving both phantoms and physician participation in the latter two, demonstrated the accuracy and usability of the proposed solution. The results showcased a significant improvement, with a reduction in number of punctures needed to reach the target location. The test was successfully completed on the first attempt in 70% of cases, as opposed to only 20% without the system. Additionally, there was a 53% reduction in procedure time, validating the effectiveness of the system. [ABSTRACT FROM AUTHOR]