Search

In this paper, we propose a novel voxel-based 3D single object tracking (3D SOT) method called Voxel Pseudo Image Tracking (VPIT). VPIT is the first method that uses voxel pseudo images for 3D SOT. The input point cloud is structured by pillar-based voxelization, and the resulting pseudo image is used as an input to a 2D-like Siamese SOT method. The pseudo image is created in the Bird's-eye View (BEV) coordinates; and therefore, the objects in it have constant size. Thus, only the object rotation can change in the new coordinate system and not the object scale. For this reason, we replace multi-scale search with a multi-rotation search, where differently rotated search regions are compared against a single target representation to predict both position and rotation of the object. Experiments on KITTI [1] Tracking dataset show that VPIT is the fastest 3D SOT method and maintains competitive Success and Precision values. Application of a SOT method in a real-world scenario meets with limitations such as lower computational capabilities of embedded devices and a latency-unforgiving environment, where the method is forced to skip certain data frames if the inference speed is not high enough. We implement a real-time evaluation protocol and show that other methods lose most of their performance on embedded devices; while, VPIT maintains its ability to track the object. [ABSTRACT FROM AUTHOR]

To navigate complex environments, walking animals must detect and overcome unexpected perturbations. One technical challenge when investigating adaptive locomotion is measuring behavioral responses to precise perturbations during naturalistic walking; another is that manipulating neural activity in sensorimotor circuits often reduces spontaneous locomotion. To overcome these obstacles, we introduce miniature treadmill systems for coercing locomotion and tracking 3D kinematics of walking Drosophila. By systematically comparing walking in three experimental setups, we show that flies compelled to walk on the linear treadmill have similar stepping kinematics to freely walking flies, while kinematics of tethered walking flies are subtly different. Genetically silencing mechanosensory neurons altered step kinematics of flies walking on the linear treadmill across all speeds. We also discovered that flies can maintain a forward heading on a split-belt treadmill by specifically adapting the step distance of their middle legs. These findings suggest that proprioceptive feedback contributes to leg motor control irrespective of walking speed and that the fly's middle legs play a specialized role in stabilizing locomotion. [Display omitted] • A new actuated treadmill system captures 3D kinematics of flies compelled to walk • Flies walking on the treadmill have similar kinematics to freely walking flies • Proprioceptive feedback is important for leg motor control at all walking speeds • Flies on a split-belt treadmill use their middle legs to counteract perturbations Pratt et al. engineer tiny treadmills that enable 3D tracking of fruit fly walking kinematics. The authors systematically compare treadmills to free and tethered walking, evaluate the role of proprioceptive feedback across walking speeds, and reveal how flies adapt their stepping patterns to maintain course on a split-belt treadmill. [ABSTRACT FROM AUTHOR]

Site-attached fish that form social groups may face a trade-off between the advantages of group living and the disadvantages related to intra-group competition for food. A possible solution for the latter is space partitioning among group members. Technological limitations related to individual tagging and underwater tracking hindered such spatial studies in grouping fishes. Here, using underwater video cameras and recent developments in deep learning tools, we successfully tracked the 3D movements of individually tagged fish in 4 groups of the damselfish Dascyllus marginatus in the coral reef of Eilat, Red Sea. Our findings, based on tracking sessions lasting 3–11 min that were recorded during a period of > 1 month, show that the individual fish kept separate foraging spaces with minimal overlap and that this separation was stable in time. When the tidally driven current reversed, the separation was kept, and a corresponding reversal was found in the positions of each fish relative to the coral and its neighbors. We propose that the stable spatial partitioning observed in our study is a primary mechanism through which site-attached species can organize themselves in order to reduce intra-group competition. [ABSTRACT FROM AUTHOR]

3D tracking of single‐port continuum surgical tools is an essential step toward their closed‐loop control in robot‐assisted‐laparoscopy, since single‐port tools possess multiple degrees‐of‐freedom (DoFs) without distal joint sensors and hence have lower motion precision compared to rigid straight‐stemmed tools used in multi‐port robotic laparoscopy. This work proposes a novel markerless 3D tracking framework for continuum surgical tools using a proposed surgical tool partial pose estimation network (STPPE‐Net) based on U‐Net and ResNet. The STPPE‐Net estimates the segmentation and a 5‐DoF pose of the tool end‐effector. This network is entirely trained by a synthetic data generator based on domain randomization (DR) and requires zero manual annotation. The 5‐DoF pose estimation from the STPPE‐Net is combined with the surgical tool axial rotation from the robot control system. Then, the entire pose is further refined via a region‐based optimization that maximizes the overlap between the tool end‐effector segmentation from the STPPE‐Net and its projection onto the image plane of the endoscopic camera. The segmentation accuracy and 6‐DoF pose estimation precision of the proposed framework are validated on the images captured from an endoscopic single‐port system. The experimental results show the effectiveness and robustness of the proposed tracking framework for continuum surgical tools. [ABSTRACT FROM AUTHOR]

In the field of medical applications, precise localization of medical instruments and bone structures is crucial to ensure computer-assisted surgical interventions. In orthopedic surgery, existing devices typically rely on stereoscopic vision. Their purpose is to aid the surgeon in screw fixation of prostheses or bone removal. This article addresses the challenge of localizing a rigid object consisting of randomly arranged planar markers using a single camera. This approach is especially vital in medical situations where accurate object alignment relative to a camera is necessary at distances ranging from 80 cm to 120 cm. In addition, the size limitation of a few tens of centimeters ensures that the resulting locator does not obstruct the work area. This rigid locator consists of a solid at the surface of which a set of plane markers (ArUco) are glued. These plane markers are randomly distributed over the surface in order to systematically have a minimum of two visible markers whatever the orientation of the locator. The calibration of the locator involves finding the relative positions between the individual planar elements and is based on a bundle adjustment approach. One of the main and known difficulties associated with planar markers is the problem of pose ambiguity. To solve this problem, our method lies in the formulation of an efficient initial solution for the optimization step. After the calibration step, the reached positioning uncertainties of the locator are better than two-tenth of a cubic millimeter and one-tenth of a degree, regardless of the orientation of the locator in space. To assess the proposed method, the locator is rigidly attached to a stylus of about twenty centimeters length. Thanks to this approach, the tip of this stylus seen by a 16.1 megapixel camera at a distance of about 1 m is localized in real time in a cube lower than 1 mm side. A surface registration application is proposed by using the stylus on an artificial scapula. [ABSTRACT FROM AUTHOR]

Quantitative tracking of rapidly moving micron-scale objects remains an elusive challenge in microscopy due to low signal-to-noise. This paper describes a novel method for tracking micron-sized motile organisms in off-axis Digital Holographic Microscope (DHM) raw holograms and/or reconstructions. We begin by processing the microscopic images with the previously reported Holographic Examination for Life-like Motility (HELM) software, which provides a variety of tracking outputs including motion history images (MHIs). MHIs are stills of videos where the frame-to-frame changes are indicated with color time-coding. This exposes tracks of objects that are difficult to identify in individual frames at a low signal-to-noise ratio. The visible tracks in the MHIs are superior to tracks identified by all tested automated tracking algorithms that start from object identification at the frame level, particularly in low signal-to-noise ratio data, but do not provide quantitative track data. In contrast to other tracking methods, like Kalman filter, where the recording is analyzed frame by frame, MHIs show the whole time span of particle movement at once and eliminate the need to identify objects in individual frames. This feature also enables post-tracking identification of low-SNR objects. We use these tracks, rather than object identification in individual frames, as a basis for quantitative tracking of Bacillus subtilis by first generating MHIs from X, Y, and t stacks (raw holograms or a projection over reconstructed planes), then using a region-tracking algorithm to identify and separate swimming pathways. Subsequently, we identify each object's Z plane of best focus at the corresponding X, Y, and t points, yielding ap full description of the swimming pathways in three spatial dimensions plus time. This approach offers an alternative to object-based tracking for processing large, low signal-to-noise datasets containing highly motile organisms.

3D tracking of single‐port continuum surgical tools is an essential step toward their closed‐loop control in robot‐assisted‐laparoscopy, since single‐port tools possess multiple degrees‐of‐freedom (DoFs) without distal joint sensors and hence have lower motion precision compared to rigid straight‐stemmed tools used in multi‐port robotic laparoscopy. This work proposes a novel markerless 3D tracking framework for continuum surgical tools using a proposed surgical tool partial pose estimation network (STPPE‐Net) based on U‐Net and ResNet. The STPPE‐Net estimates the segmentation and a 5‐DoF pose of the tool end‐effector. This network is entirely trained by a synthetic data generator based on domain randomization (DR) and requires zero manual annotation. The 5‐DoF pose estimation from the STPPE‐Net is combined with the surgical tool axial rotation from the robot control system. Then, the entire pose is further refined via a region‐based optimization that maximizes the overlap between the tool end‐effector segmentation from the STPPE‐Net and its projection onto the image plane of the endoscopic camera. The segmentation accuracy and 6‐DoF pose estimation precision of the proposed framework are validated on the images captured from an endoscopic single‐port system. The experimental results show the effectiveness and robustness of the proposed tracking framework for continuum surgical tools.

Abstract Recently, zebrafish has received more and more attention due to its wide range of applications such as regeneration promoting therapeutics and drug discovery. Therefore, vision‐based trackers are utilized to record the swimming trajectory of zebrafish. In this paper, a re‐association method is introduced in the 3D reconstruction process to generate missed targets caused by occlusion. Since the variation of the overall tracking targets has the property of continuity and stability, a topology association model (TAM) is proposed by point group similarity into the tracking framework. TAM describes the movement of zebrafish from the macroscopic level and utilizes the changes of the point group structure for tracking. Experimental results show that the tracking framework enhances the overall performance and promotes the trajectory integrity. On the latest 3D‐ZeF20 benchmark, state‐of‐the‐art results are achieved. In addition, TAM tracking framework is applied to 2D general tracking to prove that the method is useful and have great advantage in other scenarios with relatively stable amount of targets as well.

Abstract Background As well known to any photographer, controlling the “field of view” offers an extremely powerful mechanism by which to adjust target acquisition. Only a few natural sensory systems can actively control their field of view (e.g., dolphins, whales, and bats). Bats are known for their active sensing abilities and modify their echolocation signals by actively controlling their spectral and temporal characteristics. Less is known about bats’ ability to actively modify their bio-sonar field of view. Results We show that Pipistrellus kuhlii bats rapidly narrow their sensory field of view (i.e., their bio-sonar beam) when scanning a target. On-target vertical sonar beams were twofold narrower than off-target beams. Continuous measurements of the mouth gape of free-flying bats revealed that they control their bio-sonar beam by a ~3.6 mm widening of their mouth gape: namely, bats open their mouth to narrow the beam and vice versa. Conclusions Bats actively and rapidly control their echolocation vertical beam width by modifying their mouth gape. We hypothesize that narrowing their vertical beam narrows the zone of ensonification when estimating the elevation of a target. In other words, bats open their mouth to improve sensory localization.

Bacteria navigate natural habitats with a wide range of mechanical properties, from the ocean to the digestive tract and soil, by rotating helical flagella like propellers. Species differ in the number, position, and shape of their flagella, but the adaptive value of these flagellar architectures is unclear. Many species traverse multiple types of environments, such as pathogens inside and outside a host. We investigate the hypothesis that flagellar architectures mediate environment-specific benefits in the marine pathogen Vibrio alginolyticus which exhibits physiological adaptation to the mechanical environment. In addition to its single polar flagellum, the bacterium produces lateral flagella in environments that differ mechanically from water. These are known to facilitate surface motility and attachment. We use high-throughput 3D bacterial tracking to quantify chemotactic performance of both flagellar architectures in three archetypes of mechanical environments relevant to the bacterium's native habitats: water, polymer solutions, and hydrogels. We reveal that lateral flagella impede chemotaxis in water by lowering the swimming speed but improve chemotaxis in both types of complex environments. Statistical trajectory analysis reveals two distinct underlying behavioral mechanisms: In viscous solutions of the polymer PVP K90, lateral flagella increase the swimming speed. In agar hydrogels, lateral flagella improve overall chemotactic performance, despite lowering the swimming speed, by preventing trapping in pores. Our findings show that lateral flagella are multipurpose tools with a wide range of applications beyond surfaces. They implicate flagellar architecture as a mediator of environment-specific benefits and point to a rich space of bacterial navigation behaviors in complex environments. [ABSTRACT FROM AUTHOR]

Microsurgery is a general term for surgery combining surgical microscope and specialized precision instruments during operation. Training in microsurgery requires considerable time and training resources. With the rapid development of computer technologies, virtual surgery simulation has gained extensive attention over the past decades. In this work, we take advantage of mixed reality (MR) that creates an interactive environment where physical and digital objects coexist, and present an MR framework for the microsurgery simulation. It enables users to practice anastomosis skills with real microsurgical instruments rather than additional haptic feedback devices that are typically used in virtual reality-based systems, and to view a realistic rendering intra-operative scene at the same time, thus creating an immersive training experience with such a visual-tactile interactive environment. A vision-based tracking system is proposed to simultaneously track microsurgical instruments and artificial blood vessels, and a learning-based anatomical modeling approach is introduced to facilitate the development of simulations in different microsurgical specialities by rapidly creating virtual assets. Moreover, we build a prototype system for the simulation specializing in microvascular hepatic artery reconstruction to demonstrate the feasibility and applicability of our framework. [ABSTRACT FROM AUTHOR]

During the last decades, several hospitals stopped service due to severe seismic damage to nonstructural components and medical equipment. This article investigates the dynamic behavior of medical equipment deployed in a full-scale, five-story building with two different support conditions, base isolated (BI) and fixed to the base (FB). Two nonlinear mathematical models, namely, rolling and toppling, are used to simulate the observed experimental responses using recorded data and the camera projection technique (CPT). CPT generated the possibility of measuring equipment horizontal displacements, slips, rotations, rocking, and toppling responses. The Euler–Lagrange formulation, along with the Stribeck friction model, was used to numerically model the rolling and in-plane rotation behavior of a four-wheel mobile cart (4WMC), which is considered to represent the trajectory of equipment supported on caster wheels. Besides, a simple two-dimensional (2D) rigid block model was used to describe the rocking and toppling behavior of locked and free-standing equipment. It was found that the 4WMC model was sensitive to the platform and wheels' initial orientation angles, as well as the static and kinetic friction coefficients. As it should be, the toppling model was sensitive to the block dimensions and the intensity of the shaking. It was concluded that both numerical models successfully predicted the equipment rolling, in-plane rotation, rocking, and toppling behavior, as long as neither severe lateral impacts nor significant changes in the equipment mass occur during the motion. [ABSTRACT FROM AUTHOR]

Robotic guides take visitors on a tour of a facility. Such robots must always know the position of the visitor for decision-making. Current tracking algorithms largely assume that the person will be nearly always visible. In the robotic guide application, a person's visibility could be often lost for prolonged periods, especially when the robot is circumventing a corner or making a sharp turn. In such cases, a person cannot quickly come behind the limited field of view rear camera. We propose a new algorithm that can track people for prolonged times under such conditions. The algorithm is benefitted from an application-level heuristic that the person will be nearly always following the robot, which can be used to guess the motion. The proposed work uses a Particle Filter with a 'follow-the-robot' motion model for tracking. The tracking is performed in 3D using a monocular camera. Unlike approaches in the literature, the proposed work observes from a moving base that is especially challenging since a rotation of the robot can cause a large sudden change in the position of the human in the image plane that the approaches in the literature would filter out. Tracking in 3D can resolve such errors. The proposed approach is tested for three different indoor scenarios. The results showcase that the approach is significantly better than the baselines including tracking in the image and projecting in 3D, tracking using a randomized (non-social) motion model, tracking using a Kalman Filter and using LSTM for trajectory prediction. [ABSTRACT FROM AUTHOR]

Recently, zebrafish has received more and more attention due to its wide range of applications such as regeneration promoting therapeutics and drug discovery. Therefore, vision‐based trackers are utilized to record the swimming trajectory of zebrafish. In this paper, a re‐association method is introduced in the 3D reconstruction process to generate missed targets caused by occlusion. Since the variation of the overall tracking targets has the property of continuity and stability, a topology association model (TAM) is proposed by point group similarity into the tracking framework. TAM describes the movement of zebrafish from the macroscopic level and utilizes the changes of the point group structure for tracking. Experimental results show that the tracking framework enhances the overall performance and promotes the trajectory integrity. On the latest 3D‐ZeF20 benchmark, state‐of‐the‐art results are achieved. In addition, TAM tracking framework is applied to 2D general tracking to prove that the method is useful and have great advantage in other scenarios with relatively stable amount of targets as well. [ABSTRACT FROM AUTHOR]

We present a method to acquire 3D position measurements for decentralized target tracking with an asynchronous camera network. Cameras with known poses have fields of view with overlapping projections on the ground and 3D volumes above a reference ground plane. The purpose is to track targets in 3D space without constraining motion to a reference ground plane. Cameras exchange line-of-sight vectors and respective time tags asynchronously. From stereoscopy, we obtain the fused 3D measurement at the local frame capture instant. We use local decentralized Kalman information filtering and particle filtering for target state estimation to test our approach with only local estimation. Monte Carlo simulation includes communication losses due to frame processing delays. We measure performance with the average root mean square error of 3D position estimates projected on the image planes of the cameras. We then compare only local estimation to exchanging additional asynchronous communications using the Batch Asynchronous Filter and the Sequential Asynchronous Particle Filter for further fusion of information pairs' estimates and fused 3D position measurements, respectively. Similar performance occurs in spite of the additional communication load relative to our local estimation approach, which exchanges just line-of-sight vectors. [ABSTRACT FROM AUTHOR]

Super-resolution fluorescence microscopy techniques are powerful tools to investigate polymer systems. In this review, we address how these techniques have been applied to hydrogel nano- and microparticles, so-called nano- or microgels. We outline which research questions on microgels could be addressed and what new insights could be achieved. Studies of the morphology, shape, and deformation of microgels; their internal compartmentalization; the cross-linker distribution and polarity inside them; and their dynamics and diffusion are summarized. In particular, the abilities to super-resolve structures in three dimensions have boosted the research field and have also allowed researchers to obtain impressive 3D images of deformed microgels. Accessing information beyond 3D localization, such as spectral and lifetime properties and correlative imaging or the combination of data with other methods, shines new light onto polymer systems and helps us understand their complexity in detail. Such future trends and developments are also addressed. [ABSTRACT FROM AUTHOR]

Acquiring 3D trajectories of on-road vehicles is an essential visual task for autonomous driving systems. Existing 3D vehicle tracking methods either rely on point cloud data or need to be trained on visual tracking datasets. In comparison, a decoupled monocular 3D vehicle tracking framework is proposed in this paper. Because our framework is the first of its kind, a previous decoupled LiDAR-based method is taken as the baseline by substituting its detector with a monocular one. On this foundation, we further employ global coordinates to cancel out ego motion and introduce the angular rate into the 3D Kalman filter. In order to tackle the problem of long-term association, a trajectory management scheme is proposed with our novel hibernation mechanism. Furthermore, it is pointed out that current monocular 3D tracking methods have not been tailored for the depth estimation uncertainty produced by monocular 3D detectors. In this regard, we propose a depth-aware association strategy which endows remoter vehicles with larger matching regions in the data association stage. As another contribution, we discuss the defects of current metrics for evaluating 3D tracking performance and devise a nonuniform metric which is dedicated to monocular vision. Through extensive experiments conducted on the KITTI tracking benchmark, the superiority of proposed monocular 3D vehicle tracking framework and metric is demonstrated by both quantitative results and qualitative intuition. [ABSTRACT FROM AUTHOR]

Background: As well known to any photographer, controlling the "field of view" offers an extremely powerful mechanism by which to adjust target acquisition. Only a few natural sensory systems can actively control their field of view (e.g., dolphins, whales, and bats). Bats are known for their active sensing abilities and modify their echolocation signals by actively controlling their spectral and temporal characteristics. Less is known about bats' ability to actively modify their bio-sonar field of view. Results: We show that Pipistrellus kuhlii bats rapidly narrow their sensory field of view (i.e., their bio-sonar beam) when scanning a target. On-target vertical sonar beams were twofold narrower than off-target beams. Continuous measurements of the mouth gape of free-flying bats revealed that they control their bio-sonar beam by a ~3.6 mm widening of their mouth gape: namely, bats open their mouth to narrow the beam and vice versa. Conclusions: Bats actively and rapidly control their echolocation vertical beam width by modifying their mouth gape. We hypothesize that narrowing their vertical beam narrows the zone of ensonification when estimating the elevation of a target. In other words, bats open their mouth to improve sensory localization. [ABSTRACT FROM AUTHOR]

Abstract Research on diel vertical migration (DVM) is generally conducted at the population level, whereas few studies have focused on how individual animals behaviorally respond to threats when also having access to foraging opportunities. We utilized a 3D tracking platform to record the swimming behavior of Daphnia magna exposed to ultraviolet radiation (UVR) in the presence or absence of a food patch. We analyzed the vertical position of individuals before and during UVR exposure and found that the presence of food reduced the average swimming depth during both sections of the trial. Since UVR is a strong driver of zooplankton behavior, our results highlight that biotic factors, such as food patches, have profound effects on both the amplitude and the frequency of avoidance behavior. In a broader context, the trade‐off between threats and food adds to our understanding of the strength and variance of behavioral responses to threats, including DVM.

In this study, we propose a high-speed and low-latency 3D fluorescence imaging method for robotic microscopes. The prototype system consists of a focus-tunable lens called a TAG lens, which operates at several hundred kHz, an image intensifier (I.I.) that enhances faint light such as fluorescence, and a high-speed vision system that can transfer acquired images to the host PC in 500 Hz. The proposed method can acquire images at arbitrary focal lengths at frame rates on the order of 1 kHz by synchronizing the focal-length fluctuation of the TAG lens and the exposure timing of the I.I., whose duration is a few hundred nanoseconds. The low-latency we aim for in this paper is on the order of a few milliseconds. A prototype system was developed to validate the proposed method. High-speed 3D tracking of the Brownian motion of a fluorescent bead of 0.5 μm diameter was demonstrated to verify the feedback performance of the proposed low-latency 3D fluorescence imaging method. [ABSTRACT FROM AUTHOR]

3D motion tracking is a challenging task when both the tracked object and the observer are moving. In this paper, we present a multi-behavioural social force-based particle filter to track a group of moving humans from a moving robot using a limited field-of-view monocular camera. The application is a robotic guide and while moving, the robot often loses visibility of one or more people in the group, who must still be tracked. As an example, due to limited space, when the robot takes a sharp turn to avoid an obstacle or circumvent a corner, the visibility of the people at the rear is lost for some time. Therefore, several human social behavioural aspects have been implemented to predict the human's motion in a group. The model accounts for attraction and repulsion between the people of the group and those with the robot, to maintain a comfortable social distance with each other at equilibrium. Additionally, when any person leaves the group then the track is deleted and after joining the track is automatically re-initialized. In the literature, the time of invisibility is a criterion to detect a person who has left the system, which however cannot be used here since the invisibility may be due to a limited field of view or the robot making a sharp turn to avoid an obstacle or circumventing a corner. Social heuristics are used to accurately detect people leaving the robotic system. The tracked trajectory is compared with ground truth and our system gives a very less error when compared with several baseline approaches. False positives are reduced, and the accuracy also increased with our proposed model as compared to other baseline methods. This method has been tested on several scenarios to ensure its validity. [ABSTRACT FROM AUTHOR]

Abstract Background Acquiring high resolution quantitative behavioural data underwater often involves installation of costly infrastructure, or capture and manipulation of animals. Aquatic movement ecology can therefore be limited in taxonomic range and ecological coverage. Methods Here we present a novel deep-learning based, multi-individual tracking approach, which incorporates Structure-from-Motion in order to determine the 3D location, body position and the visual environment of every recorded individual. The application is based on low-cost cameras and does not require the animals to be confined, manipulated, or handled in any way. Results Using this approach, single individuals, small heterospecific groups and schools of fish were tracked in freshwater and marine environments of varying complexity. Positional tracking errors as low as 1.09 ± 0.47 cm (RSME) in underwater areas up to 500 m2 were recorded. Conclusions This cost-effective and open-source framework allows the analysis of animal behaviour in aquatic systems at an unprecedented resolution. Implementing this versatile approach, quantitative behavioural analysis can be employed in a wide range of natural contexts, vastly expanding our potential for examining non-model systems and species.

Border cells are a cluster of cells that migrate from the anterior tip of the Drosophila egg chamber to the border of the oocyte in stage 9. They serve as a useful model to study collective cell migration in a native tissue environment. Here we describe a protocol for preparing ex vivo egg chamber cultures from transgenic flies expressing fluorescent proteins in the border cells, and using confocal microscopy to take a multi-positional time-lapse movie. We include an image analysis method for tracking border cell cluster dynamics as well as tracking individual cell movements.

Research on diel vertical migration (DVM) is generally conducted at the population level, whereas few studies have focused on how individual animals behaviorally respond to threats when also having access to foraging opportunities. We utilized a 3D tracking platform to record the swimming behavior of Daphnia magna exposed to ultraviolet radiation (UVR) in the presence or absence of a food patch. We analyzed the vertical position of individuals before and during UVR exposure and found that the presence of food reduced the average swimming depth during both sections of the trial. Since UVR is a strong driver of zooplankton behavior, our results highlight that biotic factors, such as food patches, have profound effects on both the amplitude and the frequency of avoidance behavior. In a broader context, the trade‐off between threats and food adds to our understanding of the strength and variance of behavioral responses to threats, including DVM. [ABSTRACT FROM AUTHOR]

In this paper, a direction-of-arrival (DOA)-based factor graph (FG) technique is proposed for three-dimensional (3D) tracking. Multiple sensors are utilized in this system, which could measure both azimuth and elevation DOAs emitted from an anonymous target. To realize non-linear tracking, a modified extended Kalman filter (EKF) is proposed. Specifically, on the one hand, the proposed EKF observer is no longer independent of the EKF predictor, but instead takes the predicted target location into its operation. With this technique, the accuracy of detection is improved while the computational complexity is dramatically reduced. On the other hand, the variance of the EKF observer error is estimated in real time, based on the predicted Cramer-Rao bound (PCRB). Therefore, the robustness of detection can be guaranteed even with an unstable sensing environment. In this sense, the EKF observer and the EKF predictor are operated in an integrated FG framework. The advantages of the proposed technique are verified by both complexity analyses and simulation results. [ABSTRACT FROM AUTHOR]

Single Particle Tracking (SPT) is a technique used to locate fluorescent particles with nanometer precision. In the orbital tracking method the position of a particle is obtained analyzing the distribution of intensity along a circular orbit scanned around the particle. In combination with an active feedback this method allows tracking of particles in 2D and 3D with millisecond temporal resolution. Here we describe a SPT setup based on a feedback approach implemented with minimal modification of a commercially available confocal laser scanning microscope, the Zeiss LSM 510, in combination with an external piezoelectric stage scanner. The commercial microscope offers the advantage of a user-friendly software interface and pre-calibrated hardware components. The use of an external piezo-scanner allows the addition of feedback into the system but also represents a limitation in terms of its mechanical response. We describe in detail this implementation of the orbital tracking method and discuss advantages and limitations. As an example of application to live cell experiments we perform the 3D tracking of acidic vesicles in live polarized epithelial cells.

The 3D object tracking from a monocular RGB image is a challenging task. Although popular color and edge-based methods have been well studied, they are only applicable to certain cases and new solutions to the challenges in real environment must be developed. In this paper, we propose a robust 3D object tracking method with adaptively weighted local bundles called AWLB tracker to handle more complicated cases. Each bundle represents a local region containing a set of local features. To alleviate the negative effect of the features in low-confidence regions, the bundles are adaptively weighted using a spatially-variant weighting function based on the confidence values of the involved energy terms. Therefore, in each frame, the weights of the energy items in each bundle are adapted to different situations and different regions of the same frame. Experiments show that the proposed method can improve the overall accuracy in challenging cases. We then verify the effectiveness of the proposed confidence-based adaptive weighting method using ablation studies and show that the proposed method overperforms the existing single-feature methods and multi-feature methods without adaptive weighting. [ABSTRACT FROM AUTHOR]

Future anthropogenic climate change is predicted to impact sensory-driven behaviors. Building on recent improvements in computational power and tracking technology, we have developed a versatile climate-controlled wind tunnel system, in which to study the effect of climate parameters, including temperature, precipitation, and elevated greenhouse gas levels, on odor-mediated behaviors in insects. To establish a baseline for future studies, we here analyzed the host-seeking behavior of the major malaria vector mosquito, Anopheles gambiae sensu strico, to human odor and carbon dioxide (CO2), under tightly controlled climatic conditions, and isolated from potential background contamination by the presence of an experimenter. When presented with a combination of human foot odor and CO2 (case study I), mosquitoes engaged in faster crosswind flight, spent more time in the filamentous odor plume and targeted the odor source more successfully. In contrast, female An. gambiae s. s. presented with different concentrations of CO2 alone, did not display host-seeking behavior (case study II). These observations support previous findings on the role of human host-associated cues in host seeking and confirm the role of CO2 as a synergist, but not a host-seeking cue on its own. Future studies are aimed at investigating the effect of climate change on odor-mediated behavior in mosquitoes and other insects. Moreover, the system will be used to investigate detection and processing of olfactory information in various behavioral contexts, by providing a fine-scale analysis of flight behavior.

The accumulation of motile cells at solid interfaces increases the rate of surface encounters and the likelihood of surface attachment, leading to surface colonization and biofilm formation. The cell density distribution in the vicinity of a physical boundary is influenced by the interactions between the microswimmers and their physical environment, including hydrodynamic and steric interactions, as well as by stochastic effects. Disentangling the contributions of these effects remains an experimental challenge. Here, we use a custom-made four-camera view microscope to track a population of motile puller-type Chlamydomonas reinhardtii in a relatively unconstrained three-dimensional (3D) domain. Our experiments yield an extensive sample of 3D trajectories including cell-surface encounters with a planar wall, from which we extract a full description of the dynamics and the stochasticity of swimming cells. We use this large data sample and combine it with Monte Carlo simulations to determine the link between the dynamics at the single-cell level and the cell density. Our experiments demonstrate that the near-wall scattering is bimodal, corresponding to steric and hydrodynamic effects. We find, however, that this near-wall dynamics has little influence on the cell accumulation at the surface. On the other hand, we present evidence of a cell-induced surface-directed rotation leading to a vertical orbiting behavior and hopping trajectories, consistent with long-range hydrodynamic effects. We identify this long-range effect to be at the origin of the significant surface accumulation of cells. [ABSTRACT FROM AUTHOR]

Future anthropogenic climate change is predicted to impact sensory-driven behaviors. Building on recent improvements in computational power and tracking technology, we have developed a versatile climate-controlled wind tunnel system, in which to study the effect of climate parameters, including temperature, precipitation, and elevated greenhouse gas levels, on odor-mediated behaviors in insects. To establish a baseline for future studies, we here analyzed the host-seeking behavior of the major malaria vector mosquito, Anopheles gambiae sensu strico , to human odor and carbon dioxide (CO2), under tightly controlled climatic conditions, and isolated from potential background contamination by the presence of an experimenter. When presented with a combination of human foot odor and CO2 (case study I), mosquitoes engaged in faster crosswind flight, spent more time in the filamentous odor plume and targeted the odor source more successfully. In contrast, female An. gambiae s. s. presented with different concentrations of CO2 alone, did not display host-seeking behavior (case study II). These observations support previous findings on the role of human host-associated cues in host seeking and confirm the role of CO2 as a synergist, but not a host-seeking cue on its own. Future studies are aimed at investigating the effect of climate change on odor-mediated behavior in mosquitoes and other insects. Moreover, the system will be used to investigate detection and processing of olfactory information in various behavioral contexts, by providing a fine-scale analysis of flight behavior. [ABSTRACT FROM AUTHOR]

Background: Particle-tracking in 3D is an indispensable computational tool to extract critical information on dynamical processes from raw time-lapse imaging. This is particularly true with in vivo time-lapse fluorescence imaging in cell and developmental biology, where complex dynamics are observed at high temporal resolution. Common tracking algorithms used with time-lapse data in fluorescence microscopy typically assume a continuous signal where background, recognisable keypoints and independently moving objects of interest are permanently visible. Under these conditions, simple registration and identity management algorithms can track the objects of interest over time. In contrast, here we consider the case of transient signals and objects whose movements are constrained within a tissue, where standard algorithms fail to provide robust tracking. Results: To optimize 3D tracking in these conditions, we propose the merging of registration and tracking tasks into a registration algorithm that uses random sampling to solve the identity management problem. We describe the design and application of such an algorithm, illustrated in the domain of plant biology, and make it available as an open-source software implementation. The algorithm is tested on mitotic events in 4D data-sets obtained with light-sheet fluorescence microscopy on growing Arabidopsis thaliana roots expressing CYCB::GFP. We validate the method by comparing the algorithm performance against both surrogate data and manual tracking. Conclusion: This method fills a gap in existing tracking techniques, following mitotic events in challenging data-sets using transient fluorescent markers in unregistered images. [ABSTRACT FROM AUTHOR]

Animals must tune their physical performance to changing environmental conditions, and the breadth of environmental tolerance may contribute to delineating the geographic range of a species. A common environmental challenge that flying animals face is the reduction of air density at high elevation and the reduction in the effectiveness of lift production that accompanies it. As a species, turkey vultures (Cathartes aura) inhabit a >3000 m elevation range, and fly considerably higher, necessitating that they accommodate for a 27% change in air density (0.890 to 1.227 kg m-3) through behavior, physiology or biomechanics. We predicted that birds flying at high elevation would maintain aerodynamic lift performance behaviorally via higher flight speeds, rather than increases in power output or local phenotypic adaptation. We used three-dimensional videography to track turkey vultures flying at three elevations, and data supported the hypothesized negative relationship between median airspeed and air density. Additionally, neither the ratio of horizontal speed to sinking speed nor flapping behavior varied with air density., Competing Interests: Competing interests The authors declare no competing or financial interests., (© 2024. Published by The Company of Biologists Ltd.)

Multiple object tracking has been a challenging field, mainly due to noisy detection sets an identity switch caused by occlusion and similar appearance among nearby targets. Previous works rely on appearance models that are built on an individual or several selected frames for the comparison of features, but they cannot encode the long-term appearance changes caused by pose, viewing angle, and lighting conditions. In this paper, we propose an adaptive model that learns online a relatively long-term appearance change of each target. The proposed model is compatible with any feature of fixed dimension or their combination, whose learning rates are dynamically controlled by the adaptive update and spatial weighting schemes. To handle occlusion and nearby objects that are sharing a similar appearance, we also design the cross-matching and re-identification schemes based on the application of the proposed adaptive appearance models. In addition, the 3D geometry information is effectively incorporated in our formulation for data association. The proposed method outperforms all the state of the art on the MOTChallenge 3D benchmark and achieves real-time computation with only a standard desktop CPU. It has also shown superior performance over the state of the art on the 2D benchmark of MOTChallenge.

A system for the 3D tracking of underwater farmed fry using action cameras in multiple directions is presented. In a real environment, owing to light reflection and the small size of fry relative to the size of the tank, blind spots and unclear figures often appear in pictures taken by a camera. The proposed system continuously monitors the fry's figure clearly and avoids blind spots by using multiple cameras. The proposed method changes the combination of multiple viewing points from frame to frame in a movie depending on the fry's location to maintain clear fry figures and obtains the 3D coordinates of the location by the direct linear transformation. The regions of fry are sequentially obtained by taking the difference between the background image and each frame where fry appear. By taking movies of some farmed fry, we examined the tracking performance by comparing the 3D coordinates of the locations obtained by the proposed method with the correct coordinates. The mean and the standard deviation of the distance between the two points were 2.619 and 1.333 cm, respectively, i.e., it was confirmed that the proposed method can correctly capture the locations of fry because the distance was smaller than the body length of the fry. [ABSTRACT FROM AUTHOR]

Postural instability is prevalent in aging and neurodegenerative disease, decreasing quality of life and independence. Quantitatively monitoring balance control is important for assessing treatment efficacy and rehabilitation progress. However, existing technologies for assessing postural sway are complex and expensive, limiting their widespread utility. Here, we propose a monocular imaging system capable of assessing sub-millimeter 3D sway dynamics during quiet standing. Two anatomical targets with known feature geometries were placed on the lumbar and shoulder. Upper and lower trunk 3D kinematic motion were automatically assessed from a set of 2D frames through geometric feature tracking and an inverse motion model. Sway was tracked in 3D and compared between control and hypoperfusion conditions in 14 healthy young adults. The proposed system demonstrated high agreement with a commercial motion capture system (error $1.5 \times 10^{-4}\,\,\text {mm}$ , [−0.52, 0.52]). Between-condition differences in sway dynamics were observed in anterior-posterior sway during early and mid stance, and medial-lateral sway during mid stance commensurate with decreased cerebral perfusion, followed by recovered sway dynamics during late stance with cerebral perfusion recovery. This inexpensive single-camera system enables quantitative 3D sway monitoring for assessing neuromuscular balance control in weakly constrained environments. [ABSTRACT FROM AUTHOR]