Search

We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the head movement. To this end, we designed the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks. Our analysis demonstrates the effectiveness of addressing distortion specific to egocentric cameras, adopting high-capacity transformers to learn complex hand-object interactions, and fusing predictions from different views. Our study further reveals challenging scenarios intractable with state-of-the-art methods, such as fast hand motion, object reconstruction from narrow egocentric views, and close contact between two hands and objects. Our efforts will enrich the community's knowledge foundation and facilitate future hand studies on egocentric hand-object interactions., Comment: Accepted to ECCV 2024

Calcium oscillation is an important calcium homeostasis, imbalance of which is the key mechanism of initiation and progression of many major diseases. The formation and maintenance of calcium homeostasis are closely related to the spatial distribution of calcium channels. In the current paper, a theoretical framework is established by abstracting the spatial distribution of the calcium channels as a nonlinear biological complex network with calcium channels as nodes and Ca$^{2+}$ as edges. A dynamical model for a RyR is adopted to investigate the effect of spatial distribution on calcium oscillation. The mean-field model can be well reproduced from the complete graph and dense Erd\"os-R\'enyi network. The synchronization of RyRs is found important to generate a global calcium oscillation. The clique graph with a cluster structure can not produce a global oscillation due to the failure of synchronization between clusters. A more realistic geometric network is constructed in a two-dimensional plane based on the experimental information about the RyR arrangement of clusters and the frequency distribution of cluster sizes. Different from the clique graph, the global oscillation can be generated with reasonable parameters on the geometric network. The simulation also suggests that existence of small clusters and rogue RyR's plays an important role in the maintenance of global calcium oscillation through keeping synchronization between large clusters. Such results support the heterogeneous distribution of RyR's with different-size clusters, which is helpful to understand recent observations with super resolution nanoscale imaging techniques. The current theoretical framework can also be extent to investigate other phenomena in calcium signal transduction., Comment: 14 pages, 8 figures, to be published in Phys. Rev. E

Object localization in 3D space is a challenging aspect in monocular 3D object detection. Recent advances in 6DoF pose estimation have shown that predicting dense 2D-3D correspondence maps between image and object 3D model and then estimating object pose via Perspective-n-Point (PnP) algorithm can achieve remarkable localization accuracy. Yet these methods rely on training with ground truth of object geometry, which is difficult to acquire in real outdoor scenes. To address this issue, we propose MonoRUn, a novel detection framework that learns dense correspondences and geometry in a self-supervised manner, with simple 3D bounding box annotations. To regress the pixel-related 3D object coordinates, we employ a regional reconstruction network with uncertainty awareness. For self-supervised training, the predicted 3D coordinates are projected back to the image plane. A Robust KL loss is proposed to minimize the uncertainty-weighted reprojection error. During testing phase, we exploit the network uncertainty by propagating it through all downstream modules. More specifically, the uncertainty-driven PnP algorithm is leveraged to estimate object pose and its covariance. Extensive experiments demonstrate that our proposed approach outperforms current state-of-the-art methods on KITTI benchmark., Comment: CVPR 2021

The spin-1/2 antiferromagnetic Heisenberg systems are studied on three typical diamond-type hierarchical lattices (systems A, B and C) with fractal dimensions 1.63, 2 and 2.58, respectively, and the phase diagrams, critical phenomena and quantum correlations are calculated by a combination of the equivalent transformation and real-space renormalization group methods. We find that there exist a reentrant behavior for system A and a finite temperature transition in the isotropic Heisenberg limit for system C (not for system B). Unlike the ferromagnetic case, the Neel temperatures of antiferromagnetic systems A and B are inversely proportional to ln(Delta_c-Delta) (when Delta->Delta_c) and ln Delta (when Delta->0), respectively. And we also find that there is a turning point of quantum correlation in the isotropic Heisenberg limit Delta=0 where there is a peak of the contour and no matter how large the size of system is, quantum correlation will change to zero in the Ising limit for the three systems. The quantum correlation decreases with the increase of lattice size L and it is almost zero when L>=30 for system A, and for systems B and C, they still exist when L is larger than that of system A. Moreover, as an example, we discuss the error of result in system A, which is induced by the noncommutativity., Comment: 28 pages, 13 figures

For vehicles equipped with the automatic parking system, the accuracy and speed of the parking slot detection are crucial. But the high accuracy is obtained at the price of low speed or expensive computation equipment, which are sensitive for many car manufacturers. In this paper, we proposed a detector using CNN(convolutional neural networks) for faster speed and smaller model size while keeps accuracy. To achieve the optimal balance, we developed a strategy to select the best receptive fields and prune the redundant channels automatically after each training epoch. The proposed model is capable of jointly detecting corners and line features of parking slots while running efficiently in real time on average processors. The model has a frame rate of about 30 FPS on a 2.3 GHz CPU core, yielding parking slot corner localization error of 1.51$\pm$2.14 cm (std. err.) and slot detection accuracy of 98\%, generally satisfying the requirements in both speed and accuracy on on-board mobile terminals.

Brain controlled vehicle refers to the vehicle that obtains control commands by analyzing the driver's EEG through Brain-Computer Interface (BCI). The research of brain controlled vehicles can not only promote the integration of brain machines, but also expand the range of activities and living ability of the disabled or some people with limited physical activity, so the research of brain controlled vehicles is of great significance and has broad application prospects. At present, BCI has some problems such as limited recognition accuracy, long recognition time and limited number of recognition commands in the process of analyzing EEG signals to obtain control commands. If only use the driver's EEG signals to control the vehicle, the control performance is not ideal. Based on the concept of Shared control, this paper uses the fuzzy control (FC) to design an auxiliary controller to realize the cooperative control of automatic control and brain control. Designing a Shared controller which evaluates the current vehicle status and decides the switching mechanism between automatic control and brain control to improve the system control performance. Finally, based on the joint simulation platform of Carsim and MATLAB, with the simulated brain control signals, the designed experiment verifies that the control performance of the brain control vehicle can be improved by adding the auxiliary controller.

Brain-controlled unmanned aerial vehicle (uav) is a uav that can analyze human brain electrical signals through BCI to obtain flight commands. The research of brain-controlled uav can promote the integration of brain-computer and has a broad application prospect. At present, BCI still has some problems, such as limited recognition accuracy, limited recognition time and small number of recognition commands in the acquisition of control commands by analyzing eeg signals. Therefore, the control performance of the quadrotor which is controlled only by brain is not ideal. Based on the concept of Shared control, this paper designs an assistant controller using fuzzy PID control, and realizes the cooperative control between automatic control and brain control. By evaluating the current flight status and setting the switching rate, the switching mechanism of automatic control and brain control can be decided to improve the system control performance. Finally, a rectangular trajectory tracking control experiment of the same height is designed for small quadrotor to verify the algorithm.