Search

Understanding human activity is a crucial aspect of developing intelligent robots, particularly in the domain of human-robot collaboration. Nevertheless, existing systems encounter challenges such as over-segmentation, attributed to errors in the up-sampling process of the decoder. In response, we introduce a promising solution: the Temporal Fusion Graph Convolutional Network. This innovative approach aims to rectify the inadequate boundary estimation of individual actions within an activity stream and mitigate the issue of over-segmentation in the temporal dimension. Moreover, systems leveraging human activity recognition frameworks for decision-making necessitate more than just the identification of actions. They require a confidence value indicative of the certainty regarding the correspondence between observations and training examples. This is crucial to prevent overly confident responses to unforeseen scenarios that were not part of the training data and may have resulted in mismatches due to weak similarity measures within the system. To address this, we propose the incorporation of a Spectral Normalized Residual connection aimed at enhancing efficient estimation of novelty in observations. This innovative approach ensures the preservation of input distance within the feature space by imposing constraints on the maximum gradients of weight updates. By limiting these gradients, we promote a more robust handling of novel situations, thereby mitigating the risks associated with overconfidence. Our methodology involves the use of a Gaussian process to quantify the distance in feature space., Comment: 15 pages, 10 figures, The International Journal of Robotics Research

Human activities recognition is an important task for an intelligent robot, especially in the field of human-robot collaboration, it requires not only the label of sub-activities but also the temporal structure of the activity. In order to automatically recognize both the label and the temporal structure in sequence of human-object interaction, we propose a novel Pyramid Graph Convolutional Network (PGCN), which employs a pyramidal encoder-decoder architecture consisting of an attention based graph convolution network and a temporal pyramid pooling module for downsampling and upsampling interaction sequence on the temporal axis, respectively. The system represents the 2D or 3D spatial relation of human and objects from the detection results in video data as a graph. To learn the human-object relations, a new attention graph convolutional network is trained to extract condensed information from the graph representation. To segment action into sub-actions, a novel temporal pyramid pooling module is proposed, which upsamples compressed features back to the original time scale and classifies actions per frame. We explore various attention layers, namely spatial attention, temporal attention and channel attention, and combine different upsampling decoders to test the performance on action recognition and segmentation. We evaluate our model on two challenging datasets in the field of human-object interaction recognition, i.e. Bimanual Actions and IKEA Assembly datasets. We demonstrate that our classifier significantly improves both framewise action recognition and segmentation, e.g., F1 micro and F1@50 scores on Bimanual Actions dataset are improved by $4.3\%$ and $8.5\%$ respectively., Comment: 7 pages, 6 figures, IROS 2022 conference

We extend Ratner's theorem on equidistribution of individual orbits of unipotent flows on finite volume homogeneous spaces of Lie groups to trajectories of non-contracting curves definable in polynomially bounded o-minimal structures. To be precise, let $\varphi:[0,\infty)\to \text{SL}(n,\mathbb R)$ be a continuous map whose coordinate functions are definable in a polynomially bounded o-minimal structure; for example, rational functions. Suppose that $\varphi$ is non-contracting; that is, for any linearly independent vectors $v_1,\ldots,v_k$ in $\mathbb R^n$, $\varphi(t).(v_1\wedge\cdots\wedge v_k)\not\to0$ as $t\to\infty$. Then, there exists a unique smallest subgroup $H_\varphi$ of $\text{SL}(n,\mathbb R)$ generated by unipotent one-parameter subgroups such that $\varphi(t)H_\varphi\to g_0H_\varphi$ in $\text{SL}(n,\mathbb R)/H_\varphi$ as $t\to\infty$ for some $g_0\in \text{SL}(n,\mathbb R)$. Let $G$ be a closed subgroup of $\text{SL}(n,\mathbb R)$ and $\Gamma$ be a lattice in $G$. Suppose that $\varphi([0,\infty))\subset G$. Then $H_\varphi\subset G$, and for any $x\in G/\Gamma$, the trajectory $\{\varphi(t)x:t\in [0,T]\}$ gets equidistributed with respect to the measure $g_0\mu_{Lx}$ as $T\to\infty$, where $L$ is a closed subgroup of $G$ such that $\overline{Hx}=Lx$ and $Lx$ admits a unique $L$-invariant probability measure, denoted by $\mu_{Lx}$. A crucial new ingredient in this work is proving that for any finite-dimensional representation $V$ of $\text{SL}(n,\mathbb R)$, there exist $T_0>0$, $C>0$, and $\alpha>0$ such that for any $v\in G$, the map $t\mapsto \|\varphi(t)v\|$ is $(C,\alpha)$-good on $[T_0,\infty)$.

We construct a high sensitivity radon pilot detector using liquid scintillator dissolved radon for the CDEX rare-event searches program. The CDEX-10 project employs a germanium detector array immersed in a massive liquid nitrogen tank. However, radon emanated from the surface of the tank can contribute background. As a result, radon contamination in the liquid nitrogen tank must be regulated and monitored, which necessitates the use of a low-level Rn measuring device. The radon-dissolved liquid scintillator (LS) detectors utilizes cascade decay selections, yielding a unique signature. All background events occurring in the LS, with regard to the system and co-efficiency of Rn absorption, have been studied. Background activity is measured to be 59.8$\pm$18.4 $\mu$Bq per 300 mL LS. Meanwhile, it shows that radon concentration absorbed by LS will be saturated and is linearly related to radon concentration in nitrogen gas. Furthermore, the co-efficiency factor can be optimized by lowering temperature or raising pressure. Eventually the detection limit for 222^Rn in nitrogen gas is observed to be 9.6 mBq/m^3., Comment: 8 pages, 18 figures

We study the limiting distribution of dense orbits of a lattice subgroup $\Gamma\le \text{SL}(m+1,\mathbb{R})$ acting on $H\backslash\text{SL}(m+1,\mathbb{R})$, with respect to a filtration of growing norm balls. The novelty of our work is that the groups $H$ we consider have infinitely many non-trivial connected components. For a specific such $H$, the homogeneous space $H\backslash G$ identifies with $X_{m,m+1}$, a moduli space of rank $m$-discrete subgroups in $\mathbb{R}^{m+1}$. This study is motivated by the work of Shapira-Sargent who studied random walks on $X_{2,3}$., Comment: All comments are welcome!

We study the distribution of orbits of a lattice $\Gamma\leq\text{SL}(3,\mathbb R)$ in the moduli space $X_{2,3}$ of covolume one rank-two discrete subgroups in $\mathbb R^3$. Each orbit is dense, and our main result is the limiting distribution of these orbits with respect to norm balls, where the norm is given by the sum of squares. Specifically, we consider $\Gamma_T=\{\gamma\in\Gamma:\|\gamma\|\leq T\}$ and show that, for any fixed $x_0\in X_{2,3}$ and $\varphi\in C_c(X_{2,3})$, $$\lim_{T\to\infty}\frac{1}{\#\Gamma_T}\sum_{\gamma\in\Gamma_T}\varphi(x_0\cdot\gamma)=\int_{X_{2,3}}\varphi(x)d \tilde\nu_{x_0}(x),$$ where $\tilde\nu_{x_0}$ is an explicit probability measure on $X_{2,3}$ depending on $x_0$. To prove our result, we use the duality principle developed by Gorodnik and Weiss which recasts the above problem into the problem of computation of certain volume estimates of growing skewed balls in $H$ and proving ergodic theorems of the left action of the skewed balls on $\text{SL}(3,\mathbb{R})/\Gamma$. The ergodic theorems are proven by applying theorems of Shah building on the linearisation technique. The main contribution of the paper is the application of the duality principle in the case where $H$ has infinitely many non-compact connected components., Comment: More details of proofs added. Some treatments of proofs in this article are different from arXiv:2307.12085

In this article we introduce the notion of badly approximable matrices of higher order using higher sucessive minima in $\mathbb R^d$. We prove that for order less than $d$, they have Lebesgue measure zero and the gaps between them still have full Hausdorff dimension., Comment: arXiv admin note: text overlap with arXiv:1901.06602 by other authors

In this article, we study the relation between lattice basis and successive minima and give an estimate for the measure-theoretical distribution of successive minima. As consequences, we also discuss some logarithm laws associated to higher sucessive minima.

Constructing high-quality character image datasets is challenging because real-world images are often affected by image degradation. There are limitations when applying current image restoration methods to such real-world character images, since (i) the categories of noise in character images are different from those in general images; (ii) real-world character images usually contain more complex image degradation, e.g., mixed noise at different noise levels. To address these problems, we propose a real-world character restoration network (RCRN) to effectively restore degraded character images, where character skeleton information and scale-ensemble feature extraction are utilized to obtain better restoration performance. The proposed method consists of a skeleton extractor (SENet) and a character image restorer (CiRNet). SENet aims to preserve the structural consistency of the character and normalize complex noise. Then, CiRNet reconstructs clean images from degraded character images and their skeletons. Due to the lack of benchmarks for real-world character image restoration, we constructed a dataset containing 1,606 character images with real-world degradation to evaluate the validity of the proposed method. The experimental results demonstrate that RCRN outperforms state-of-the-art methods quantitatively and qualitatively., Comment: Accepted to ACM MM 2022

Supervised learning depth estimation methods can achieve good performance when trained on high-quality ground-truth, like LiDAR data. However, LiDAR can only generate sparse 3D maps which causes losing information. Obtaining high-quality ground-truth depth data per pixel is difficult to acquire. In order to overcome this limitation, we propose a novel approach combining structure information from a promising Plane and Parallax geometry pipeline with depth information into a U-Net supervised learning network, which results in quantitative and qualitative improvement compared to existing popular learning-based methods. In particular, the model is evaluated on two large-scale and challenging datasets: KITTI Vision Benchmark and Cityscapes dataset and achieve the best performance in terms of relative error. Compared with pure depth supervision models, our model has impressive performance on depth prediction of thin objects and edges, and compared to structure prediction baseline, our model performs more robustly., Comment: Pattern Recognition, May 2022

In skeleton-based action recognition, Graph Convolutional Networks model human skeletal joints as vertices and connect them through an adjacency matrix, which can be seen as a local attention mask. However, in most existing Graph Convolutional Networks, the local attention mask is defined based on natural connections of human skeleton joints and ignores the dynamic relations for example between head, hands and feet joints. In addition, the attention mechanism has been proven effective in Natural Language Processing and image description, which is rarely investigated in existing methods. In this work, we proposed a new adaptive spatial attention layer that extends local attention map to global based on relative distance and relative angle information. Moreover, we design a new initial graph adjacency matrix that connects head, hands and feet, which shows visible improvement in terms of action recognition accuracy. The proposed model is evaluated on two large-scale and challenging datasets in the field of human activities in daily life: NTU-RGB+D and Kinetics skeleton. The results demonstrate that our model has strong performance on both dataset., Comment: 26th International Conference on Pattern Recognition, 2022

Social recommendation which aims to leverage social connections among users to enhance the recommendation performance. With the revival of deep learning techniques, many efforts have been devoted to developing various neural network-based social recommender systems, such as attention mechanisms and graph-based message passing frameworks. However, two important challenges have not been well addressed yet: (i) Most of existing social recommendation models fail to fully explore the multi-type user-item interactive behavior as well as the underlying cross-relational inter-dependencies. (ii) While the learned social state vector is able to model pair-wise user dependencies, it still has limited representation capacity in capturing the global social context across users. To tackle these limitations, we propose a new Social Recommendation framework with Hierarchical Graph Neural Networks (SR-HGNN). In particular, we first design a relation-aware reconstructed graph neural network to inject the cross-type collaborative semantics into the recommendation framework. In addition, we further augment SR-HGNN with a social relation encoder based on the mutual information learning paradigm between low-level user embeddings and high-level global representation, which endows SR-HGNN with the capability of capturing the global social contextual signals. Empirical results on three public benchmarks demonstrate that SR-HGNN significantly outperforms state-of-the-art recommendation methods. Source codes are available at: https://github.com/xhcdream/SR-HGNN., Comment: Published as a full paper at ICDM 2020

Social recommendation task aims to predict users' preferences over items with the incorporation of social connections among users, so as to alleviate the sparse issue of collaborative filtering. While many recent efforts show the effectiveness of neural network-based social recommender systems, several important challenges have not been well addressed yet: (i) The majority of models only consider users' social connections, while ignoring the inter-dependent knowledge across items; (ii) Most of existing solutions are designed for singular type of user-item interactions, making them infeasible to capture the interaction heterogeneity; (iii) The dynamic nature of user-item interactions has been less explored in many social-aware recommendation techniques. To tackle the above challenges, this work proposes a Knowledge-aware Coupled Graph Neural Network (KCGN) that jointly injects the inter-dependent knowledge across items and users into the recommendation framework. KCGN enables the high-order user- and item-wise relation encoding by exploiting the mutual information for global graph structure awareness. Additionally, we further augment KCGN with the capability of capturing dynamic multi-typed user-item interactive patterns. Experimental studies on real-world datasets show the effectiveness of our method against many strong baselines in a variety of settings. Source codes are available at: https://github.com/xhcdream/KCGN., Comment: Published as a paper at AAAI 2021