Search

Event cameras are biologically inspired sensors that emit events asynchronously with remarkable temporal resolution, garnering significant attention from both industry and academia. Mainstream methods favor frame and voxel representations, which reach a satisfactory performance while introducing time-consuming transformation, bulky models, and sacrificing fine-grained temporal information. Alternatively, Point Cloud representation demonstrates promise in addressing the mentioned weaknesses, but it ignores the polarity information, and its models have limited proficiency in abstracting long-term events' features. In this paper, we propose a frequency-aware network named FECNet that leverages Event Cloud representations. FECNet fully utilizes 2S-1T-1P Event Cloud by innovating the event-based Group and Sampling module. To accommodate the long sequence events from Event Cloud, FECNet embraces feature extraction in the frequency domain via the Fourier transform. This approach substantially extinguishes the explosion of Multiply Accumulate Operations (MACs) while effectively abstracting spatial-temporal features. We conducted extensive experiments on event-based object classification, action recognition, and human pose estimation tasks, and the results substantiate the effectiveness and efficiency of FECNet., Comment: Under Review

Conventional frame-based cameras inevitably produce blurry effects due to motion occurring during the exposure time. Event camera, a bio-inspired sensor offering continuous visual information could enhance the deblurring performance. Effectively utilizing the high-temporal-resolution event data is crucial for extracting precise motion information and enhancing deblurring performance. However, existing event-based image deblurring methods usually utilize voxel-based event representations, losing the fine-grained temporal details that are mathematically essential for fast motion deblurring. In this paper, we first introduce point cloud-based event representation into the image deblurring task and propose a Multi-Temporal Granularity Network (MTGNet). It combines the spatially dense but temporally coarse-grained voxel-based event representation and the temporally fine-grained but spatially sparse point cloud-based event. To seamlessly integrate such complementary representations, we design a Fine-grained Point Branch. An Aggregation and Mapping Module (AMM) is proposed to align the low-level point-based features with frame-based features and an Adaptive Feature Diffusion Module (AFDM) is designed to manage the resolution discrepancies between event data and image data by enriching the sparse point feature. Extensive subjective and objective evaluations demonstrate that our method outperforms current state-of-the-art approaches on both synthetic and real-world datasets., Comment: 12 pages, 8 figures

Recently, there is growing demand for effective and efficient long sequence modeling, with State Space Models (SSMs) proving to be effective for long sequence tasks. To further reduce energy consumption, SSMs can be adapted to Spiking Neural Networks (SNNs) using spiking functions. However, current spiking-formalized SSMs approaches still rely on float-point matrix-vector multiplication during inference, undermining SNNs' energy advantage. In this work, we address the efficiency and performance challenges of long sequence learning in SNNs simultaneously. First, we propose a decoupled reset method for parallel spiking neuron training, reducing the typical Leaky Integrate-and-Fire (LIF) model's training time from $O(L^2)$ to $O(L\log L)$, effectively speeding up the training by $6.57 \times$ to $16.50 \times$ on sequence lengths $1,024$ to $32,768$. To our best knowledge, this is the first time that parallel computation with a reset mechanism is implemented achieving equivalence to its sequential counterpart. Secondly, to capture long-range dependencies, we propose a Parallel Resonate and Fire (PRF) neuron, which leverages an oscillating membrane potential driven by a resonate mechanism from a differentiable reset function in the complex domain. The PRF enables efficient long sequence learning while maintaining parallel training. Finally, we demonstrate that the proposed spike-driven architecture using PRF achieves performance comparable to Structured SSMs (S4), with two orders of magnitude reduction in energy consumption, outperforming Transformer on Long Range Arena tasks., Comment: arXiv admin note: text overlap with arXiv:2208.04933 by other authors

Although deep learning based models for underwater image enhancement have achieved good performance, they face limitations in both lightweight and effectiveness, which prevents their deployment and application on resource-constrained platforms. Moreover, most existing deep learning based models use data compression to get high-level semantic information in latent space instead of using the original information. Therefore, they require decoder blocks to generate the details of the output. This requires additional computational cost. In this paper, a lightweight network named lightweight selective attention network (LSNet) based on the top-k selective attention and transmission maps mechanism is proposed. The proposed model achieves a PSNR of 97\% with only 7K parameters compared to a similar attention-based model. Extensive experiments show that the proposed LSNet achieves excellent performance in state-of-the-art models with significantly fewer parameters and computational resources. The code is available at https://github.com/FuhengZhou/LSNet}{https://github.com/FuhengZhou/LSNet., Comment: 10 pages

Event cameras, drawing inspiration from biological systems, efficiently detect changes in ambient light with low latency and high dynamic range while consuming minimal power. The most current approach to processing event data often involves converting it into frame-based representations, which is well-established in traditional vision. However, this approach neglects the sparsity of event data, loses fine-grained temporal information during the transformation process, and increases the computational burden, making it ineffective for characterizing event camera properties. In contrast, Point Cloud is a popular representation for 3D processing and is better suited to match the sparse and asynchronous nature of the event camera. Nevertheless, despite the theoretical compatibility of point-based methods with event cameras, the results show a performance gap that is not yet satisfactory compared to frame-based methods. In order to bridge the performance gap, we propose EventMamba, an efficient and effective Point Cloud framework that achieves competitive results even compared to the state-of-the-art (SOTA) frame-based method in both classification and regression tasks. This notable accomplishment is facilitated by our rethinking of the distinction between Event Cloud and Point Cloud, emphasizing effective temporal information extraction through optimized network structures. Specifically, EventMamba leverages temporal aggregation and State Space Model (SSM) based Mamba boasting enhanced temporal information extraction capabilities. Through a hierarchical structure, EventMamba is adept at abstracting local and global spatial features and implicit and explicit temporal features. By adhering to the lightweight design principle, EventMamba delivers impressive results with minimal computational resource utilization, demonstrating its efficiency and effectiveness., Comment: Extension Journal of TTPOINT and PEPNet, modify the dataset split method

Event cameras exhibit remarkable attributes such as high dynamic range, asynchronicity, and low latency, making them highly suitable for vision tasks that involve high-speed motion in challenging lighting conditions. These cameras implicitly capture movement and depth information in events, making them appealing sensors for Camera Pose Relocalization (CPR) tasks. Nevertheless, existing CPR networks based on events neglect the pivotal fine-grained temporal information in events, resulting in unsatisfactory performance. Moreover, the energy-efficient features are further compromised by the use of excessively complex models, hindering efficient deployment on edge devices. In this paper, we introduce PEPNet, a simple and effective point-based network designed to regress six degrees of freedom (6-DOFs) event camera poses. We rethink the relationship between the event camera and CPR tasks, leveraging the raw Point Cloud directly as network input to harness the high-temporal resolution and inherent sparsity of events. PEPNet is adept at abstracting the spatial and implicit temporal features through hierarchical structure and explicit temporal features by Attentive Bi-directional Long Short-Term Memory (A-Bi-LSTM). By employing a carefully crafted lightweight design, PEPNet delivers state-of-the-art (SOTA) performance on both indoor and outdoor datasets with meager computational resources. Specifically, PEPNet attains a significant 38% and 33% performance improvement on the random split IJRR and M3ED datasets, respectively. Moreover, the lightweight design version PEPNet$_{tiny}$ accomplishes results comparable to the SOTA while employing a mere 0.5% of the parameters., Comment: Accepted by CVPR 2024

Spiking neural networks (SNNs) are promising brain-inspired energy-efficient models. Compared to conventional deep Artificial Neural Networks (ANNs), SNNs exhibit superior efficiency and capability to process temporal information. However, it remains a challenge to train SNNs due to their undifferentiable spiking mechanism. The surrogate gradients method is commonly used to train SNNs, but often comes with an accuracy disadvantage over ANNs counterpart. We link the degraded accuracy to the vanishing of gradient on the temporal dimension through the analytical and experimental study of the training process of Leaky Integrate-and-Fire (LIF) Neuron-based SNNs. Moreover, we propose the Complementary Leaky Integrate-and-Fire (CLIF) Neuron. CLIF creates extra paths to facilitate the backpropagation in computing temporal gradient while keeping binary output. CLIF is hyperparameter-free and features broad applicability. Extensive experiments on a variety of datasets demonstrate CLIF's clear performance advantage over other neuron models. Furthermore, the CLIF's performance even slightly surpasses superior ANNs with identical network structure and training conditions. The code is available at https://github.com/HuuYuLong/Complementary-LIF.

Event cameras are bio-inspired sensors that respond to local changes in light intensity and feature low latency, high energy efficiency, and high dynamic range. Meanwhile, Spiking Neural Networks (SNNs) have gained significant attention due to their remarkable efficiency and fault tolerance. By synergistically harnessing the energy efficiency inherent in event cameras and the spike-based processing capabilities of SNNs, their integration could enable ultra-low-power application scenarios, such as action recognition tasks. However, existing approaches often entail converting asynchronous events into conventional frames, leading to additional data mapping efforts and a loss of sparsity, contradicting the design concept of SNNs and event cameras. To address this challenge, we propose SpikePoint, a novel end-to-end point-based SNN architecture. SpikePoint excels at processing sparse event cloud data, effectively extracting both global and local features through a singular-stage structure. Leveraging the surrogate training method, SpikePoint achieves high accuracy with few parameters and maintains low power consumption, specifically employing the identity mapping feature extractor on diverse datasets. SpikePoint achieves state-of-the-art (SOTA) performance on four event-based action recognition datasets using only 16 timesteps, surpassing other SNN methods. Moreover, it also achieves SOTA performance across all methods on three datasets, utilizing approximately 0.3\% of the parameters and 0.5\% of power consumption employed by artificial neural networks (ANNs). These results emphasize the significance of Point Cloud and pave the way for many ultra-low-power event-based data processing applications., Comment: Accepted by ICLR 2024 (Spotlight)

Event cameras have gained popularity in computer vision due to their data sparsity, high dynamic range, and low latency. As a bio-inspired sensor, event cameras generate sparse and asynchronous data, which is inherently incompatible with the traditional frame-based method. Alternatively, the point-based method can avoid additional modality transformation and naturally adapt to the sparsity of events. Still, it typically cannot reach a comparable accuracy as the frame-based method. We propose a lightweight and generalized point cloud network called TTPOINT which achieves competitive results even compared to the state-of-the-art (SOTA) frame-based method in action recognition tasks while only using 1.5 % of the computational resources. The model is adept at abstracting local and global geometry by hierarchy structure. By leveraging tensor-train compressed feature extractors, TTPOINT can be designed with minimal parameters and computational complexity. Additionally, we developed a straightforward downsampling algorithm to maintain the spatio-temporal feature. In the experiment, TTPOINT emerged as the SOTA method on three datasets while also attaining SOTA among point cloud methods on all five datasets. Moreover, by using the tensor-train decomposition method, the accuracy of the proposed TTPOINT is almost unaffected while compressing the parameter size by 55 % in all five datasets.

Sequential recommender models typically generate predictions in a single step during testing, without considering additional prediction correction to enhance performance as humans would. To improve the accuracy of these models, some researchers have attempted to simulate human analogical reasoning to correct predictions for testing data by drawing analogies with the prediction errors of similar training data. However, there are inherent gaps between testing and training data, which can make this approach unreliable. To address this issue, we propose an \textit{Abductive Prediction Correction} (APC) framework for sequential recommendation. Our approach simulates abductive reasoning to correct predictions. Specifically, we design an abductive reasoning task that infers the most probable historical interactions from the future interactions predicted by a recommender, and minimizes the discrepancy between the inferred and true historical interactions to adjust the predictions.We perform the abductive inference and adjustment using a reversed sequential model in the forward and backward propagation manner of neural networks. Our APC framework is applicable to various differentiable sequential recommender models. We implement it on three backbone models and demonstrate its effectiveness. We release the code at https://github.com/zyang1580/APC.

The defunctionalization translation that eliminates higher-order functions from programs forms a key part of many compilers. However, defunctionalization for dependently-typed languages has not been formally studied. We present the first formally-specified defunctionalization translation for a dependently-typed language and establish key metatheoretical properties such as soundness and type preservation. The translation is suitable for incorporation into type-preserving compilers for dependently-typed languages

The phenomenon of phase separation into antiferromagnetic (AFM) and superconducting (SC) or normal-state regions has great implication for the origin of high-temperature (high-Tc) superconductivity. However, the occurrence of an intrinsic antiferromagnetism above the Tc of (Li, Fe)OHFeSe superconductor is questioned. Here we report a systematic study on a series of (Li, Fe)OHFeSe single crystal samples with Tc up to ~41 K. We observe an evident drop in the static magnetization at Tafm ~125 K, in some of the SC (Tc < ~38 K, cell parameter c < ~9.27 {\AA}) and non-SC samples. We verify that this AFM signal is intrinsic to (Li, Fe)OHFeSe. Thus, our observations indicate mesoscopic-to-macroscopic coexistence of an AFM state with the normal (below Tafm) or SC (below Tc) state in (Li, Fe)OHFeSe. We explain such coexistence by electronic phase separation, similar to that in high-Tc cuprates and iron arsenides. However, such an AFM signal can be absent in some other samples of (Li, Fe)OHFeSe, particularly it is never observed in the SC samples of Tc > ~38 K, owing to a spatial scale of the phase separation too small for the macroscopic magnetic probe. For this case, we propose a microscopic electronic phase separation. It is suggested that the microscopic static phase separation reaches vanishing point in high-Tc (Li, Fe)OHFeSe, by the occurrence of two-dimensional AFM spin fluctuations below nearly the same temperature as Tafm reported previously for a (Li, Fe)OHFeSe (Tc ~42 K) single crystal. A complete phase diagram is thus established. Our study provides key information of the underlying physics for high-Tc superconductivity., Comment: 13 pages, 5 figures

The superconducting film of (Li1-xFex)OHFeSe is reported for the first time. The thin film exhibits a small in-plane crystal mosaic of 0.22 deg, in terms of the FWHM (full-width-at-half-maximum) of x-ray rocking curve, and an excellent out-of-plane orientation by x-ray phi-scan. Its bulk superconducting transition temperature (Tc) of 42.4 K is characterized by both zero electrical resistance and diamagnetization measurements. The upper critical field (Hc2) is estimated to be 79.5 T and 443 T, respectively, for the magnetic field perpendicular and parallel to the ab plane. Moreover, a large critical current density (Jc) of a value over 0.5 MA/cm2 is achieved at ~20 K. Such a (Li1-xFex)OHFeSe film is therefore not only important to the fundamental research for understanding the high-Tc mechanism, but also promising in the field of high-Tc superconductivity application, especially in high-performance electronic devices and large scientific facilities such as superconducting accelerator., Comment: 9 pages, 4 figures

Stabilized FeSe thin films in ambient pressure with tunable superconductivity would be a promising candidate for superconducting electronic devices yet its superconducting transition temperature (Tc) is below 10 K in bulk materials. By carefully controlling the depositions on twelve kinds of substrates using pulsed laser deposition technique, high quality single crystalline FeSe samples were fabricated with full width of half maximum 0.515? in the rocking curve and clear four-fold symmetry in phi-scan from x-ray diractions. The films have a maximum Tc 15 K on the CaF2 substrate and do not show obvious decay in the air for more than half a year. Slightly tuning the stoichiometry of the FeSe targets, the Tc becomes adjustable from 15 to < 2 K with quite narrow transition widths less than 2 K, and shows a positive relation with the out-of-plane (c-axis) lattice parameter of the films. However, there is no clear relation between the Tc and the surface atomic distance of the substrates. By reducing the thickness of the films, the Tc decreases and fades away in samples of less than 10 nm, suggesting that the strain effect is not responsible for the enhancement of Tc in our experiments., Comment: 5 pages, 4 figures

We report the success in introducing Mn into (Li1-xFex)OHFe1-ySe superconducting crystals by applying two different hydrothermal routes, ion exchange (1-Step) and ion release/introduction (2-Step). The micro-region x-ray diffraction and energy dispersive x-ray spectroscopy analyses indicate that the Mn has been doped into the lattice, and its content in the 1-Step fabricated sample is higher than that in the 2-Step one. Magnetic susceptibility and electric transport properties reveal that Mn doping influences little on the superconducting transition, regardless of 1-Step or 2-Step routes. By contrast, the characteristic temperature, T*, where the negative Hall coefficient reaches its minimum, is significantly reduced by Mn doping. This implies that the reduction of the hole carriers contribution is obviously modified, and hence the hole band might have no direct relationship with the superconductivity in (Li1-xFex)OHFe1-ySe superconductors. Our present hydrothermal methods of ion exchange and ion release/introduction provide an efficient way for elements substitution/doping into (Li1-xFex)OHFe1-ySe superconductors, which will promote the in-depth investigations on the role of multiple electron and hole bands and their interplay with the high-temperature superconductivity in the FeSe-based superconductors., Comment: 8 pages, 5 figures, accept by Chinese Physics B

In this paper, a new robust Student's t based stochastic cubature filter (RSTSCF) is proposed for nonlinear state-space model with heavy-tailed process and measurement noises. The heart of the RSTSCF is a stochastic Student's t spherical radial cubature rule (SSTSRCR), which is derived based on the third-degree unbiased spherical rule and the proposed third-degree unbiased radial rule. The existing stochastic integration rule is a special case of the proposed SSTSRCR when the degrees of freedom parameter tends to infinity. The proposed filter is applied to a manoeuvring bearings-only tracking example, where an agile target is tracked and the bearing is observed in clutter. Simulation results show that the proposed RSTSCF can achieve higher estimation accuracy than the existing Gaussian approximate filter, Gaussian sum filter, Huber-based nonlinear Kalman filter, maximum correntropy criterion based Kalman filter, and robust Student's t based nonlinear filters, and is computationally much more efficient than the existing particle filter., Comment: 9 pages, 2 figures