Search

This paper reports on the NTIRE 2023 Quality Assessment of Video Enhancement Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2023. This challenge is to address a major challenge in the field of video processing, namely, video quality assessment (VQA) for enhanced videos. The challenge uses the VQA Dataset for Perceptual Video Enhancement (VDPVE), which has a total of 1211 enhanced videos, including 600 videos with color, brightness, and contrast enhancements, 310 videos with deblurring, and 301 deshaked videos. The challenge has a total of 167 registered participants. 61 participating teams submitted their prediction results during the development phase, with a total of 3168 submissions. A total of 176 submissions were submitted by 37 participating teams during the final testing phase. Finally, 19 participating teams submitted their models and fact sheets, and detailed the methods they used. Some methods have achieved better results than baseline methods, and the winning methods have demonstrated superior prediction performance.

Multimodal learning has seen great success mining data features from multiple modalities with remarkable model performance improvement. Meanwhile, federated learning (FL) addresses the data sharing problem, enabling privacy-preserved collaborative training to provide sufficient precious data. Great potential, therefore, arises with the confluence of them, known as multimodal federated learning. However, limitation lies in the predominant approaches as they often assume that each local dataset records samples from all modalities. In this paper, we aim to bridge this gap by proposing an Unimodal Training - Multimodal Prediction (UTMP) framework under the context of multimodal federated learning. We design HA-Fedformer, a novel transformer-based model that empowers unimodal training with only a unimodal dataset at the client and multimodal testing by aggregating multiple clients' knowledge for better accuracy. The key advantages are twofold. Firstly, to alleviate the impact of data non-IID, we develop an uncertainty-aware aggregation method for the local encoders with layer-wise Markov Chain Monte Carlo sampling. Secondly, to overcome the challenge of unaligned language sequence, we implement a cross-modal decoder aggregation to capture the hidden signal correlation between decoders trained by data from different modalities. Our experiments on popular sentiment analysis benchmarks, CMU-MOSI and CMU-MOSEI, demonstrate that HA-Fedformer significantly outperforms state-of-the-art multimodal models under the UTMP federated learning frameworks, with 15%-20% improvement on most attributes., Comment: 10 pages,5 figures

Vision-Centric Bird-Eye-View (BEV) perception has shown promising potential and attracted increasing attention in autonomous driving. Recent works mainly focus on improving efficiency or accuracy but neglect the domain shift problem, resulting in severe degradation of transfer performance. With extensive observations, we figure out the significant domain gaps existing in the scene, weather, and day-night changing scenarios and make the first attempt to solve the domain adaption problem for multi-view 3D object detection. Since BEV perception approaches are usually complicated and contain several components, the domain shift accumulation on multi-latent spaces makes BEV domain adaptation challenging. In this paper, we propose a novel Multi-level Multi-space Alignment Teacher-Student ($M^{2}ATS$) framework to ease the domain shift accumulation, which consists of a Depth-Aware Teacher (DAT) and a Multi-space Feature Aligned (MFA) student model. Specifically, DAT model adopts uncertainty guidance to sample reliable depth information in target domain. After constructing domain-invariant BEV perception, it then transfers pixel and instance-level knowledge to student model. To further alleviate the domain shift at the global level, MFA student model is introduced to align task-relevant multi-space features of two domains. To verify the effectiveness of $M^{2}ATS$, we conduct BEV 3D object detection experiments on four cross domain scenarios and achieve state-of-the-art performance (e.g., +12.6% NDS and +9.1% mAP on Day-Night). Code and dataset will be released.

Face anti-spoofing researches are widely used in face recognition and has received more attention from industry and academics. In this paper, we propose the EulerNet, a new temporal feature fusion network in which the differential filter and residual pyramid are used to extract and amplify abnormal clues from continuous frames, respectively. A lightweight sample labeling method based on face landmarks is designed to label large-scale samples at a lower cost and has better results than other methods such as 3D camera. Finally, we collect 30,000 live and spoofing samples using various mobile ends to create a dataset that replicates various forms of attacks in a real-world setting. Extensive experiments on public OULU-NPU show that our algorithm is superior to the state of art and our solution has already been deployed in real-world systems servicing millions of users., 6 pages,7 figures,The 34th International Conference on Software Engineering & Knowledge Engineering

Post-acute sequelae of COVID-19 (PASC) represent an emerging global crisis. However, quantifiable risk factors for PASC and their biological associations are poorly resolved. We executed a deep multi-omic, longitudinal investigation of 309 COVID-19 patients from initial diagnosis to convalescence (2-3months later), integrated with clinical data and patient-reported symptoms. We resolved four PASC-anticipating risk factors at the time of initial COVID-19 diagnosis: type 2 diabetes, SARS-CoV-2 RNAemia, Epstein-Barr virus viremia, and specific auto-antibodies. In patients with gastrointestinal PASC, SARS-CoV-2-specific and CMV-specific CD8+ Tcells exhibited unique dynamics during recovery from COVID-19. Analysis of symptom-associated immunological signatures revealed coordinated immunity polarization into four endotypes, exhibiting divergent acute severity and PASC. We find that immunological associations between PASC factors diminish over time, leading to distinct convalescent immune states. Detectability of most PASC factors at COVID-19 diagnosis emphasizes the importance of early disease measurements for understanding emergent chronic conditions and suggests PASC treatment strategies.

When facing changing environments in the real world, the lightweight model on client devices suffers from severe performance drops under distribution shifts. The main limitations of the existing device model lie in (1) unable to update due to the computation limit of the device, (2) the limited generalization ability of the lightweight model. Meanwhile, recent large models have shown strong generalization capability on the cloud while they can not be deployed on client devices due to poor computation constraints. To enable the device model to deal with changing environments, we propose a new learning paradigm of Cloud-Device Collaborative Continual Adaptation, which encourages collaboration between cloud and device and improves the generalization of the device model. Based on this paradigm, we further propose an Uncertainty-based Visual Prompt Adapted (U-VPA) teacher-student model to transfer the generalization capability of the large model on the cloud to the device model. Specifically, we first design the Uncertainty Guided Sampling (UGS) to screen out challenging data continuously and transmit the most out-of-distribution samples from the device to the cloud. Then we propose a Visual Prompt Learning Strategy with Uncertainty guided updating (VPLU) to specifically deal with the selected samples with more distribution shifts. We transmit the visual prompts to the device and concatenate them with the incoming data to pull the device testing distribution closer to the cloud training distribution. We conduct extensive experiments on two object detection datasets with continually changing environments. Our proposed U-VPA teacher-student framework outperforms previous state-of-the-art test time adaptation and device-cloud collaboration methods. The code and datasets will be released.

Bird's-Eye-View (BEV) 3D Object Detection is a crucial multi-view technique for autonomous driving systems. Recently, plenty of works are proposed, following a similar paradigm consisting of three essential components, i.e., camera feature extraction, BEV feature construction, and task heads. Among the three components, BEV feature construction is BEV-specific compared with 2D tasks. Existing methods aggregate the multi-view camera features to the flattened grid in order to construct the BEV feature. However, flattening the BEV space along the height dimension fails to emphasize the informative features of different heights. For example, the barrier is located at a low height while the truck is located at a high height. In this paper, we propose a novel method named BEV Slice Attention Network (BEV-SAN) for exploiting the intrinsic characteristics of different heights. Instead of flattening the BEV space, we first sample along the height dimension to build the global and local BEV slices. Then, the features of BEV slices are aggregated from the camera features and merged by the attention mechanism. Finally, we fuse the merged local and global BEV features by a transformer to generate the final feature map for task heads. The purpose of local BEV slices is to emphasize informative heights. In order to find them, we further propose a LiDAR-guided sampling strategy to leverage the statistical distribution of LiDAR to determine the heights of local slices. Compared with uniform sampling, LiDAR-guided sampling can determine more informative heights. We conduct detailed experiments to demonstrate the effectiveness of BEV-SAN. Code will be released.

We present an integrated analysis of the clinical measurements, immune cells, and plasma multi-omics of 139 COVID-19 patients representing all levels of disease severity, from serial blood draws collected during the first week of infection following diagnosis. We identify a major shift between mild and moderate disease, at which point elevated inflammatory signaling is accompanied by the loss of specific classes of metabolites and metabolic processes. Within this stressed plasma environment at moderate disease, multiple unusual immune cell phenotypes emerge and amplify with increasing disease severity. We condensed over 120,000 immune features into a single axis to capture how different immune cell classes coordinate in response to SARS-CoV-2. This immune-response axis independently aligns with the major plasma composition changes, with clinical metrics of blood clotting, and with the sharp transition between mild and moderate disease. This study suggests that moderate disease may provide the most effective setting for therapeutic intervention.

Thus far, Ultrasound has been proved to be useful for noninvasively stimulating brain activity and has hope to play a positive role in the treatment of neurological diseases. Epilepsy is a neurological disorder in which brain activity becomes abnormal, causing seizures or periods of unusual behavior, and sometimes loss of awareness. In this paper, we established a focused ultrasound (FUS) neuromodulation system for the treatment of epilepsy. We used the chemically-induced rat epilepsy model to explore the effect of pulsed focused ultrasound on epilepsy, and obtained preliminary experimental results. We have proved the feasibility of this system through experiments, which can be used in the treatment of epilepsy by ultrasound neuromodulation. Keywords: Ultrasound; Neuromodulation; epilepsy, Applied Science and Computer Science Publications

Zn2+ ions doped quasi-one-dimensional spin-chain materials LiCu1-xZnxVO4(x=0, 0.1, 0.2) were prepared by solid-state reaction and the effect of nonmagnetic Zn2+ ions on the crystal structure, valence state and magnetic properties of them were studied by X-ray diffraction(XRD), X-ray photoelectron spectroscopy(XPS) and dc magnetization. The elongation of a and b axes and the shrinkage of c axis indicate a reduced effect of the Jahn-Teller distortion after Zn2+ ions doping, while the valence state of transition metal ions in LiCu1–xZnxVO4 remains unchanged since the equivalence doping of Zn2+ ions on the Cu2+ sites. The corrected Bonner-Fisher equation fitting to the susceptibility indicates that isolated dimers and spins are created along with the departure from infinite spin chains to finite ones.