Search

The design of new lightweight and dexterous configurations is a major research focus for continuum robotics. This work proposes a cruciform continuum robot. Its unique feature is that it is formed by multiple cruciform-arranged elastic sheets with a single dimension of motion connected in series, and thus it has low-coupling motion characteristics. In addition, the cruciform continuum robot has the advantages of lighter weight (65 g), better dexterity, and higher motion accuracy. In this paper, the forward and inverse kinematics models of the cruciform continuum robot are established by geometric methods based on the assumption of constant curvature, and its workspace is analysed. It is experimentally verified that the tip position errors are less than 1 mm, and the cable length errors are less than 0.4 mm. Further, the cruciform continuum robot is successfully used for the nucleic acid detection simulation experiment, which confirms its good dexterity and man–machine safety. The main contribution of this paper is to provide a new configuration for the lightweight and dexterous continuum robots, and to further provide a reference method for improving their modelling accuracy from the perspective of structure.

Despite their success in real data synthesis, diffusion models (DMs) often suffer from slow and costly training and sampling issues, limiting their broader applications. To mitigate this, we propose a Fast Diffusion Model (FDM) which improves the diffusion process of DMs from a stochastic optimization perspective to speed up both training and sampling. Specifically, we first find that the diffusion process of DMs accords with the stochastic optimization process of stochastic gradient descent (SGD) on a stochastic time-variant problem. Note that momentum SGD uses both the current gradient and an extra momentum, achieving more stable and faster convergence. We are inspired to introduce momentum into the diffusion process to accelerate both training and sampling. However, this comes with the challenge of deriving the noise perturbation kernel from the momentum-based diffusion process. To this end, we frame the momentum-based process as a Damped Oscillation system whose critically damped state -- the kernel solution -- avoids oscillation and thus has a faster convergence speed of the diffusion process. Empirical results show that our FDM can be applied to several popular DM frameworks, e.g. VP, VE, and EDM, and reduces their training cost by about 50% with comparable image synthesis performance on CIFAR-10, FFHQ, and AFHQv2 datasets. Moreover, FDM decreases their sampling steps by about $3\times$ to achieve similar performance under the same deterministic samplers. The code is available at https://github.com/sail-sg/FDM.

Temporal sentence localization in videos (TSLV) aims to retrieve the most interested segment in an untrimmed video according to a given sentence query. However, almost of existing TSLV approaches suffer from the same limitations: (1) They only focus on either frame-level or object-level visual representation learning and corresponding correlation reasoning, but fail to integrate them both; (2) They neglect to leverage the rich semantic contexts to further benefit the query reasoning. To address these issues, in this paper, we propose a novel Hierarchical Visual- and Semantic-Aware Reasoning Network (HVSARN), which enables both visual- and semantic-aware query reasoning from object-level to frame-level. Specifically, we present a new graph memory mechanism to perform visual-semantic query reasoning: For visual reasoning, we design a visual graph memory to leverage visual information of video; For semantic reasoning, a semantic graph memory is also introduced to explicitly leverage semantic knowledge contained in the classes and attributes of video objects, and perform correlation reasoning in the semantic space. Experiments on three datasets demonstrate that our HVSARN achieves a new state-of-the-art performance., Comment: Accepted by ICASSP2023

BackgroundMyositis-specific autoantibodies (MSAs) are clinically used to diagnose and define idiopathic inflammatory myopathy (IIM) subsets. However, the underlying pathogenic mechanisms of patients with different MSAs remain unclear.MethodsA total of 158 Chinese patients with IIM and 167 gender- and age-matched healthy controls (HCs) were enrolled. Transcriptome sequencing (RNA-Seq) was performed with peripheral blood mononuclear cells (PBMCs), followed by the identification of differentially expressed genes (DEGs) and analysis of gene set enrichment analysis, immune cell infiltration, and WGCNA. Monocyte subsets and related cytokines/chemokines were quantified. The expressions of interferon (IFN)-related genes were validated using qRT-PCR and Western blot in both PBMCs and monocytes. We also performed correlation analysis and ROC analysis to explore the potential clinical significance of the IFN-related genes.ResultsThere were 1,364 genes altered in patients with IIM, including 952 upregulated and 412 downregulated genes. The type I interferon (IFN-I) pathway was remarkably activated in patients with IIM. Compared with patients with other MSAs, IFN-I signatures were significantly activated in patients with anti-melanoma differentiation-associated gene 5 (MDA5) antibodies. In total, 1,288 hub genes associated with IIM onset were identified using WGCNA, including 29 key DEGs associated with IFN signaling. The patients had more CD14brightCD16- classical, CD14brightCD16+ intermediate, and fewer CD14dimCD16+ non-classical monocyte subsets. Plasma cytokines like IL-6 and TNF and chemokines including CCL3 and MCPs increased. The validation of IFN-I-related gene expressions was consistent with the findings from RNA-Seq. The IFN-related genes were correlated with laboratory parameters and helpful for IIM diagnosis.ConclusionGene expressions were remarkably altered in the PBMCs of IIM patients. Anti-MDA5+ IIM patients had a more pronounced activated IFN signature than others. Monocytes exhibited a proinflammatory feature and contributed to the IFN signature of IIM patients.

Temporal sentence grounding (TSG) is crucial and fundamental for video understanding. Although the existing methods train well-designed deep networks with a large amount of data, we find that they can easily forget the rarely appeared cases in the training stage due to the off-balance data distribution, which influences the model generalization and leads to undesirable performance. To tackle this issue, we propose a memory-augmented network, called Memory-Guided Semantic Learning Network (MGSL-Net), that learns and memorizes the rarely appeared content in TSG tasks. Specifically, MGSL-Net consists of three main parts: a cross-modal inter-action module, a memory augmentation module, and a heterogeneous attention module. We first align the given video-query pair by a cross-modal graph convolutional network, and then utilize a memory module to record the cross-modal shared semantic features in the domain-specific persistent memory. During training, the memory slots are dynamically associated with both common and rare cases, alleviating the forgetting issue. In testing, the rare cases can thus be enhanced by retrieving the stored memories, resulting in better generalization. At last, the heterogeneous attention module is utilized to integrate the enhanced multi-modal features in both video and query domains. Experimental results on three benchmarks show the superiority of our method on both effectiveness and efficiency, which substantially improves the accuracy not only on the entire dataset but also on rare cases., Accepted by AAAI2022

This paper addresses temporal sentence grounding. Previous works typically solve this task by learning frame-level video features and align them with the textual information. A major limitation of these works is that they fail to distinguish ambiguous video frames with subtle appearance differences due to frame-level feature extraction. Recently, a few methods adopt Faster R-CNN to extract detailed object features in each frame to differentiate the fine-grained appearance similarities. However, the object-level features extracted by Faster R-CNN suffer from missing motion analysis since the object detection model lacks temporal modeling. To solve this issue, we propose a novel Motion-Appearance Reasoning Network (MARN), which incorporates both motion-aware and appearance-aware object features to better reason object relations for modeling the activity among successive frames. Specifically, we first introduce two individual video encoders to embed the video into corresponding motion-oriented and appearance-aspect object representations. Then, we develop separate motion and appearance branches to learn motion-guided and appearance-guided object relations, respectively. At last, both motion and appearance information from two branches are associated to generate more representative features for final grounding. Extensive experiments on two challenging datasets (Charades-STA and TACoS) show that our proposed MARN significantly outperforms previous state-of-the-art methods by a large margin., Accepted by AAAI2022

Given an untrimmed video, temporal sentence grounding (TSG) aims to locate a target moment semantically according to a sentence query. Although previous respectable works have made decent success, they only focus on high-level visual features extracted from the consecutive decoded frames and fail to handle the compressed videos for query modelling, suffering from insufficient representation capability and significant computational complexity during training and testing. In this paper, we pose a new setting, compressed-domain TSG, which directly utilizes compressed videos rather than fully-decompressed frames as the visual input. To handle the raw video bit-stream input, we propose a novel Three-branch Compressed-domain Spatial-temporal Fusion (TCSF) framework, which extracts and aggregates three kinds of low-level visual features (I-frame, motion vector and residual features) for effective and efficient grounding. Particularly, instead of encoding the whole decoded frames like previous works, we capture the appearance representation by only learning the I-frame feature to reduce delay or latency. Besides, we explore the motion information not only by learning the motion vector feature, but also by exploring the relations of neighboring frames via the residual feature. In this way, a three-branch spatial-temporal attention layer with an adaptive motion-appearance fusion module is further designed to extract and aggregate both appearance and motion information for the final grounding. Experiments on three challenging datasets shows that our TCSF achieves better performance than other state-of-the-art methods with lower complexity., Accepted by CVPR-23

BackgroundBy 2020, the prevalence of Obstructive Sleep Apnea Syndrome (OSAS) in the US has reached 26. 6–43.2% in men and 8.7–27.8% in women. OSAS promotes hypertension, diabetes, and tumor growth through unknown means. Chronic intermittent hypoxia (CIH), sleep fragmentation, and increased pleural pressure are central mechanisms of OSAS complications. CIH exacerbates ferroptosis, which is closely related to malignancies. The mechanism of ferroptosis in OSAS disease progression remains unknown.MethodsOSAS-related datasets (GSE135917 and GSE38792) were obtained from the GEO. Differentially expressed genes (DEGs) were screened using the R software and intersected with the ferroptosis database (FerrDb V2) to get ferroptosis-related DEGs (f-DEGs). GO, DO, KEGG, and GSEA enrichment were performed, a PPI network was constructed and hub genes were screened. The TCGA database was used to obtain the thyroid cancer (THCA) gene expression profile, and hub genes were analyzed for differential and survival analysis. The mechanism was investigated using GSEA and immune infiltration. The hub genes were validated with RT-qPCR, IHC, and other datasets. Sprague-Dawley rats were randomly separated into normoxia and CIH groups. ROS, MDA, and GSH methods were used to detect CIH-induced ferroptosis and oxidative stress.ResultsGSEA revealed a statistically significant difference in ferroptosis in OSAS (FDR < 0.05). HIF1A, ATM, HSPA5, MAPK8, MAPK14, TLR4, and CREB1 were identified as hub genes among 3,144 DEGs and 74 f-DEGs. HIF1A and ATM were the only two validated genes. F-DEGs were mainly enriched in THCA. HIF1A overexpression in THCA promotes its development. HIF1A is associated with CD8 T cells and macrophages, which may affect the immunological milieu. The result found CIH increased ROS and MDA while lowering GSH indicating that it could cause ferroptosis. In OSAS patients, non-invasive ventilation did not affect HIF1A and ATM expression. Carvedilol, hydralazine, and caffeine may be important in the treatment of OSAS since they suppress HIF1A and ATM.ConclusionsOur findings revealed that the genes HIF1A and ATM are highly expressed in OSAS, and can serve as biomarkers and targets for OSAS.

We propose to perform video question answering (VideoQA) in a Contrastive manner via a Video Graph Transformer model (CoVGT). CoVGT's uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations and dynamics, for complex spatio-temporal reasoning. 2) It designs separate video and text transformers for contrastive learning between the video and text to perform QA, instead of multi-modal transformer for answer classification. Fine-grained video-text communication is done by additional cross-modal interaction modules. 3) It is optimized by the joint fully- and self-supervised contrastive objectives between the correct and incorrect answers, as well as the relevant and irrelevant questions respectively. With superior video encoding and QA solution, we show that CoVGT can achieve much better performances than previous arts on video reasoning tasks. Its performances even surpass those models that are pretrained with millions of external data. We further show that CoVGT can also benefit from cross-modal pretraining, yet with orders of magnitude smaller data. The results demonstrate the effectiveness and superiority of CoVGT, and additionally reveal its potential for more data-efficient pretraining. We hope our success can advance VideoQA beyond coarse recognition/description towards fine-grained relation reasoning of video contents. Our code is available at https://github.com/doc-doc/CoVGT., Accepted by IEEE T-PAMI'23

A high-altitude military UAV air-to-ground scenario model is established for the battlefield scenario without base station, the MIMO antenna array technology is used to improve the channel communication performance, and the spatial statistical channel model (SSCM) of this scenario is constructed in this paper. According to the simulation results, the dependence of the channel on T-R separation distance, frequency, rain rate and antenna HPBW parameters was investigated, the large-scale fading and small-scale fading characteristics was analyzed. By calculating and analyzing the condition number and rank of the channel transmission matrix, the spatial multiplexing under different MIMO antenna arrays was investigated. The simulation results can provide a theoretical basis for future high-altitude military UAV battlefield situation awareness frequency selection and antenna design.

Molecular imaging can dynamically and quantitatively record the biochemical changes in a systemic view. In this research, SARS-CoV-2 pseudovirus was intramuscularly injected to simulate the vaccination with inactivated virus. New Zealand white rabbits were evaluated with (18)F-FDG PET for inflammation and (68)Ga-cyc-DX600 PET for ACE2 fluctuation, which were performed before and at 3, 7 and 14 days post injection (d P.I.); furthermore, one rabbit was vaccinated with two cycles with interval of 14 days for a longer period evaluation. Different with the vaccination-induced inflammatory response that was random and individual, ACE2 regulation was systemic and organ-specific: the liver and spleen were of a moderate decrease post injection but rebound at 14 d P.I., while there were a downward trend in heart, testis and bone marrow; besides, similar pattern of ACE2 regulation were recorded after the second injection with a relatively greater volatility. In conclusion, ACE2 PET gave a more comprehensive view on host response post vaccination, hold substantial promise in continuous monitoring of coronavirus vaccine administration and effectiveness.

Given an untrimmed video, temporal sentence localization (TSL) aims to localize a specific segment according to a given sentence query. Though respectable works have made decent achievements in this task, they severely rely on dense video frame annotations, which require a tremendous amount of human effort to collect. In this paper, we target another more practical and challenging setting: one-shot temporal sentence localization (one-shot TSL), which learns to retrieve the query information among the entire video with only one annotated frame. Particularly, we propose an effective and novel tree-structure baseline for one-shot TSL, called Multiple Hypotheses Segment Tree (MHST), to capture the query-aware discriminative frame-wise information under the insufficient annotations. Each video frame is taken as the leaf-node, and the adjacent frames sharing the same visual-linguistic semantics will be merged into the upper non-leaf node for tree building. At last, each root node is an individual segment hypothesis containing the consecutive frames of its leaf-nodes. During the tree construction, we also introduce a pruning strategy to eliminate the interference of query-irrelevant nodes. With our designed self-supervised loss functions, our MHST is able to generate high-quality segment hypotheses for ranking and selection with the query. Experiments on two challenging datasets demonstrate that MHST achieves competitive performance compared to existing methods., Accepted by AAAI2023

While the use of graph-structured data in various fields is becoming increasingly popular, it also raises concerns about the potential unauthorized exploitation of personal data for training commercial graph neural network (GNN) models, which can compromise privacy. To address this issue, we propose a novel method for generating unlearnable graph examples. By injecting delusive but imperceptible noise into graphs using our Error-Minimizing Structural Poisoning (EMinS) module, we are able to make the graphs unexploitable. Notably, by modifying only $5\%$ at most of the potential edges in the graph data, our method successfully decreases the accuracy from ${77.33\%}$ to ${42.47\%}$ on the COLLAB dataset., Comment: This paper is accepted as a poster for NDSS 2023

End-to-end training with global optimization have popularized graph neural networks (GNNs) for node classification, yet inadvertently introduced vulnerabilities to adversarial edge-perturbing attacks. Adversaries can exploit the inherent opened interfaces of GNNs' input and output, perturbing critical edges and thus manipulating the classification results. Current defenses, due to their persistent utilization of global-optimization-based end-to-end training schemes, inherently encapsulate the vulnerabilities of GNNs. This is specifically evidenced in their inability to defend against targeted secondary attacks. In this paper, we propose the Graph Agent Network (GAgN) to address the aforementioned vulnerabilities of GNNs. GAgN is a graph-structured agent network in which each node is designed as an 1-hop-view agent. Through the decentralized interactions between agents, they can learn to infer global perceptions to perform tasks including inferring embeddings, degrees and neighbor relationships for given nodes. This empowers nodes to filtering adversarial edges while carrying out classification tasks. Furthermore, agents' limited view prevents malicious messages from propagating globally in GAgN, thereby resisting global-optimization-based secondary attacks. We prove that single-hidden-layer multilayer perceptrons (MLPs) are theoretically sufficient to achieve these functionalities. Experimental results show that GAgN effectively implements all its intended capabilities and, compared to state-of-the-art defenses, achieves optimal classification accuracy on the perturbed datasets.

Recently, ChatGPT has gained significant attention in research due to its ability to interact with humans effectively. The core idea behind this model is reinforcement learning (RL) fine-tuning, a new paradigm that allows language models to align with human preferences, i.e., InstructGPT. In this study, we propose BadGPT, the first backdoor attack against RL fine-tuning in language models. By injecting a backdoor into the reward model, the language model can be compromised during the fine-tuning stage. Our initial experiments on movie reviews, i.e., IMDB, demonstrate that an attacker can manipulate the generated text through BadGPT., Comment: This paper is accepted as a poster in NDSS2023

Existing methods of privacy-preserving action recognition (PPAR) mainly focus on frame-level (spatial) privacy removal through 2D CNNs. Unfortunately, they have two major drawbacks. First, they may compromise temporal dynamics in input videos, which are critical for accurate action recognition. Second, they are vulnerable to practical attacking scenarios where attackers probe for privacy from an entire video rather than individual frames. To address these issues, we propose a novel framework STPrivacy to perform video-level PPAR. For the first time, we introduce vision Transformers into PPAR by treating a video as a tubelet sequence, and accordingly design two complementary mechanisms, i.e., sparsification and anonymization, to remove privacy from a spatio-temporal perspective. In specific, our privacy sparsification mechanism applies adaptive token selection to abandon action-irrelevant tubelets. Then, our anonymization mechanism implicitly manipulates the remaining action-tubelets to erase privacy in the embedding space through adversarial learning. These mechanisms provide significant advantages in terms of privacy preservation for human eyes and action-privacy trade-off adjustment during deployment. We additionally contribute the first two large-scale PPAR benchmarks, VP-HMDB51 and VP-UCF101, to the community. Extensive evaluations on them, as well as two other tasks, validate the effectiveness and generalization capability of our framework.

Objective. Chronic hepatitis B liver fibrosis is an important intermediate link in the development of liver cirrhosis. A retrospective cohort study was conducted in Longhua Hospital affiliated to the Shanghai University of Traditional Chinese Medicine in order to prove whether integrated traditional Chinese and Western medicine could improve the incidence of CHB complications and clinical prognosis. There are 130 patients with hepatitis B liver fibrosis (being treated from 2011–2021) included in the study, and the patients were divided into 64 TCM users (NAs combined with TCM) and 66 TCM nonusers (NAs antiviral therapy). The serum noninvasive diagnostic model (APRI, FIB-4) and LSM value were used to classify the stages of fibrosis. The results showed that the LSM value was decreased significantly in TCM users compared with TCM nonusers (40.63% versus 28.79%). Indicators of FIB-4 and APRI of TCM users have improved significantly compared with that of TCM nonusers (32.81% versus 10.61% and 35.94% versus 24.24%). The AST, TBIL, and HBsAg levels in TCM users were lower than those in TCM nonusers, and the HBsAg level was inversely correlated with the CD3+, CD4+, and CD8+ in TCM users. The PLT and spleen thickness of TCM users also were improved considerably. The incidence rate of end-point events (decompensated cirrhosis/liver cancer) in TCM nonusers was higher than that of TCM users (16.67% versus 1.56%). The long course of the disease and a family history of hepatitis B were the risk factors for disease progression, and long-term oral administration of TCM was the protective factor. As a result, the serum noninvasive fibrosis index and imaging parameters in TCM users were lower than those of TCM nonusers. Patients in the treatment of NAs combined with TCM had better prognoses such as a lower HBsAg level, a more stable lymphocyte function, and a lower incidence of end-point events. The present findings suggest the effect of TCM combined with NAs in the treatment of chronic hepatitis B liver fibrosis is better than that of single drug treatment.

Despite its success in image synthesis, we observe that diffusion probabilistic models (DPMs) often lack contextual reasoning ability to learn the relations among object parts in an image, leading to a slow learning process. To solve this issue, we propose a Masked Diffusion Transformer (MDT) that introduces a mask latent modeling scheme to explicitly enhance the DPMs' ability of contextual relation learning among object semantic parts in an image. During training, MDT operates on the latent space to mask certain tokens. Then, an asymmetric masking diffusion transformer is designed to predict masked tokens from unmasked ones while maintaining the diffusion generation process. Our MDT can reconstruct the full information of an image from its incomplete contextual input, thus enabling it to learn the associated relations among image tokens. Experimental results show that MDT achieves superior image synthesis performance, e.g. a new SoTA FID score on the ImageNet dataset, and has about 3x faster learning speed than the previous SoTA DiT. The source code is released at https://github.com/sail-sg/MDT.

The rise of pre-trained unified foundation models breaks down the barriers between different modalities and tasks, providing comprehensive support to users with unified architectures. However, the backdoor attack on pre-trained models poses a serious threat to their security. Previous research on backdoor attacks has been limited to uni-modal tasks or single tasks across modalities, making it inapplicable to unified foundation models. In this paper, we make proof-of-concept level research on the backdoor attack for pre-trained unified foundation models. Through preliminary experiments on NLP and CV classification tasks, we reveal the vulnerability of these models and suggest future research directions for enhancing the attack approach., Comment: This paper is accepted as a poster for NDSS 2023

Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets., Accepted by EMNLP Findings, 2022

This paper addresses the temporal sentence grounding (TSG). Although existing methods have made decent achievements in this task, they not only severely rely on abundant video-query paired data for training, but also easily fail into the dataset distribution bias. To alleviate these limitations, we introduce a novel Equivariant Consistency Regulation Learning (ECRL) framework to learn more discriminative query-related frame-wise representations for each video, in a self-supervised manner. Our motivation comes from that the temporal boundary of the query-guided activity should be consistently predicted under various video-level transformations. Concretely, we first design a series of spatio-temporal augmentations on both foreground and background video segments to generate a set of synthetic video samples. In particular, we devise a self-refine module to enhance the completeness and smoothness of the augmented video. Then, we present a novel self-supervised consistency loss (SSCL) applied on the original and augmented videos to capture their invariant query-related semantic by minimizing the KL-divergence between the sequence similarity of two videos and a prior Gaussian distribution of timestamp distance. At last, a shared grounding head is introduced to predict the transform-equivariant query-guided segment boundaries for both the original and augmented videos. Extensive experiments on three challenging datasets (ActivityNet, TACoS, and Charades-STA) demonstrate both effectiveness and efficiency of our proposed ECRL framework.

This paper presents a new, effective, and facile method to enhance the humidity sensitivity of lignin-based QCM sensor through fringing field effect. The biodegradable and renewable lignin was deposited on the surface of metal electrode as humidity-sensitive material. The microscopic morphology, hydrophilic groups, and water contact angle of the lignin sample were characterized. Then, the humidity-sensing performances of the lignin-based QCM sensors with symmetric and ringed electrode configurations were investigated through the oscillating circuit method. The experimental results showed that the humidity sensitivity of the ringed electrode QCM (61 Hz/%RH) is higher than that of the symmetric electrode QCM in the range of 11.3%-97.3% RH. Moreover, the ringed electrode QCM based on lignin exhibited low humidity hysteresis (~6.2% RH), good logarithmic relationship, rapid response/recovery time (28/5 s), excellent repeatability, and long-term stability. In addition, the Langmuir adsorption isotherm model and the Gibbs free energy were used to analyze the adsorption of water molecules on the surface of the lignin film. Finally, the finite element method (FEM) was used to analyze the fringing field distributions of the QCMs with symmetric and ringed electrode. The sensitivity enhancement mechanism of the lignin-based QCM humidity sensor with ringed electrode configuration was discussed through the equivalent electronic circuit analysis and simulation methods. This work demonstrates that the lignin is a potential candidate material for humidity detection, and it is an excellent method to improve the humidity sensitivity of QCM sensors through optimizing the electrode structure configuration using fringing field effect.

Vision-Language Pre-Training (VLP) has shown promising capabilities to align image and text pairs, facilitating a broad variety of cross-modal learning tasks. However, we observe that VLP models often lack the visual grounding/localization capability which is critical for many downstream tasks such as visual reasoning. In this work, we propose a novel Position-guided Text Prompt (PTP) paradigm to enhance the visual grounding ability of cross-modal models trained with VLP. Specifically, in the VLP phase, PTP divides the image into $N\times N$ blocks, and identifies the objects in each block through the widely used object detector in VLP. It then reformulates the visual grounding task into a fill-in-the-blank problem given a PTP by encouraging the model to predict the objects in the given blocks or regress the blocks of a given object, e.g. filling `P" or ``O" in aPTP ``The block P has a O". This mechanism improves the visual grounding capability of VLP models and thus helps them better handle various downstream tasks. By introducing PTP into several state-of-the-art VLP frameworks, we observe consistently significant improvements across representative cross-modal learning model architectures and several benchmarks, e.g. zero-shot Flickr30K Retrieval (+4.8 in average recall@1) for ViLT \cite{vilt} baseline, and COCO Captioning (+5.3 in CIDEr) for SOTA BLIP \cite{blip} baseline. Moreover, PTP achieves comparable results with object-detector based methods, and much faster inference speed since PTP discards its object detector for inference while the later cannot. Our code and pre-trained weight will be released at \url{https://github.com/sail-sg/ptp}., Camera-ready version, code is in https://github.com/sail-sg/ptp

A transition-metal-free method for the construction of 3-substituted or 3,4-disubstituted quinolines from readily available N,N-dimethyl enaminones and o-aminobenzyl alcohols is reported. The direct oxidative cyclocondensation reaction tolerates broad functional groups, allowing the efficient synthesis of various quinolines in moderate to excellent yields. The reaction involves a C (sp3)-O bond cleavage and a C=N bind and a C=C bond formation during the oxidative cyclization process, and the mechanism was proposed.

This paper studies the multimedia problem of temporal sentence grounding (TSG), which aims to accurately determine the specific video segment in an untrimmed video according to a given sentence query. Traditional TSG methods mainly follow the top-down or bottom-up framework and are not end-to-end. They severely rely on time-consuming post-processing to refine the grounding results. Recently, some transformer-based approaches are proposed to efficiently and effectively model the fine-grained semantic alignment between video and query. Although these methods achieve significant performance to some extent, they equally take frames of the video and words of the query as transformer input for correlating, failing to capture their different levels of granularity with distinct semantics. To address this issue, in this paper, we propose a novel Hierarchical Local-Global Transformer (HLGT) to leverage this hierarchy information and model the interactions between different levels of granularity and different modalities for learning more fine-grained multi-modal representations. Specifically, we first split the video and query into individual clips and phrases to learn their local context (adjacent dependency) and global correlation (long-range dependency) via a temporal transformer. Then, a global-local transformer is introduced to learn the interactions between the local-level and global-level semantics for better multi-modal reasoning. Besides, we develop a new cross-modal cycle-consistency loss to enforce interaction between two modalities and encourage the semantic alignment between them. Finally, we design a brand-new cross-modal parallel transformer decoder to integrate the encoded visual and textual features for final grounding. Extensive experiments on three challenging datasets show that our proposed HLGT achieves a new state-of-the-art performance.

In recent years, the problem of atmospheric pollution has been concerning in the Beijing–Tianjin–Hebei region, due to the frequent haze. It has become a significant issue to improve regional air quality through appropriate emission reduction measures. In this study, considering the regional atmospheric transmission of air pollutants, the WRF/CALPUFF model (the Weather Research and Forecasting model coupled with the California Puff air quality model) was used to describe the impact of each city’s pollutant emissions on the concentrations of every city. Then, a new optimization model was designed to calculate the maximum allowable emissions of every city. The results showed that NOx and PM2.5 emissions need to be reduced by 44% and 48%, respectively, in the traditional mitigation scenario (any city’s pollutant emissions are not allowed to increase). However, in the optimized scenario, NOx and PM2.5 emissions should be reduced by 23% and 46%, respectively, to meet the national secondary standard. The emissions of cities with low transfer coefficients, such as Zhangjiakou, Qinhuangdao, and Chengde, could even be appropriately increased. This means that the optimized scenario could reduce the pressure on emission reduction. Although the optimization results are theoretical and idealistic, this research study provides a new idea for formulating emission mitigation policies in various regions to reduce the impact on the economy.

In deep learning, different kinds of deep networks typically need different optimizers, which have to be chosen after multiple trials, making the training process inefficient. To relieve this issue and consistently improve the model training speed across deep networks, we propose the ADAptive Nesterov momentum algorithm, Adan for short. Adan first reformulates the vanilla Nesterov acceleration to develop a new Nesterov momentum estimation (NME) method, which avoids the extra overhead of computing gradient at the extrapolation point. Then Adan adopts NME to estimate the gradient's first- and second-order moments in adaptive gradient algorithms for convergence acceleration. Besides, we prove that Adan finds an $\epsilon$-approximate first-order stationary point within $O(\epsilon^{-3.5})$ stochastic gradient complexity on the non-convex stochastic problems (e.g., deep learning problems), matching the best-known lower bound. Extensive experimental results show that Adan consistently surpasses the corresponding SoTA optimizers on vision, language, and RL tasks and sets new SoTAs for many popular networks and frameworks, e.g., ResNet, ConvNext, ViT, Swin, MAE, DETR, GPT-2, Transformer-XL, and BERT. More surprisingly, Adan can use half of the training cost (epochs) of SoTA optimizers to achieve higher or comparable performance on ViT, GPT-2, MAE, e.t.c., and also shows great tolerance to a large range of minibatch size, e.g., from 1k to 32k. Code is released at https://github.com/sail-sg/Adan, and has been used in multiple popular deep learning frameworks or projects.

This paper proposes a Video Graph Transformer (VGT) model for Video Quetion Answering (VideoQA). VGT's uniqueness are two-fold: 1) it designs a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations, and dynamics for complex spatio-temporal reasoning; and 2) it exploits disentangled video and text Transformers for relevance comparison between the video and text to perform QA, instead of entangled cross-modal Transformer for answer classification. Vision-text communication is done by additional cross-modal interaction modules. With more reasonable video encoding and QA solution, we show that VGT can achieve much better performances on VideoQA tasks that challenge dynamic relation reasoning than prior arts in the pretraining-free scenario. Its performances even surpass those models that are pretrained with millions of external data. We further show that VGT can also benefit a lot from self-supervised cross-modal pretraining, yet with orders of magnitude smaller data. These results clearly demonstrate the effectiveness and superiority of VGT, and reveal its potential for more data-efficient pretraining. With comprehensive analyses and some heuristic observations, we hope that VGT can promote VQA research beyond coarse recognition/description towards fine-grained relation reasoning in realistic videos. Our code is available at https://github.com/sail-sg/VGT., ECCV'22

In the factory, the steel round bar sawing machine is applied to finish the sawing mission. Due to the process of the sawing steel is operating in high speed, the grinding-wheel-saw (GWS) is quickly worn by the friction and replaced frequently. The automatic replacement system of GWSs is seldom built yet. To improve the efficiency of the replacement for GWSs, the machine-vision-based intelligent robotic replacement system (MIRRS) for GWSs is developed and reported in the study. The MIRRS is mainly consisted of the six degrees-of-freedom (DOF) manipulator and the six-axes mechanical arm system to successfully complete the task of the replacement. The finite element method (FEM) is applied to analyze and design the subsystem of the MIRRS. Besides, the machine-vision positioning system associated with the digital camera are also setup to provide the required feedback detections for the MIRRS. The hardware system architecture and the control flow chart are described in detail and the hundreds of realistic testing results are collected to verify the reliability and offer the well working efficiency of the designed MIRRS.

For unsupervised pretraining, mask-reconstruction pretraining (MRP) approaches, e.g. MAE and data2vec, randomly mask input patches and then reconstruct the pixels or semantic features of these masked patches via an auto-encoder. Then for a downstream task, supervised fine-tuning the pretrained encoder remarkably surpasses the conventional ``supervised learning'' (SL) trained from scratch. However, it is still unclear 1) how MRP performs semantic feature learning in the pretraining phase and 2) why it helps in downstream tasks. To solve these problems, we first theoretically show that on an auto-encoder of a two/one-layered convolution encoder/decoder, MRP can capture all discriminative features of each potential semantic class in the pretraining dataset. Then considering the fact that the pretraining dataset is of huge size and high diversity and thus covers most features in downstream dataset, in fine-tuning phase, the pretrained encoder can capture as much features as it can in downstream datasets, and would not lost these features with theoretical guarantees. In contrast, SL only randomly captures some features due to lottery ticket hypothesis. So MRP provably achieves better performance than SL on the classification tasks. Experimental results testify to our data assumptions and also our theoretical implications.

Graph neural networks (GNNs) have achieved state-of-the-art performance in many graph-based tasks such as node classification and graph classification. However, many recent works have demonstrated that an attacker can mislead GNN models by slightly perturbing the graph structure. Existing attacks to GNNs are either under the less practical threat model where the attacker is assumed to access the GNN model parameters, or under the practical black-box threat model but consider perturbing node features that are shown to be not enough effective. In this paper, we aim to bridge this gap and consider black-box attacks to GNNs with structure perturbation as well as with theoretical guarantees. We propose to address this challenge through bandit techniques. Specifically, we formulate our attack as an online optimization with bandit feedback. This original problem is essentially NP-hard due to the fact that perturbing the graph structure is a binary optimization problem. We then propose an online attack based on bandit optimization which is proven to be {sublinear} to the query number $T$, i.e., $\mathcal{O}(\sqrt{N}T^{3/4})$ where $N$ is the number of nodes in the graph. Finally, we evaluate our proposed attack by conducting experiments over multiple datasets and GNN models. The experimental results on various citation graphs and image graphs show that our attack is both effective and efficient. Source code is available at~\url{https://github.com/Metaoblivion/Bandit_GNN_Attack}, Comment: Accepted by CVPR 2022 Oral presentation

The transportation sector has a significant impact on the synergistic reduction of CO2 and pollutants. Using a structural path decomposition, we reveals the direct and embodied CO2 and NOx emissions from the transportation sector triggered by key supply chain pathways. Meanwhile, the emission abatement potential and economic costs of 33 vehicle specific abatement technology options are analyzed based on the input-output analysis for life cycle assessment. The results show that most of the vehicle technology options can reduce CO2 and NOx emissions while saving economic costs. Among these technologies, both the pure electric technology for passenger cars and the parallel hybrid technology for heavy-duty trucks have high abatement potential. Compared to costly pure electrification technologies for passenger cars, parallel hybrid technologies for heavy-duty trucks offer the highest economic benefits as well as a better driving mileage. Therefore, hybrid heavy-duty trucks will be a more comprehensive solution in the near future.

Sepsis-induced acute lung injury (ALI) is associated with poor survival rates. The identification of potential therapeutic targets for preventing sepsis-induced ALI has clinical importance. Here, we show that estrogen-related receptor alpha (ERRα) protects against sepsis-induced ALI through ERRα-mediated apoptosis and autophagy. ERRα was upregulated in inflammation models from lipopolysaccharides (LPS)-treated rat pulmonary microvascular endothelial cells (PMVECs) and cecal ligation and puncture (CLP)-induced septic rats. In vitro , overexpression of ERRα ameliorated LPS-induced degradation of adherens junctional molecules, upregulation of bax, cleaved caspase 3 and cleaved caspase 9 levels and downregulation of anti-apoptotic protein Bcl-2 level, and promoted the formation of autophagic flux, while the knockdown of ERRα exacerbated LPS-induced apoptosis and inhibited the activation of autophagy. Rats injected with ERRα agonist were subjected to CLP to elucidate the role of ERRα in sepsis-induced ALI in vivo . Administration of ERRα agonist alleviated the pathological damage of lung tissue, increased the levels of tight junction proteins and adherens junction proteins, and decreased the expression of apoptosis-related proteins. Promoting the expression of ERRα significantly enhanced the process of autophagy and reduced CLP-induced ALI. Mechanistically, ERRα is essential to regulate the balance between autophagy and apoptosis to maintain the adherens junctional integrity. These findings indicate that activation of ERRα provides a new therapeutic opportunity to prevent sepsis-induced ALI.

Multilabel data share important features, including label imbalance, which has a significant influence on the performance of classifiers. Because of this problem, a widely used multilabel classification algorithm, the multilabel k-nearest neighbor (ML-kNN) algorithm, has poor performance on imbalanced multilabel data. To address this problem, this study proposes an improved ML-kNN algorithm based on value and weight. In this improved algorithm, labels are divided into minority and majority, and different strategies are adopted for different labels. By considering the label of latent information carried by the nearest neighbors, a value calculation method is proposed and used to directly classify majority labels. Additionally, to address the misclassification problem caused by a lack of nearest neighbor information for minority labels, weight calculation is proposed. The proposed weight calculation converts distance information with and without label sets in the nearest neighbors into weights. The experimental results on multilabel datasets from different benchmarks demonstrate the performance of the algorithm, especially for datasets with high imbalance. Different evaluation metrics show that the results are improved by approximately 2–10%. The verified algorithm could be applied to a multilabel classification of various fields involving label imbalance, such as drug molecule identification, building identification, and text categorization.

Proliferation of smooth muscle cells, oxidative stress, and pulmonary vasoconstriction resulting from intermittent hypoxia (IH) facilitate pulmonary hypertension (PH) in patients with obstructive sleep apnea. The role of Phosphodiesterase 4 B (PDE4B) in PH has not yet been established. Herein, we investigated whether PDE4B inhibition ameliorates experimental PH by modulating cAMP signaling. We performed an integrative analysis of PDE4B expression in Gene Expression Omnibus datasets, experimental IH-induced rat PH samples, and IH-induced pulmonary arterial smooth muscle cells (PASMCs). PDE4B expression was modulated using siRNA in vitro and a specific adeno-associated virus serotype 1 in vivo. In the databases of mouse models of IH-induced and sustained hypoxia-induced PH and in a rat model of six weeks of IH, the expression of PDE4B was up-regulated. Inhibition of PDE4B attenuated IH-induced pulmonary vascular remodeling and right ventricular hypertrophy. Our results also showed that PDE4B deficiency inhibited IH-induced proliferation of PASMCs with less mitochondrial reactive oxygen species and mitochondrial damage. Meanwhile, IH induced an increase in ATF4, which positively regulated the expression of PDE4B through transcription, and inhibition of ATF4 exerted effects similar to those of PDE4B inhibition. Mechanistically, downregulating the expression of PDE4B resulted in the activation of the cAMP/PKA/p-CREB/PGC-1α pathway in PASMCs after IH. Taken together, our present study provides evidence that inhibition of PDE4B attenuates IH-induced PH by regulating cAMP signaling.

The novel method for the synthesis of gem-bisarylthio enamines and α-phenylthio ketones was developed via the coupling of α-substituted vinyl azides with thiols in the presence of tetrabutylammonium iodide (TBAI) as a redox catalyst and electrolyte at room temperature. Electronic properties were crucial in the generated products. This protocol features metal- and oxidant-free materials, broad tolerance of substrates, and mild reaction conditions.

In the period from 2011 to 2020, 4 large-scale population-based studies were conducted to identify gout in patients with obstructive sleep apnea (OSA), as well as one study to detect the frequency of sleep apnea in patients with gout.All recently published studies confirm connection between the development of OSA and an increase in the level of uric acid in blood serum, both against the background of clinically significant gout arthritis and without it. However, only one big population-based study during 2010 — 2020 assessed the development of OSA against the background of gout. According to this study, incidence of OSA was 14.3 per 1.000 person-years in patients with gout, compared with 3.9 per 1.000 person-years without gout, with HR = 2.07 (95 % CI 2.00 — 2.15 %), which is significantly higher than the incidence of gout in patients with OSA, even taking into account hypertension and hyperlipidemia.The data from these studies showed an undoubted relationship between the two diseases, but to date it is not completely known how much it is due to common risk factors and how interlinked the development mechanisms are. Older age and restricted mobility put patients with gout at a significant risk during OSA episodes. In cases where the treatment of both diseases is conducted independently, the volume of drug therapy and the likelihood of developing negative side effects increase. In this regard, it is necessary to monitor patients with gout who develop sleep apnea differently to gout only patients, as well as to prescribe adequate treatment for both diseases.In our review, we aimed to show the most recent information on the mechanisms of gout-OSA development, shared pathways, targets and biomarkers, emphasizing the likely ways that cause the progression of OSA in patients with gout. In addition, analysis of the latest publications to date showed nonuniformity in study subjects, allowing to identify different subtypes of gout-OSA patients: combined with metabolic syndrome (most common), combined with renal dysfunction without obesity, and others (dietary violations, genetic diseases, acidosis).

Catalytic asymmetric conjugate allylation of unsaturated carbonyl compounds is usually difficult to achieve, as 1,2-addition proceeds dominantly and high asymmetric induction is a challenging task. Herein, we disclose a copper(I)-NHC complex catalyzed asymmetric 1,6-conjugate allylation of 2,2-dimethyl-6-alkenyl-4H-1,3-dioxin-4-ones. The phenolic hydroxyl group in NHC ligands is found to be pivotal to obtain the desired products. Both aryl group and alkyl group at δ-position are well tolerated with the corresponding products generated in moderate to high yields and high enantioselectivity. Moreover, both 2-substituted and 3-substituted allylboronates serve as acceptable allylation reagents. At last, the synthetic utility of the products is demonstrated in several transformations by means of the versatile terminal olefin and dioxinone groups., Catalytic asymmetric conjugate allylation of unsaturated carbonyl compounds is usually difficult to achieve due to competing 1,2-addition. Here, the authors report a copper(I)-catalyzed asymmetric 1,6-conjugate allylation of 2,2-dimethyl-6-alkenyl-4H-1,3-dioxin-4-ones with good scope and high enantioselectivity.

In self-supervised learning, multi-granular features are heavily desired though rarely investigated, as different downstream tasks (e.g., general and fine-grained classification) often require different or multi-granular features, e.g.~fine- or coarse-grained one or their mixture. In this work, for the first time, we propose an effective MUlti-Granular Self-supervised learning (Mugs) framework to explicitly learn multi-granular visual features. Mugs has three complementary granular supervisions: 1) an instance discrimination supervision (IDS), 2) a novel local-group discrimination supervision (LGDS), and 3) a group discrimination supervision (GDS). IDS distinguishes different instances to learn instance-level fine-grained features. LGDS aggregates features of an image and its neighbors into a local-group feature, and pulls local-group features from different crops of the same image together and push them away for others. It provides complementary instance supervision to IDS via an extra alignment on local neighbors, and scatters different local-groups separately to increase discriminability. Accordingly, it helps learn high-level fine-grained features at a local-group level. Finally, to prevent similar local-groups from being scattered randomly or far away, GDS brings similar samples close and thus pulls similar local-groups together, capturing coarse-grained features at a (semantic) group level. Consequently, Mugs can capture three granular features that often enjoy higher generality on diverse downstream tasks over single-granular features, e.g.~instance-level fine-grained features in contrastive learning. By only pretraining on ImageNet-1K, Mugs sets new SoTA linear probing accuracy 82.1$\%$ on ImageNet-1K and improves previous SoTA by $1.1\%$. It also surpasses SoTAs on other tasks, e.g. transfer learning, detection and segmentation., code and models are available at https://github.com/sail-sg/mugs

End-to-end (E2E) speech recognition architectures assemble all components of traditional speech recognition system into a single model. Although it simplifies ASR system, it introduces contextual ASR drawback: the E2E model has worse performance on utterances containing infrequent proper nouns. In this work, we propose to add a contextual bias attention (CBA) module to attention based encoder decoder (AED) model to improve its ability of recognizing the contextual phrases. Specifically, CBA utilizes the context vector of source attention in decoder to attend to a specific bias embedding. Jointly learned with the basic AED parameters, CBA can tell the model when and where to bias its output probability distribution. At inference stage, a list of bias phrases is preloaded and we adapt the posterior distributions of both CTC and attention decoder according to the attended bias phrase of CBA. We evaluate the proposed method on GigaSpeech and achieve a consistent relative improvement on recall rate of bias phrases ranging from 15% to 28% compared to the baseline model. Meanwhile, our method shows a strong anti-bias ability as the performance on general tests only degrades 1.7% even 2,000 bias phrases are present., 5 pages, 5 tabels, 1 figure

It is of great significance to identify the critical influential factors of pollutant emissions for emission mitigation. However, city disparity implies different priorities for regional mitigation. This study aims to estimate the consumption-based emissions of 309 prefecture-level cities in China based on the multi-region input-output table and the sectoral NOx emission inventory and investigate the emission transfer phenomenon among cities and sectors. In addition, a geographically weighted regression method is used to analyze the spatial heterogeneity in the driving factors of regional consumption-based emissions. The results reveal that the top 10 cities in consumption-based emissions account for 25.2% of emissions and contribute 22.6% to GDP. The consumption-based emissions are mainly driven by local demand (72.79%) at the regional level and by construction activities (94.43%) at the sectoral level. Besides, the results also show the spatial variances in contributions of driving forces to consumption-based emissions. Economic growth has been identified as the most important factor which promotes consumption-based emissions. However, disposable personal income, per capita road area, urbanization, and percentage of tertiary industry GDP are conducive to reduce consumption-based emissions in some cities of China. It could be concluded that policies without consideration of the emissions from a consumption perspective are difficult to achieve effective emission reduction.