Search

Over the past decade, domain adaptation has become a widely studied branch of transfer learning which aims to improve performance on target domains by leveraging knowledge from the source domain. Conventional domain adaptation methods often assume access to both source and target domain data simultaneously, which may not be feasible in real-world scenarios due to privacy and confidentiality concerns. As a result, the research of Source-Free Domain Adaptation (SFDA) has drawn growing attention in recent years, which only utilizes the source-trained model and unlabeled target data to adapt to the target domain. Despite the rapid explosion of SFDA work, there has been no timely and comprehensive survey in the field. To fill this gap, we provide a comprehensive survey of recent advances in SFDA and organize them into a unified categorization scheme based on the framework of transfer learning. Instead of presenting each approach independently, we modularize several components of each method to more clearly illustrate their relationships and mechanisms in light of the composite properties of each method. Furthermore, we compare the results of more than 30 representative SFDA methods on three popular classification benchmarks, namely Office-31, Office-home, and VisDA, to explore the effectiveness of various technical routes and the combination effects among them. Additionally, we briefly introduce the applications of SFDA and related fields. Drawing on our analysis of the challenges confronting SFDA, we offer some insights into future research directions and potential settings.

Building multi-person pose estimation (MPPE) models that can handle complex foreground and uncommon scenes is an important challenge in computer vision. Aside from designing novel models, strengthening training data is a promising direction but remains largely unexploited for the MPPE task. In this article, we systematically identify the key deficiencies of existing pose datasets that prevent the power of well-designed models from being fully exploited and propose the corresponding solutions. Specifically, we find that the traditional data augmentation techniques are inadequate in addressing the two key deficiencies, imbalanced instance complexity (IC) (evaluated by our new metric IC) and insufficient realistic scenes. To overcome these deficiencies, we propose a model-agnostic full-view data generation (Full-DG) method to enrich the training data from the perspectives of both poses and scenes. By hallucinating images with more balanced pose complexity and richer real-world scenes, Full-DG can help improve pose estimators' robustness and generalizability. In addition, we introduce a plug-and-play adaptive category-aware loss (AC-loss) to alleviate the severe pixel-level imbalance between keypoints and backgrounds (i.e., around 1:600). Full-DG together with AC-loss can be readily applied to both the bottom-up and top-down models to improve their accuracy. Notably, plugging into the representative estimators HigherHRNet and HRNet, our method achieves substantial performance gains of 1.0%-2.9% AP on the COCO benchmark, and 1.0%-5.1% AP on the CrowdPose benchmark.

ALI is a grave medical ailment that manifests as abrupt inflammation of the lungs and diminished oxygen levels. It poses a considerable challenge to the medical fraternity, with elevated rates of morbidity and mortality. Our research endeavors to investigate the potential of hibifolin, a flavonoid glucuronide, imbued with potent antioxidant properties, and its molecular mechanism to combat LPS-induced ALI in mice. The study utilized ICR mice to create an ALI model induced by LPS. Prior to LPS administration, hibifolin was given at 10, 30, or 50 mg/kg, or dexamethasone was given at 1 mg/kg to assess its preventative impact. Changes in lung tissue, pulmonary edema, and lipid peroxidation were analyzed using H&E stain assay, lung wet/dry ratio assay, and MDA formation assay, respectively. Activity assay kits were used to measure MPO activity and antioxidative enzymes (SOD, CAT, GPx) activity in the lungs. Western blot assay was used to determine the phosphorylation of Nrf-2 and AMPK2 in the lungs. Hibifolin demonstrated a concentration-dependent improvement in LPS-induced histopathologic pulmonary changes. This treatment notably mitigated pulmonary edema, lipid peroxidation, and MPO activity in ALI mice. Additionally, hibifolin successfully restored antioxidative enzyme activity in the lungs of ALI mice. Moreover, hibifolin effectively promoted Nrf-2 phosphorylation and reinstated AMPK2 phosphorylation in the lungs of ALI mice. The results indicate that hibifolin could effectively alleviate the pathophysiological impact of LPS-induced ALI. This is likely due to its antioxidative properties, which help to restore antioxidative enzyme activity and activate the AMPK2/Nrf2 pathway. These findings are valuable in terms of enhancing our knowledge of ALI treatment and pave the way for further investigation into hibifolin as a potential therapeutic option for lung injuries., (© 2024 Wiley Periodicals LLC.)

Chronic obstructive pulmonary disease (COPD) is the world's leading lung disease and lacks effective and specific clinical strategies. Probiotics are increasingly used to support the improvement of the course of inflammatory diseases. In this study, we evaluated the potential of a lactic acid bacteria (LAB) combination containing Limosilactobacillus reuteri GMNL-89 and Lacticaseibacillus paracasei GMNL-133 to decrease lung inflammation and emphysema in a COPD mouse model. This model was induced by intranasal stimulation with elastase and LPS for 4 weeks, followed by 2 weeks of oral LAB administration. The results showed that the LAB combination decreased lung emphysema and reduced inflammatory cytokines (IL-1β, IL-6, TNF-α) in the lung tissue of COPD mice. Microbiome analysis revealed that Bifidobacterium and Akkermansia muciniphila, reduced in the gut of COPD mice, could be restored after LAB treatment. Microbial α-diversity in the lungs decreased in COPD mice but was reversed after LAB administration, which also increased the relative abundance of Candidatus arthromitus in the gut and decreased Burkholderia in the lungs. Furthermore, LAB-treated COPD mice exhibited increased levels of short-chain fatty acids, specifically acetic acid and propionic acid, in the cecum. Additionally, pulmonary emphysema and inflammation negatively correlated with C. arthromitus and Adlercreutzia levels. In conclusion, the combination of L. reuteri GMNL-89 and L. paracasei GMNL-133 demonstrates beneficial effects on pulmonary emphysema and inflammation in experimental COPD mice, correlating with changes in gut and lung microbiota, and providing a potential strategy for future adjuvant therapy., (© 2024. The Author(s).)

Given an input image, scene graph generation (SGG) aims to generate comprehensive visual relationships between objects in the form of graphs. Recently, more attention to the design of complex networks and complicated strategies has been paid to the long tail issue caused by the imbalanced class distribution. However, most existing methods adopt the concatenated features of two objects in real space as the final relation representation for a given triplet. We mainly argue that such a simple concatenation may neglect the importance of complex interactions between objects, which results in the diversity of visual relations. In addition, the representation learning in real space is also inadequate to express this property. To alleviate these issues, we seamlessly incorporate Hermitian inner product into existing models to facilitate the generation of scene graphs by learning Relation Embedding in Complex space (CoRE). More specifically, we first introduce the concept of complex-valued representations for entities and then formulate the relation triplets with Hermitian inner product in complex space. Finally, we investigate the effect of utilizing only real component or both of Hermitian inner product on inferring more reasonable interaction between objects for scene graphs. Comprehensive experiments on two widely used benchmark datasets, Visual Genome (VG) and Open Image, demonstrate our effectiveness, superiority, and generalization on various metrics for biased or unbiased inference.

Contrastive learning has been successfully applied in unsupervised representation learning. However, the generalization ability of representation learning is limited by the fact that the loss of downstream tasks (e.g., classification) is rarely taken into account while designing contrastive methods. In this article, we propose a new contrastive-based unsupervised graph representation learning (UGRL) framework by 1) maximizing the mutual information (MI) between the semantic information and the structural information of the data and 2) designing three constraints to simultaneously consider the downstream tasks and the representation learning. As a result, our proposed method outputs robust low-dimensional representations. Experimental results on 11 public datasets demonstrate that our proposed method is superior over recent state-of-the-art methods in terms of different downstream tasks. Our code is available at https://github.com/LarryUESTC/GRLC.

Breast cancer is a leading cause of cancer mortality in women worldwide. Using the Infinium MethylationEPIC BeadChip, we analyzed plasma sample methylation to identify the SRCIN1 gene in breast cancer patients. We assessed SRCIN1 -related roles and pathways for their biomarker potential. To verify the methylation status, quantitative methylation-specific PCR (qMSP) was performed on genomic DNA and circulating cell-free DNA samples, and mRNA expression analysis was performed using RT‒qPCR. The results were validated in a Western population; for this analysis, the samples included plasma samples from breast cancer patients from the USA and from The Cancer Genome Atlas (TCGA) cohort. To study the SRCIN1 pathway, we conducted cell viability assays, gene manipulation and RNA sequencing. SRCIN1 hypermethylation was identified in 61.8% of breast cancer tissues from Taiwanese patients, exhibiting specificity to this malignancy. Furthermore, its presence correlated significantly with unfavorable 5-year overall survival outcomes. The levels of methylated SRCIN1 in the blood of patients from Taiwan and the USA correlated with the stage of breast cancer. The proportion of patients with high methylation levels increased from 0% in healthy individuals to 63.6% in Stage 0, 80% in Stage I and 82.6% in Stage II, with a sensitivity of 78.5%, an accuracy of 90.3% and a specificity of 100%. SRCIN1 hypermethylation was significantly correlated with increased SRCIN1 mRNA expression ( p < 0.001). Knockdown of SRCIN1 decreased the viability of breast cancer cells. SRCIN1 silencing resulted in the downregulation of ESR1, BCL2 and various cyclin protein expressions. SRCIN1 hypermethylation in the blood may serve as a noninvasive biomarker, facilitating early detection and prognosis evaluation, and SRCIN1 -targeted therapies could be used in combination regimens for breast cancer patients.

The presence of label noise in the training data has a profound impact on the generalization of deep neural networks (DNNs). In this study, we introduce and theoretically demonstrate a simple feature noise (FN) method, which directly adds noise to the features of training data and can enhance the generalization of DNNs under label noise. Specifically, we conduct theoretical analyses to reveal that label noise leads to weakened DNN generalization by loosening the generalization bound, and FN results in better DNN generalization by imposing an upper bound on the mutual information between the model weights and the features, which constrains the generalization bound. Furthermore, we conduct a qualitative analysis to discuss the ideal type of FN that obtains good label noise generalization. Finally, extensive experimental results on several popular datasets demonstrate that the FN method can significantly enhance the label noise generalization of state-of-the-art methods. The source codes of the FN method are available on https://github.com/zlzenglu/FN.

Temporal language grounding (TLG) is one of the most challenging cross-modal video understanding tasks, which aims at retrieving the most relevant video segment from an untrimmed video according to a natural language sentence. The existing methods can be separated into two dominant types: 1) proposal-based and 2) proposal-free methods, where the former conduct contextual interactions and the latter localizes timestamps flexibly. However, the constant-scale candidates in proposal-based methods limit the localization precision and bring extra computational costs. In contrast, the proposal-free methods perform well on high-precision metrics-based on the fine-grained features but suffer from a lack of coarse-grained interactions, which cause degeneration when the video becomes complex. In this article, we propose a novel framework termed semantic decoupling network (SDN) that combines the advantages of proposal-based and proposal-free methods and overcomes their defects. It contains three key components: 1) semantic decoupling module (SDM); 2) context modeling block (CMB); and 3) semantic cross-level aggregation module (SCAM). By capturing the video-text contexts in multilevel semantics, the SDM and CMB effectively utilize the benefits of proposal-based methods. Meanwhile, the SCAM maintains the merit of proposal-free methods in that it localizes timestamps precisely. The experiments on three challenge datasets, i.e., Charades-STA, TACoS, and ActivityNet-Caption, show that our proposed SDN method significantly outperforms recent state-of-the-art methods, especially the proposal-free methods. Extensive analyses, as well as the implementation code of the proposed SDN method, are provided at https://github.com/CFM-MSG/Code&#95;SDN.

Despite its significant progress, cross-modal retrieval still suffers from one-to-many matching cases, where the multiplicity of semantic instances in another modality could be acquired by a given query. However, existing approaches usually map heterogeneous data into the learned space as deterministic point vectors. In spite of their remarkable performance in matching the most similar instance, such deterministic point embedding suffers from the insufficient representation of rich semantics in one-to-many correspondence. To address the limitations, we intuitively extend a deterministic point into a closed geometry and develop geometric representation learning methods for cross-modal retrieval. Thus, a set of points inside such a geometry could be semantically related to many candidates, and we could effectively capture the semantic uncertainty. We then introduce two types of geometric matching for one-to-many correspondence, i.e., point-to-rectangle matching (dubbed P2RM) and rectangle-to-rectangle matching (termed R2RM). The former treats all retrieved candidates as rectangles with zero volume (equivalent to points) and the query as a box, while the latter encodes all heterogeneous data into rectangles. Therefore, we could evaluate semantic similarity among heterogeneous data by the Euclidean distance from a point to a rectangle or the volume of intersection between two rectangles. Additionally, both strategies could be easily employed for off-the-self approaches and further improve the retrieval performance of baselines. Under various evaluation metrics, extensive experiments and ablation studies on several commonly used datasets, two for image-text matching and two for video-text retrieval, demonstrate our effectiveness and superiority.

We investigated fourteen antibiotics, three illegal drugs, and two toxic elements in commercially available gastropods from southeast China. The data revealed high detection frequencies (DFs) for florfenicol (61.32%), florfenicol amine (47.33%), and thiamphenicol (39.88%), with maximum concentrations of 1110, 2222, and 136 μg/kg wet weight (ww), respectively. The DFs of illegal drugs were 3.54% for leucomalachite green and 0.3% for chloramphenicol. The average levels of Cd and As were 1.17 and 6.12 mg/kg ww, respectively. All chemicals presented diverse DFs in different sampling months. The highest DFs of florfenicol, florfenicol amine, and thiamphenicol were in July. The health risk assessment showed that targeted hazard quotients (THQs) of antibiotics, Cd, and As for children, teens, and adults were all less than one. Notably, the toxic elements (Cd and As) were identified as the primary health risk in gastropods, contributing to over 90% of the total THQs.

Unsupervised domain adaptation (UDA) aims to alleviate the domain shift by transferring knowledge learned from a labeled source dataset to an unlabeled target domain. Although UDA has seen promising progress recently, it requires access to data from both domains, making it problematic in source data-absent scenarios. In this article, we investigate a practical task source-free domain adaptation (SFDA) that alleviates the limitations of the widely studied UDA in simultaneously acquiring source and target data. In addition, we further study the imbalanced SFDA (ISFDA) problem, which addresses the intra-domain class imbalance and inter-domain label shift in SFDA. We observe two key issues in SFDA that: 1) target data form clusters in the representation space regardless of whether the target data points are aligned with the source classifier and 2) target samples with higher classification confidence are more reliable and have less variation in their classification confidence during adaptation. Motivated by these observations, we propose a unified method, named intrinsic consistency preservation with adaptively reliable samples (ICPR), to jointly cope with SFDA and ISFDA. Specifically, ICPR first encourages the intrinsic consistency in the predictions of neighbors for unlabeled samples with weak augmentation (standard flip-and-shift), regardless of their reliability. ICPR then generates strongly augmented views specifically for adaptively selected reliable samples and is trained to fix the intrinsic consistency between weakly and strongly augmented views of the same image concerning predictions of neighbors and their own. Additionally, we propose to use a prototype-like classifier to avoid the classification confusion caused by severe intra-domain class imbalance and inter-domain label shift. We demonstrate the effectiveness and general applicability of ICPR on six benchmarks of both SFDA and ISFDA tasks. The reproducible code of our proposed ICPR method is available at https://github.com/CFM-MSG/Code&#95;ICPR.

Unsupervised anomaly detection (UAD) attracts a lot of research interest and drives widespread applications, where only anomaly-free samples are available for training. Some UAD applications intend to locate the anomalous regions further even without any anomaly information. Although the absence of anomalous samples and annotations deteriorates the UAD performance, an inconspicuous, yet powerful statistics model, the normalizing flows, is appropriate for anomaly detection (AD) and localization in an unsupervised fashion. The flow-based probabilistic models, only trained on anomaly-free data, can efficiently distinguish unpredictable anomalies by assigning them much lower likelihoods than normal data. Nevertheless, the size variation of unpredictable anomalies introduces another inconvenience to the flow-based methods for high-precision AD and localization. To generalize the anomaly size variation, we propose a novel multiscale flow-based framework (MSFlow) composed of asymmetrical parallel flows followed by a fusion flow to exchange multiscale perceptions. Moreover, different multiscale aggregation strategies are adopted for image-wise AD and pixel-wise anomaly localization according to the discrepancy between them. The proposed MSFlow is evaluated on three AD datasets, significantly outperforming existing methods. Notably, on the challenging MVTec AD benchmark, our MSFlow achieves a new state-of-the-art (SOTA) with a detection AUORC score of up to 99.7%, localization AUCROC score of 98.8% and PRO score of 97.1%.

Semi-supervised Action Quality Assessment (AQA) using limited labeled and massive unlabeled samples to achieve high-quality assessment is an attractive but challenging task. The main challenge relies on how to exploit solid and consistent representations of action sequences for building a bridge between labeled and unlabeled samples in the semi-supervised AQA. To address the issue, we propose a Self-supervised sub-Action Parsing Network (SAP-Net) that employs a teacher-student network structure to learn consistent semantic representations between labeled and unlabeled samples for semi-supervised AQA. We perform actor-centric region detection and generate high-quality pseudo-labels in the teacher branch and assists the student branch in learning discriminative action features. We further design a self-supervised sub-action parsing solution to locate and parse fine-grained sub-action sequences. Then, we present the group contrastive learning with pseudo-labels to capture consistent motion-oriented action features in the two branches. We evaluate our proposed SAP-Net on four public datasets: the MTL-AQA, FineDiving, Rhythmic Gymnastics, and FineFS datasets. The experiment results show that our approach outperforms state-of-the-art semi-supervised methods by a significant margin.

Existing methods of multiple human parsing (MHP) apply deep models to learn instance-level representations for segmenting each person into non-overlapped body parts. However, learned representations often contain many spurious correlations that degrade model generalization, leading learned models to be vulnerable to visually contextual variations in images (e.g., unseen image styles/external interventions). To tackle this, we present a causal property integrated parsing model termed CPI-Parser, which is driven by fundamental causal principles involving two causal properties for human parsing (i.e., the causal diversity and the causal invariance). Specifically, we assume that an image is constructed by a mix of causal factors (the characteristics of body parts) and non-causal factors (external contexts), where only the former ones decide the essence of human parsing. Since causal/non-causal factors are unobservable, the proposed CPI-Parser is required to separate key factors that satisfy the causal properties from an image. In this way, the parser is able to rely on causal factors w.r.t relevant evidence rather than non-causal factors w.r.t spurious correlations, thus alleviating model degradation and yielding improved parsing ability. Notably, the CPI-Parser is designed in a flexible way and can be integrated into any existing MHP frameworks. Extensive experiments conducted on three widely used benchmarks demonstrate the effectiveness and generalizability of our method. Code and models are released (https://github.com/HAG-uestc/CPI-Parser) for research purpose.

Cross-modal retrieval (e.g., query a given image to obtain a semantically similar sentence, and vice versa) is an important but challenging task, as the heterogeneous gap and inconsistent distributions exist between different modalities. The dominant approaches struggle to bridge the heterogeneity by capturing the common representations among heterogeneous data in a constructed subspace which can reflect the semantic closeness. However, insufficient consideration is taken into the fact that learned latent representations are actually heavily entangled with those semantic-unrelated features, which obviously further compounds the challenges of cross-modal retrieval. To alleviate the difficulty, this work makes an assumption that the data are jointly characterized by two independent features: semantic-shared and semantic-unrelated representations. The former presents characteristics of consistent semantics shared by different modalities, while the latter reflects the characteristics with respect to the modality yet unrelated to semantics, such as background, illumination, and other low-level information. Therefore, this paper aims to disentangle the shared semantics from the entangled features, andthus the purer semantic representation can promote the closeness of paired data. Specifically, this paper designs a novel Semantics Disentangling approach for Cross-Modal Retrieval (termed as SDCMR) to explicitly decouple the two different features based on variational auto-encoder. Next, the reconstruction is performed by exchanging shared semantics to ensure the learning of semantic consistency. Moreover, a dual adversarial mechanism is designed to disentangle the two independent features via a pushing-and-pulling strategy. Comprehensive experiments on four widely used datasets demonstrate the effectiveness and superiority of the proposed SDCMR method by achieving a new bar on performance when compared against 15 state-of-the-art methods.

The majority of existing works explore Unsupervised Domain Adaptation (UDA) with an ideal assumption that samples in both domains are available and complete. In real-world applications, however, this assumption does not always hold. For instance, data-privacy is becoming a growing concern, the source domain samples may be not publicly available for training, leading to a typical Source-Free Domain Adaptation (SFDA) problem. Traditional UDA methods would fail to handle SFDA since there are two challenges in the way: the data incompleteness issue and the domain gaps issue. In this paper, we propose a visually SFDA method named Adversarial Style Matching (ASM) to address both issues. Specifically, we first train a style generator to generate source-style samples given the target images to solve the data incompleteness issue. We use the auxiliary information stored in the pre-trained source model to ensure that the generated samples are statistically aligned with the source samples, and use the pseudo labels to keep semantic consistency. Then, we feed the target domain samples and the corresponding source-style samples into a feature generator network to reduce the domain gaps with a self-supervised loss. An adversarial scheme is employed to further expand the distributional coverage of the generated source-style samples. The experimental results verify that our method can achieve comparative performance even compared with the traditional UDA methods with source samples for training.

Approximating the uncertainty of an emotional AI agent is crucial for improving the reliability of such agents and facilitating human-in-the-loop solutions, especially in critical scenarios. However, none of the existing systems for emotion recognition in conversation (ERC) has attempted to estimate the uncertainty of their predictions. In this article, we present HU-Dialogue, which models hierarchical uncertainty for the ERC task. We perturb contextual attention weight values with source-adaptive noises within each modality, as a regularization scheme to model context-level uncertainty and adapt the Bayesian deep learning method to the capsule-based prediction layer to model modality-level uncertainty. Furthermore, a weight-sharing triplet structure with conditional layer normalization is introduced to detect both invariance and equivariance among modalities for ERC. We provide a detailed empirical analysis for extensive experiments, which shows that our model outperforms previous state-of-the-art methods on three popular multimodal ERC datasets.