Search

In this paper, we investigate the problem of procedure-aware action quality assessment, which analyzes the action quality by delving into the semantic and spatial-temporal relationships among various composed steps of the action. Most existing action quality assessment methods regress on deep features of entire videos to learn diverse scores, which ignore the relationships among different fine-grained steps in actions and result in limitations in visual interpretability and generalization ability. To address these issues, we construct a fine-grained competitive sports video dataset called FineDiving with detailed semantic and temporal annotations, which helps understand the internal structures of each action. We also propose a new approach (i.e., spatial-temporal segmentation attention, STSA) that introduces procedure segmentation to parse an action into consecutive steps, learns powerful representations from these steps by constructing spatial motion attention and procedure-aware cross-attention, and designs a fine-grained contrastive regression to achieve an interpretable scoring mechanism. In addition, we build a benchmark on the FineDiving dataset to evaluate the performance of representative action quality assessment methods. Then, we expand FineDiving to FineDiving+ and construct three new benchmarks to investigate the transferable abilities between different diving competitions, between synchronized and individual dives, and between springboard and platform dives to demonstrate the generalization abilities of our STSA in unknown scenarios, scoring rules, action types, and difficulty degrees. Extensive experiments demonstrate that our approach, designed for procedure-aware action quality assessment, achieves substantial improvements. Our dataset and code are available at https://github.com/xujinglin/FineDiving. [ABSTRACT FROM AUTHOR]

Current image-based keypoint detection methods for animal (including human) bodies and faces are generally divided into fully supervised and few-shot class-agnostic approaches. The former typically relies on laborious and time-consuming manual annotations, posing considerable challenges in expanding keypoint detection to a broader range of keypoint categories and animal species. The latter, though less dependent on extensive manual input, still requires necessary support images with annotation for reference during testing. To realize zero-shot keypoint detection without any prior annotation, we introduce the Open-Vocabulary Keypoint Detection (OVKD) task, which is innovatively designed to use text prompts for identifying arbitrary keypoints across any species. In pursuit of this goal, we have developed a novel framework named Open-Vocabulary Keypoint Detection with Semantic-feature Matching (KDSM). This framework synergistically combines vision and language models, creating an interplay between language features and local keypoint visual features. KDSM enhances its capabilities by integrating Domain Distribution Matrix Matching (DDMM) and other special modules, such as the Vision-Keypoint Relational Awareness (VKRA) module, improving the framework's generalizability and overall performance. Our comprehensive experiments demonstrate that KDSM significantly outperforms the baseline in terms of performance and achieves remarkable success in the OVKD task. Impressively, our method, operating in a zero-shot fashion, still yields results comparable to state-of-the-art few-shot species class-agnostic keypoint detection methods. Codes and data are available at https://github.com/zhanghao5201/KDSM. [ABSTRACT FROM AUTHOR]

This paper stems from the project A World of Possibilities. Modal pathways over an extra-long period of time: the diachrony of modality in the Latin language (WoPoss) which involves a corpus-based approach to the study of modality in the history of the Latin language. Linguistic annotation and, in particular, the semantic annotation of modality is a keystone of the project. Besides the difficulties intrinsic to any annotation task dealing with semantics, our annotation scheme involves multiple layers of annotation that are interconnected, adding complexity to the task. Considering the intricacies of our fine-grained semantic annotation, we needed to develop well-documented schemas in order to control the consistency of the annotation, but also to enable an efficient reuse of our annotated corpus. This paper presents the different elements involved in the annotation task, and how the description and the relations between the different linguistic components were formalised and documented, combining schema languages with XML documentation. [ABSTRACT FROM AUTHOR]

Accurately representing changes in mental states over time is crucial for understanding their complex dynamics. However, there is little methodological research on the validity and reliability of human-produced continuous-time annotation of these states. We present a psychometric perspective on valid and reliable construct assessment, examine the robustness of interval-scale (e.g., values between zero and one) continuous-time annotation, and identify three major threats to validity and reliability in current approaches. We then propose a novel ground truth generation pipeline that combines emerging techniques for improving validity and robustness. We demonstrate its effectiveness in a case study involving crowd-sourced annotation of perceived violence in movies, where our pipeline achieves a.95 Spearman correlation in summarized ratings compared to a.15 baseline. These results suggest that highly accurate ground truth signals can be produced from continuous annotations using additional comparative annotation (e.g., a versus b) to correct structured errors, highlighting the need for a paradigm shift in robust construct measurement over time. [ABSTRACT FROM AUTHOR]

Achieving reliable automatic left ventricle (LV) segmentation from echocardiograms is challenging due to the inherent sparsity of annotations in the dataset, as clinicians typically only annotate two specific frames for diagnostic purposes. Here we aim to address this challenge by introducing simplified LV segmentation (SimLVSeg), a novel paradigm that enables video-based networks for consistent LV segmentation from sparsely annotated echocardiogram videos. SimLVSeg consists of two training stages: (i) self-supervised pre-training with temporal masking, which involves pre-training a video segmentation network by capturing the cyclic patterns of echocardiograms from largely unannotated echocardiogram frames, and (ii) weakly supervised learning tailored for LV segmentation from sparse annotations. We extensively evaluated SimLVSeg using EchoNet-Dynamic, the largest echocardiography dataset. SimLVSeg outperformed state-of-the-art solutions by achieving a 93.32% (95% confidence interval: 93.21–93.43%) dice score while being more efficient. We further conducted an out-of-distribution test to showcase SimLVSeg's generalizability on distribution shifts (CAM US dataset). Our findings show that SimLVSeg exhibits excellent performance on LV segmentation with a relatively cheaper computational cost. This suggests that adopting video-based networks for LV segmentation is a promising research direction to achieve reliable LV segmentation. Our code is publicly available at https://github.com/BioMedIA-MBZUAI/SimLVSeg. [ABSTRACT FROM AUTHOR]

Storytelling is a long-established tradition and listening to stories is still a popular leisure activity. Caused by technization, storytelling media expands, e.g., to social robots acting as multi-modal storytellers, using different multimodal behaviours such as facial expressions or body postures. With the overarching goal to automate robotic storytelling, we have been annotating stories with emotion labels which the robot can use to automatically adapt its behavior. With it, three different approaches are compared in two studies in this paper: 1) manual labels by human annotators (MA), 2) software-based word-sensitive annotation using the Linguistic Inquiry and Word Count program (LIWC), and 3) a machine learning based approach (ML). In an online study showing videos of a storytelling robot, the annotations were validated, with LIWC and MA achieving the best, and ML the worst results. In a laboratory user study, the three versions of the story were compared regarding transportation and cognitive absorption, revealing no significant differences but a positive trend towards MA. On this empirical basis, the Automated Robotic Storyteller was implemented using manual annotations. Future iterations should include other robots and modalities, fewer emotion labels and their probabilities. [ABSTRACT FROM AUTHOR]

MVImgNet is a large-scale dataset that contains multi-view images of ~220k real-world objects in 238 classes. As a counterpart of ImageNet, it introduces 3D visual signals via multi-view shooting, making a soft bridge between 2D and 3D vision. This paper constructs the MVImgNet2.0 dataset that expands MVImgNet into a total of ~520k objects and 515 categories, which derives a 3D dataset with a larger scale that is more comparable to ones in the 2D domain. In addition to the expanded dataset scale and category range, MVImgNet2.0 is of a higher quality than MVImgNet owing to four new features: (i) most shoots capture 360° views of the objects, which can support the learning of object reconstruction with completeness; (ii) the segmentation manner is advanced to produce foreground object masks of higher accuracy; (iii) a more powerful structure-from-motion method is adopted to derive the camera pose for each frame of a lower estimation error; (iv) higher-quality dense point clouds are reconstructed via advanced methods for objects captured in 360° views, which can serve for downstream applications. Extensive experiments confirm the value of the proposed MVImgNet2.0 in boosting the performance of large 3D reconstruction models. MVImgNet2.0 will be public at luyues.github.io/mvimgnet2, including multi-view images of all 520k objects, the reconstructed high-quality point clouds, and data annotation codes, hoping to inspire the broader vision community. [ABSTRACT FROM AUTHOR]

Recent advances in multi-view multi-label learning are often hampered by the prevalent challenges of incomplete views and missing labels, common in real-world data due to uncertainties in data collection and manual annotation. These challenges restrict the capacity of the model to fully utilize the diverse semantic information of each sample, posing significant barriers to effective learning. Despite substantial scholarly efforts, many existing methods inadequately capture the depth of semantic information, focusing primarily on shallow feature extractions that fail to maintain semantic consistency. To address these shortcomings, we propose a novel Deep semantic structure-preserving (SSP) model that effectively tackles both incomplete views and missing labels. SSP innovatively incorporates a graph constraint learning (GCL) scheme to ensure the preservation of semantic structure throughout the feature extraction process across different views. Additionally, the SSP integrates a pseudo-labeling self-paced learning (PSL) strategy to address the often-overlooked issue of missing labels, enhancing the classification accuracy while preserving the distribution structure of data. The SSP model creates a unified framework that synergistically employs GCL and PSL to maintain the integrity of semantic structural information during both feature extraction and classification phases. Extensive evaluations across five real datasets demonstrate that the SSP method outperforms existing approaches, including lrMMC, MVL-IV, MvEL, iMSF, iMvWL, NAIML, and DD-IMvMLC-net. It effectively mitigates the impacts of data incompleteness and enhances semantic representation fidelity. [ABSTRACT FROM AUTHOR]

Calcium (Ca) and cadmium (Cd) are transition metals coexisting in the ecosystem. Ca is indispensable for the growth and development of plants as well as animals, while Cd is regarded as a toxic heavy metal for the living system. The transportation of Cd in the biological systems often used the pathways of Ca because of chemical similarities. High concentrations of cadmium replace Ca, Mn, and Zn from their respective metalloprotein sites and strongly associated with them. Replaced minerals from their metalloprotein sites are often released as an oxidative ion that is detrimental to it. The common transportation mechanism of Ca and Cd is implicit in the role of common and similar transporters for transporting them in plants. Thus, our study was done to identify the transporters for Ca and Cd and characterize them for similarity in terms of cotransportation system. A profile-based search program identified 44 transporters genes for Ca transportation and 70 genes for cadmium transportation. They were categorized into different groups based on the presence of signature motifs and domains. Identified transporters were characterized for genomic distribution, gene structure, annotation, conserved signature motifs, and domain. Further, cis motif analysis, heat map, gene ontology, and protein–protein interaction were conducted for Ca and Cd transporter genes. In silico expression showed Os05g0319800-1304 and Os0319800-6065 transporter genes were overexpressed for Ca and Os07g00232800-40298 and Os07g00384500-25924 transporter genes overexpressed for Cd transporter. These genes could be used as a candidate genes for enhancing the Ca concentration with reduced Cd content in rice using biotechnological approaches. Twenty-seven genes were found as the common transporters for Ca and Cd. Both active and passive transporter mechanisms act as cotransporters for Ca and Cd. The common signature motifs and domains can be targeted for the characterization of cotransporters of different minerals. [ABSTRACT FROM AUTHOR]

In this study, a chromosome-level genome of the tropical sea cucumber Stichopus monotuberculatus was generated by a combination of Nanopore long-read, Illumina short-read, and Hi-C sequencing technologies. The final assembly was 810.54 Mb in length, with contig N50 and scaffold N50 values of 10.15 Mb and 35.36 Mb, respectively. This assembly comprised 23 pseudo-chromosomes, covering 99.82% of the genome. Completeness analysis using BUSCO indicated that 97.8% of the metazoan conserved genes were presented in their entirety. A total of 29,596 protein-coding genes were predicted, with functional annotations available for 94.43% of these genes. The high-quality genome assembly produced in this study may provide an essential foundation for future researches on resource conservation and genetic breeding of S. monotuberculatus. [ABSTRACT FROM AUTHOR]

Given a set of labels, multi-label text classification (MLTC) aims to assign multiple relevant labels for a text. Recently, deep learning models get inspiring results in MLTC. Training a high-quality deep MLTC model typically demands large-scale labeled data. And comparing with annotations for single-label data samples, annotations for multi-label samples are typically more time-consuming and expensive. Active learning can enable a classification model to achieve optimal prediction performance using fewer labeled samples. Although active learning has been considered for deep learning models, there are few studies on active learning for deep multi-label classification models. In this work, for the deep MLTC model, we propose a deep Active Learning method based on Bayesian deep learning and Expected confidence (BEAL). It adopts Bayesian deep learning to derive the deep model's posterior predictive distribution and defines a new expected confidence-based acquisition function to select uncertain samples for deep MLTC model training. Moreover, we perform experiments with a BERT-based MLTC model, where BERT can achieve satisfactory performance by fine-tuning in various classification tasks. The results on benchmark datasets demonstrate that BEAL enables more efficient model training, allowing the deep model to achieve training convergence with fewer labeled samples. [ABSTRACT FROM AUTHOR]

Traditional methods for producing architectural drawings require extensive manual labor. This paper proposes an automated method for generating a comprehensive set of three‐view drawings, including the standardized labeling of doors and annotation of dimensions and areas. The output drawings are software‐readable and editable, and the method is applicable to intricate structures with non‐orthogonal or curved walls. To fully validate the accuracy of the proposed method, two distinct building scenarios were selected for experimentation. [ABSTRACT FROM AUTHOR]

When the data distributions between the source and target domains are inconsistent, unsupervised cross domain object detection improves the performance of the detector without the need to add any additional annotation information. However, existing advanced methods overlook the semantic consistency of images, which leads to incorrect transfer knowledge in feature learning and reduces adaptive performance. Therefore, we propose a feature correlation learning method, called Semantic Consistency Knowledge Transfer (SCKT) for unsupervised cross domain object detection, which focuses on similar semantic information and achieves fine-grained corresponding scales domain adaptation. Specifically, we design a Multi-channel Adaptive Correction (MAC) module for image enhancement, which alleviates pixel distortion and maintains semantic consistency. Furthermore, the proposed Related Knowledge Mining (RKM) module consists of Feature Correlation Learning (FCL) and Consistent Knowledge Learning (CKL), which use generated semantic consistency features to capture different levels of image details and semantic information, achieving corresponding domain alignment at multiple levels. Most importantly, SCKT does not rely on a specific network structure, and does not reduce inference speed, which is suitable for different object detectors. Extensive experiments show that our SCKT can exhibit optimal performance in multiple domain migration scenarios. Especially for the challenging PASCAL VOC → Clipart, SCKT generates 54.6% mAP. [ABSTRACT FROM AUTHOR]

Measurement techniques often result in domain gaps among batches of cellular data from a specific modality. The effectiveness of cross-batch annotation methods is influenced by inductive bias, which refers to a set of assumptions that describe the behavior of model predictions. Different annotation methods possess distinct inductive biases, leading to varying degrees of generalizability and interpretability. Given that certain cell types exhibit unique functional patterns, we hypothesize that the inductive biases of cell annotation methods should align with these biological patterns to produce meaningful predictions. In this study, we propose KIDA, Knowledge-based Inductive bias and Domain Adaptation. The knowledge-based inductive bias constrains the prediction rules learned from the reference dataset, composed of multiple batches, to functional patterns relevant to biology, thereby enhancing the generalization of the model to unseen batches. Since the query dataset also contains gaps from multiple batches, KIDA's domain adaptation employs pseudo labels for self-knowledge distillation, effectively narrowing the distribution gap between model predictions and the query dataset. Benchmark experiments demonstrate that KIDA is capable of achieving accurate cross-batch cell type annotation. Knowledge-based inductive bias and domain adaptation can enhance the cell type annotation accuracy of deep learning models. [ABSTRACT FROM AUTHOR]

Incorporating medical text annotations compensates for the quality deficiencies of image data, effectively overcoming the limitations of medical image segmentation. Many existing approaches achieve high-quality segmentation results by integrating text into the image modality. However, these approaches require matched image-text pairs during inference to maintain their performance, and the absence of corresponding text annotations results in degraded model performance. Additionally, these methods often assume that the input text annotations are ideal, overlooking the impact of poor-quality text on model performance in practical scenarios. To address these issues, we propose a novel generative medical image segmentation model, Cap2Seg (Leveraging Caption Generation for Enhanced Segmentation of COVID-19 Medical Images). Cap2Seg not only segments lesion areas but also generates related medical text descriptions, guiding the segmentation process. This design enables the model to perform optimal segmentation without requiring text input during inference. To mitigate the impact of inaccurate text on model performance, we consider the consistency between generated textual features and visual features and introduce the Scale-aware Textual Attention Module (SATaM), which reduces the model's dependency on irrelevant or misleading text information. Subsequently, we design a word-pixel fusion decoding mechanism that effectively integrates textual features into visual features, ensuring that the text information effectively supplements and enhances the image segmentation task. Extensive experiments on two public datasets, MosMedData+ and QaTa-COV19, demonstrate that our method outperforms the current state-of-the-art models under the same conditions. Additionally, ablation studies have been conducted to demonstrate the effectiveness of each proposed module. The code is available at https://github.com/AllenZzzzzzzz/Cap2Seg. [ABSTRACT FROM AUTHOR]

This study explored the pupil's reading comprehension using the Preview, Question, Read, Reflect, Recite, and Review (PQ4R) and Annotations as a basis for an instructional tool. Grade six learners are struggling with reading comprehension, especially during the pandemic. It is pivotal that educational strategy and technique should be tested and explored to address the gap. This study used the PQ4R and annotations to help the learners understand the concepts and subject matter. This study utilized a sequential explanatory mixed-method research design. This design followed a sequence of gathering data from quantitative for analysis and interpretation to qualitative for exploration and triangulation. For the specific design of this method, a Quasi-experimental will be used to understand the numerical value of this study and a descriptive qualitative for the qualitative data. Purposive sampling design will be used in this study following criteria to identify the prospected respondents. There were 30 respondents from Tuble Elementary School. This study utilized a validated instrument. The findings of the study stipulated that the results of the experimental group (p-value = 0.28) and the control group (p-value = 0.34) established no significant difference. With this, teachers must explore more strategies and techniques in education so that learners can get the best learning experiences that they deserve. This will provide a reference on how teachers can create instructional tools to increase classroom productivity. [ABSTRACT FROM AUTHOR]

In semiconductor manufacturing, defect detection is pivotal for enhancing productivity and yield. This paper introduces a novel weakly supervised method, the Implicit Cross Domain Diffusion Model (ICDDM), designed to tackle defect pattern segmentation challenges in the absence of detailed pixel-wise annotations. ICDDM employs a generative model to estimate the joint distribution of images depicting defect patterns and background circuits, formulating this estimation as a Markov Chain and optimizing it through denoising score matching. Building on this, we propose the Cross Domain Latent Diffusion Model (CDLDM), inspired by the Latent Diffusion Model, which simplifies the diffusion process into a lower-dimensional latent space to boost detection efficiency. Further enhancing our model, we introduce the Knowledge Distillation Cross Domain Diffusion Model (KDCDDM), which utilizes CDLDM as a teacher model and a Generative Adversarial Network (GAN) as a student model. This approach significantly accelerates the diffusion process by reducing the number of necessary denoising iterations while maintaining robust model performance. This suite of techniques offers a comprehensive solution for efficient and effective defect detection in semiconductor production environments. [ABSTRACT FROM AUTHOR]

State-of-the-art instance segmentation approaches, such as Mask R-CNN, have exhibited remarkable performance under clear weather conditions. However, their effectiveness is significantly compromised in foggy environments, primarily due to reduced visibility and obscured object details. To address this challenge, we introduce a joint defogging learning with boundary refinement (JDLMask) framework. Unlike conventional strategies that treat image dehazing as a preprocessing step, JDLMask employs a shared structure that enables the joint learning of defogging and instance segmentation. This integrated approach greatly bolsters the model's adaptability to foggy scenarios. Recognizing challenges in feature extraction due to fog interference, we propose a multi-scale feature fusion mask head, based on the encoder–decoder architecture. This component is designed to acquire both local and global information, thereby enhancing the model's feature representation capacity. Furthermore, we integrate a boundary refinement module, which sharpens the model's localization accuracy by focusing on critical boundary details. Addressing the scarcity of datasets tailored, for instance, segmentation in real-world foggy scenes, we have enriched the Foggy Driving dataset with meticulously crafted instance mask annotations and named it the Foggy Driving InstanceSeg. Comprehensive experiments demonstrate JDLMask's superiority. Compared to the baseline Mask R-CNN, JDLMask achieves improvements of 4.4% and 3.8% in mask AP on the Foggy Cityscapes and Cityscapes validation sets, respectively, and a 2.5% gain on the Foggy Driving InstanceSeg dataset. [ABSTRACT FROM AUTHOR]

With the rapid development of depth sensor, more and more RGB-D videos could be obtained. Identifying the foreground in RGB-D videos is a fundamental and important task. However, the existing salient object detection (SOD) works only focus on either static RGB-D images or RGB videos, ignoring the collaborating of RGB-D and video information. In this paper, we first collect a new annotated RGB-D video SOD (ViDSOD-100) dataset, which contains 100 videos within a total of 9362 frames, acquired from diverse natural scenes. All the frames in each video are manually annotated to a high-quality saliency annotation. Moreover, we propose a new baseline model, named attentive triple-fusion network (ATF-Net), for RGB-D video salient object detection. Our method aggregates the appearance information from an input RGB image, spatio-temporal information from an estimated motion map, and the geometry information from the depth map by devising three modality-specific branches and a multi-modality integration branch. The modality-specific branches extract the representation of different inputs, while the multi-modality integration branch combines the multi-level modality-specific features by introducing the encoder feature aggregation (MEA) modules and decoder feature aggregation (MDA) modules. The experimental findings conducted on both our newly introduced ViDSOD-100 dataset and the well-established DAVSOD dataset highlight the superior performance of the proposed ATF-Net.This performance enhancement is demonstrated both quantitatively and qualitatively, surpassing the capabilities of current state-of-the-art techniques across various domains, including RGB-D saliency detection, video saliency detection, and video object segmentation. We shall release our data, our results, and our code upon the publication of this work. [ABSTRACT FROM AUTHOR]

Amplicon sequencing has long served as a robust method for characterising microbial communities, and despite inherent resolution limitations, it remains a preferred technique, offering cost‐ and time‐effective insights into bacterial compositions. Here, we introduce ONT‐AmpSeq, a user‐friendly pipeline designed for processing amplicon sequencing data generated from Oxford Nanopore Technology (ONT) devices. Our pipeline enables efficient creation of taxonomically annotated operational taxonomic unit (OTU) tables from ONT sequencing data, with the flexibility to multiplex amplicons on the same barcode. The pipeline encompasses six main steps—statistics, quality filtering, alignment, clustering, polishing, and taxonomic classification—integrating various state‐of‐the‐art software tools. We provide a detailed description of each step, along with performance tests and robustness evaluations using both test data and a ZymoBIOMICS® Microbial Community Standard mock community dataset. Our results demonstrate the ability of ONT‐AmpSeq to effectively process ONT amplicon data, offering valuable insights into microbial community composition. Additionally, we discuss the influence of polishing tools on taxonomic insight and the impact of taxonomic annotation methods on the derived microbial composition. Overall, ONT‐AmpSeq represents a comprehensive solution for analysing ONT amplicon sequencing data, facilitating streamlined and reliable microbial community analysis. The pipeline, along with test data, is freely available for public use. [ABSTRACT FROM AUTHOR]

Copyright of ABI Technik is the property of De Gruyter and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Deep learning is revolutionizing various scientific fields, with medical applications at the forefront. One key focus is automating image segmentation, a process crucial in many clinical services. However, medical images are often ambiguous and challenging even for experts. To address this, reliable models need to quantify their uncertainty, allowing physicians to understand the model's confidence in its segmentation. This paper explores how the performance and uncertainty of a model are influenced by the number of annotations per input sample. We examine the effects of both single and multiple manual annotations on various deep learning architectures. To tackle this question, we employ three widely recognized deep learning architectures and evaluate them across four publicly available datasets. Furthermore, we explore the effects of dropout rates on Monte Carlo models by examining uncertainty models with dropout rates of 20%, 40%, 60%, and 80%. Subsequently, we evaluate the models using various measurement metrics. The findings reveal that the influence of multiple annotations varies significantly depending on the datasets. Additionally, we observe that the dropout rate has minimal or no impact on the model's performance unless there is a substantial loss of training data, primarily evident in the 80% dropout rate scenario. [ABSTRACT FROM AUTHOR]

Accurate crack detection is essential for structural damage assessment after earthquake disasters. However, due to the gap between the target domain of the detected structure and the source domain, it is challenging to achieve high-precision crack segmentation when performing crack detection based on deep learning (DL) in actual engineering. This article proposes a crack segmentation transfer learning method based on a self-supervised learning mechanism and a high-quality pseudo-label generation method, which can significantly improve the detection accuracy in the target domain without pre-made annotations. Besides, to improve the crack segmentation model's ability to extract local and global features, this article proposes a SegCrackFormer model, which embeds convolutional layers and multi-head self-attention modules. An experiment of the crack segmentation transfer learning method is performed on two open-source crack datasets, METU and Crack500, and a newly proposed LD dataset. The experimental results show that the crack segmentation transfer learning method proposed in this article can improve the mean intersection over union (mIoU) by 38.41% and 15.66% on the Crack500 and LD datasets, respectively. The proposed SegCrackFormer is evaluated through comparative experiments, which demonstrate its superiority over existing crack segmentation models on the METU dataset. Additionally, the proposed method is shown to require significantly less computational resources than other existing models, which highlights the potential of SegCrackFormer as a powerful and efficient model for crack segmentation in practical applications. [ABSTRACT FROM AUTHOR]

Sparsely annotated image segmentation has gained popularity due to its ability to significantly reduce the labeling burden on training data. However, existing methods still struggle to learn complete object structures, especially for complex shadow objects. This paper discusses two prevalent issues existing in previous methods, i.e., generating noisy pseudo labels and misdetecting ambiguous regions. To tackle these challenges, we propose a novel weakly-supervised learning framework to boost sparsely annotated shadow detection. Concretely, a reliable label propagation (RLP) scheme is first designed to diffuse sparse annotations into unlabeled regions, thereby generating denser pseudo shadow masks. This scheme effectively reduces the number of noisy labels by incorporating uncertainty analysis. Then, a multi-cue semantic calibration (MSC) strategy is presented to refine the semantic features extracted from the backbone by employing edge, global, and adjacent priors. Embedded with MSC, the detection network becomes more discriminative against ambiguous regions. By combining RLP and MSC, the proposed weakly-supervised framework can detect complete and accurate shadow regions from sparse annotations. Experimental results on three benchmark datasets demonstrate that our method achieves comparable performance to recent fully-supervised methods, while requiring only about 4.5% of the pixels to be labeled. Boosting sparsely annotated shadow detection [ABSTRACT FROM AUTHOR]

This article presents a pipeline for automated fact-checking leveraging publicly available language models and data. The objective is to assess the accuracy of textual claims using evidence from a ground-truth evidence corpus. The pipeline consists of two main modules—the evidence retrieval and the claim veracity evaluation. Our primary focus is on the ease of deployment in various languages that remain unexplored in the field of automated fact-checking. Unlike most similar pipelines, which work with evidence sentences, our pipeline processes data on a paragraph level, simplifying the overall architecture and data requirements. Given the high cost of annotating language-specific fact-checking training data, our solution builds on the question answering for claim generation method, which we adapt and use to generate the data for all models of the pipeline. Our strategy enables the introduction of new languages through machine translation of only two fixed datasets of moderate size. Subsequently, any number of training samples can be generated based on an evidence corpus in the target language. We provide open access to all data and fine-tuned models for Czech, English, Polish, and Slovak pipelines, as well as to our codebase that may be used to reproduce the results. We comprehensively evaluate the pipelines for all four languages, including human annotations and per-sample difficulty assessment using Pointwise V -information. The presented experiments are based on full Wikipedia snapshots to promote reproducibility. To facilitate implementation and user interaction, we develop the FactSearch application featuring the proposed pipeline and the preliminary feedback on its performance. [ABSTRACT FROM AUTHOR]

Gesture recognition and its applications have been widely studied and received much attention in recent years. Existing works on hand gesture recognition aim to train classification models based on several discrete categories, which suffers from time-consuming data collection and low perceptual granularity. Differently, this work proposes a contrastive framework for fine-grained few-shot gesture recognition. To achieve this, we construct a general and compact gesture embedding space to represent arbitrary intricate hand gestures. The embedding distance between hand gestures is consistent with their similarity, reflecting their subtle variations accurately. To learn such an embedding space, we build up a large-scale hand gesture similarity dataset named SimGesture, relying on 944,482 hand image triplets of gesture comparison annotations. Based on SimGesture, we utilize contrastive learning to train a neural network named SimGesNet being capable of projecting arbitrary hand images into a compact gesture embedding space. Our experimental results demonstrate that the learned embedding can be used with great success for few-shot gesture recognition and achieves SOTA results. We also show that our proposed gesture embedding outperforms existing embeddings in representing fine-grained gestures. [ABSTRACT FROM AUTHOR]

Genome annotation is an important but challenging task. Accurate identification of short interspersed nuclear elements (SINEs) is particularly difficult due to their lack of highly conserved sequences. AnnoSINE is state-of-the-art software for annotating SINEs in plant genomes, but it is computationally inefficient for large genomes. Moreover, its applicability to animals is limited due to the absence of animal pHMMs in its HMM library. Therefore, we propose AnnoSINE_v2, which extends accurate SINE annotation for animal genomes with greatly optimized computational efficiency. Our results show that AnnoSINE_v2's annotation of SINEs has over 20% higher F1-score compared to the existing tools on animal genomes and enables the processing of complicated genomes, like human and zebrafish, which were beyond the capabilities of AnnoSINE_v1. AnnoSINE_v2 is freely available on Conda and GitHub: https://github.com/liaoherui/AnnoSINE_v2. [ABSTRACT FROM AUTHOR]

Generating pixel-level annotations for semantic segmentation tasks of high-resolution remote sensing images is both time-consuming and labor-intensive, which has led to increased interest in unsupervised methods. Therefore, in this paper, we propose an unsupervised multi-scale hybrid feature extraction network based on the CNN-Transformer architecture, referred to as MSHFE-Net. The MSHFE-Net consists of three main modules: a Multi-Scale Pixel-Guided CNN Encoder, a Multi-Scale Aggregation Transformer Encoder, and a Parallel Attention Fusion Module. The Multi-Scale Pixel-Guided CNN Encoder is designed for multi-scale, fine-grained feature extraction in unsupervised tasks, efficiently recovering local spatial information in images. Meanwhile, the Multi-Scale Aggregation Transformer Encoder introduces a multi-scale aggregation module, which further enhances the unsupervised acquisition of multi-scale contextual information, obtaining global features with stronger feature representation. The Parallel Attention Fusion Module employs an attention mechanism to fuse global and local features in both channel and spatial dimensions in parallel, enriching the semantic relations extracted during unsupervised training and improving the performance of unsupervised semantic segmentation. K-means clustering is then performed on the fused features to achieve high-precision unsupervised semantic segmentation. Experiments with MSHFE-Net on the Potsdam and Vaihingen datasets demonstrate its effectiveness in significantly improving the accuracy of unsupervised semantic segmentation. [ABSTRACT FROM AUTHOR]

Electric bicycle datasets for lift control have played a crucial role in the development of electric bicycle lift entry detection algorithms. However, existing datasets often encounter issues, including a small dataset size, a high false detection rate, and the absence of a publicly available benchmark dataset. This paper addresses these challenges by constructing a new electric bicycle detection dataset for lift control based on the research demands of the electric bicycle detection task for lift control. XHNet_EB, a car scene image dataset covering various types of electric bicycles, was constructed based on images taken by a camera from above. Segmented annotation was employed, framing the front and rear of the electric bicycle independently. This approach effectively addressed the problems of many irrelevant features and a high false detection rate caused by the use of whole-electric-bicycle annotations in mainstream datasets. AP_50, AP_75, the mAP, and the number of false detections in the images were used for quantitative analysis of the dataset. The experimental results indicated that the object detection accuracy of the XHNet_EB dataset was excellent. Quantitative analysis and evaluation of the number of false detections in images were conducted using four mainstream lightweight detection model algorithms. The results demonstrated that segmented annotation reduced the false detection rate more effectively than entire electric bicycle annotation. This study identified drawbacks in existing datasets. The dataset proposed in this paper overcomes the shortcomings of existing commercial data and solves problems such as the high false detection rate caused by the inclusion of many irrelevant features caused by the "whole-electric-bicycle annotation" method, which could help with the development of electric bicycle detection applications in lifts. [ABSTRACT FROM AUTHOR]

In this rapidly evolving era of multimodal generation, diffusion models exhibit impressive generative capabilities, significantly enhancing the realm of creative image synthesis by intricately textual prompts. Yet, their effectiveness is limited in certain niche sectors, like depicting Chinese ancient architecture. This limitation is primarily due to the insufficient data that fails to encompass the unique architectural features and corresponding text information. Hence, we build an extensive multimodal dataset capturing the essence of Chinese architectures mostly from the Tang to the Yuan Dynasties. The dataset is categorized on the types, including image&text, video, and style models. In details, images and videos are methodically categorized based on locations. All images are annotated at two levels: initial annotations and descriptive terms based on distinctive characteristics and official information. Moreover, seven artistic styles fine-tuning models are provided in our dataset for further innovations. Significantly, this is the first Chinese ancient architecture dataset and the instance of using the Pinyin system to annotate unique terms related to Chinese architectural styles. [ABSTRACT FROM AUTHOR]

The conventional approach to event extraction requires predefined event types and their corresponding annotations to train event extractors. However, these prerequisites are often difficult to satisfy in real-world applications. To automatically induct event types, most work has been devoted to clustering event triggers, where a cluster of event triggers is represented as an event type. Some works use trigger semantics, while others use co-occurrence relationships to cluster triggers. However, the clustering results of event triggers obtained by the above work are not sufficiently detailed in describing event types, making it difficult to accurately determine the corresponding event types manually. This paper proposes an open-domain event type induction framework that automatically discovers a set of event types from a given corpus. Unlike previous work on event trigger clustering, this paper takes into consideration the hierarchical relationship of event types to partition the event trigger clusters into event mains and subtypes. The framework employs a latent variable-based neural generation module and a semantic-based clustering module, the former of which obtains event trigger clusters representing the main types of events by jointly projecting the co-occurrence and semantic information of event triggers into a latent space for event type latent variable mining, and the latter of which further divides these event trigger clusters into event subtypes based on semantic information. Finally, experiment results show that, compared with the benchmark model, the ETGen-Clus can improve event type quality scores of 6.23% and 3.11% on the two datasets, respectively. [ABSTRACT FROM AUTHOR]

Recovering the complete structure from partial point clouds in arbitrary poses is challenging. Recently, many efforts have been made to address this problem by developing SO(3)‐equivariant completion networks or aligning the partial point clouds with a predefined canonical space before completion. However, these approaches are limited to random rotations only or demand costly pose annotation for model training. In this paper, we present a novel Network for Point cloud Completion with Learnable Canonical space (PCLC‐Net) to reduce the need for pose annotations and extract SE(3)‐invariant geometry features to improve the completion quality in arbitrary poses. Without pose annotations, our PCLC‐Net utilizes self‐supervised pose estimation to align the input partial point clouds to a canonical space that is learnable for an object category and subsequently performs shape completion in the learned canonical space. Our PCLC‐Net can complete partial point clouds with arbitrary SE(3) poses without requiring pose annotations for supervision. Our PCLC‐Net achieves state‐of‐the‐art results on shape completion with arbitrary SE(3) poses on both synthetic and real scanned data. To the best of our knowledge, our method is the first to achieve shape completion in arbitrary poses without pose annotations during network training. [ABSTRACT FROM AUTHOR]

Motivation Recent advances in sequencing technology provide opportunities to study biological processes at a higher resolution. Cell type annotation is an important step in scRNA-seq analysis, which often relies on established marker genes. However, most of the previous methods divide the identification of cell types into two stages, clustering and assignment, whose performances are susceptible to the clustering algorithm, and the marker information cannot effectively guide the clustering process. Furthermore, their linear heuristic-based cell assignment process is often insufficient to capture potential dependencies between cells and types. Results Here, we present I nterpretable C ell T ype A nnotation based on s elf-training (sICTA), a marker-based cell type annotation method that combines the self-training strategy with pseudo-labeling and the nonlinear association capturing capability of Transformer. In addition, we incorporate biological priori knowledge of genes and pathways into the classifier through an attention mechanism to enhance the transparency of the model. A benchmark analysis on 11 publicly available single-cell datasets demonstrates the superiority of sICTA compared to state-of-the-art methods. The robustness of our method is further validated by evaluating the prediction accuracy of the model on different cell types for each single-cell data. Moreover, ablation studies show that self-training and the ability to capture potential dependencies between cells and cell types, both of which are mutually reinforcing, work together to improve model performance. Finally, we apply sICTA to the pancreatic dataset, exemplifying the interpretable attention matrix captured by sICTA. Availability and implementation The source code of sICTA is available in public at https://github.com/nbnbhwyy/sICTA. The processed datasets can be found at https://drive.google.com/drive/folders/1jbqSxacL%5fIDIZ4uPjq220C9Kv024m9eL. The final version of the model will be permanently available at https://doi.org/10.5281/zenodo.13474010 [ABSTRACT FROM AUTHOR]

Tarikh-e Beyhaghi (Beyhaghi’s History) is a seminal work in Persian literature and historiography. Fayyaz dedicated his life to correcting and improving this text. In 2009, a revised edition of the work edited by Yahagghi and Seyyedi was published. The primary advantage of this new edition was the detailed introduction, annotations, and supplementary materials, which greatly facilitated the work of readers and researchers. However, the editors had made numerous unjustified interventions in the text, to the extent that this benefit was largely overshadowed by the distortion of the original work of such great significance. It was anticipated that after more than a decade since the initial publication and appearance of numerous scholarly articles on the topic, many of the problems would have been addressed in the new edition. Regrettably, the 2022 edition, which claimed to incorporate "complete revision, additions, and corrections", failed to live up to these expectations as only a few of the numerous issues were rectified. Due to the impracticality of addressing all the flaws of this editorial effort in a single article, the present work examined select problems under the following headings: "Departure from the normative contextual meaning", "Disregard for Bayhaghi's stylistic features", "Ignoring the frequency of evidence", "Unjustified textual changes", "Unsupported readings of the text", and "Neglect of literary sensibilities in text correction". [ABSTRACT FROM AUTHOR]

Recent advancements in neural machine translation (NMT) offer promising potential for generating cross-language clinical natural language processing (NLP) resources. There is a pressing need to be able to foster the development of clinical NLP tools that extract key clinical entities in a comparable way for a multitude of medical application scenarios that are hindered by lack of multilingual annotated data. This study explores the efficacy of using NMT and annotation projection techniques with expert-in-the-loop validation to develop named entity recognition (NER) systems for an under-resourced target language (Catalan) by leveraging Spanish clinical corpora annotated by domain experts. We employed a state-of-the-art NMT system to translate three clinical case corpora. The translated annotations were then projected onto the target language texts and subsequently validated and corrected by clinical domain experts. The efficacy of the resulting NER systems was evaluated against manually annotated test sets in the target language. Our findings indicate that this approach not only facilitates the generation of high-quality training data for the target language (Catalan) but also demonstrates the potential to extend this methodology to other languages, thereby enhancing multilingual clinical NLP resource development. The generated corpora and components are publicly accessible, potentially providing a valuable resource for further research and application in multilingual clinical settings. [ABSTRACT FROM AUTHOR]

This article proposes a weakly supervised multi-feature fusion pedestrian re-identification method, which introduces a multi-feature fusion mechanism to extract feature information from different layers into the same feature space and fuse them into the deep and shallow joint features. The goal is to fully utilize the rich information in the image and improve the performance and robustness of the pedestrian re-identification model. Secondly, by matching the target character with unprocessed surveillance videos, one only needs to know that the identity of a person appears in the video, without annotating the identity of a person in any of the frames of the video during the training process. This simplifies the annotation of training images by replacing accurate annotations with broad annotations; that is, it puts the pedestrian identities that appeared in the video in one package and assigns a video-level label to each package. This greatly reduces the annotation work and transforms this weakly supervised pedestrian re-identification challenge into a multi-instance and multi-label learning problem. The experimental results show that the method proposed in this paper is effective and can significantly improve mAP. [ABSTRACT FROM AUTHOR]

Interactive image segmentation extremely accelerates the generation of high-quality annotation image datasets, which are the pillars of the applications of deep learning. However, these methods suffer from the insignificance of interaction information and excessively high optimization costs, resulting in unexpected segmentation outcomes and increased computational burden. To address these issues, this paper focuses on interactive information mining from the network architecture and optimization procedure. In terms of network architecture, the issue mentioned above arises from two perspectives: the less representative feature of interactive regions in each layer and the interactive information weakened by the network hierarchy structure. Therefore, the paper proposes a network called EnNet. The network addresses the two aforementioned issues by employing attention mechanisms to integrate user interaction information across the entire image and incorporating interaction information twice in a design that progresses from coarse to fine. In terms of optimization, this paper proposes a method of using zero-order optimization during the first four iterations of training. This approach can reduce computational overhead with only a minimal reduction in accuracy. The experimental results on GrabCut, Berkeley, DAVIS, and SBD datasets validate the effectiveness of the proposed method, with our approach achieving an average NOC@90 that surpasses RITM by 0.35. [ABSTRACT FROM AUTHOR]

How to effectively leverage the plentiful existing datasets to train a robust and high-performance model is of great significance for many practical applications. However, a model trained on a naive merge of different datasets tends to obtain poor performance due to annotation conflicts and domain divergence. In this paper, we attempt to train a unified model that is expected to perform well across domains on several popularity segmentation datasets. We conduct a comprehensive analysis to assess the impact of various training schemes and model selection on multi-domain learning with extensive experiments. Based on the analysis, we propose a robust solution that consistently enhances the model performance across different domains. Our solution ranks 2nd on RVC 2022 semantic segmentation task, with a dataset only 1/3 size of the 1st model used. [ABSTRACT FROM AUTHOR]

Markerless methods for animal posture tracking have been rapidly developing recently, but frameworks and benchmarks for tracking large animal groups in 3D are still lacking. To overcome this gap in the literature, we present 3D-MuPPET, a framework to estimate and track 3D poses of up to 10 pigeons at interactive speed using multiple camera views. We train a pose estimator to infer 2D keypoints and bounding boxes of multiple pigeons, then triangulate the keypoints to 3D. For identity matching of individuals in all views, we first dynamically match 2D detections to global identities in the first frame, then use a 2D tracker to maintain IDs across views in subsequent frames. We achieve comparable accuracy to a state of the art 3D pose estimator in terms of median error and Percentage of Correct Keypoints. Additionally, we benchmark the inference speed of 3D-MuPPET, with up to 9.45 fps in 2D and 1.89 fps in 3D, and perform quantitative tracking evaluation, which yields encouraging results. Finally, we showcase two novel applications for 3D-MuPPET. First, we train a model with data of single pigeons and achieve comparable results in 2D and 3D posture estimation for up to 5 pigeons. Second, we show that 3D-MuPPET also works in outdoors without additional annotations from natural environments. Both use cases simplify the domain shift to new species and environments, largely reducing annotation effort needed for 3D posture tracking. To the best of our knowledge we are the first to present a framework for 2D/3D animal posture and trajectory tracking that works in both indoor and outdoor environments for up to 10 individuals. We hope that the framework can open up new opportunities in studying animal collective behaviour and encourages further developments in 3D multi-animal posture tracking. [ABSTRACT FROM AUTHOR]

Shadow detection is essential for scene understanding and image restoration. Existing paradigms for producing shadow detection training data usually rely on densely labeling each image pixel, which will lead to a bottleneck when scaling up the number of images. To tackle this problem, by labeling shadow images with only a few strokes, this paper designs a learning framework for Weakly supervised Shadow Detection, namely WSD. Firstly, it creates two shadow detection datasets with scribble annotations, namely Scr-SBU and Scr-ISTD. Secondly, it proposes an uncertainty-guided label augmentation scheme based on graph convolutional networks, which can propagate the sparse scribble annotations to more reliable regions, and then avoid the model converging to an undesired local minima as intra-class discontinuity. Finally, it introduces a multi-task learning framework to jointly learn for shadow detection and edge detection, which encourages generated shadow maps to be comprehensive and well aligned with shadow boundaries. Experimental results on benchmark datasets demonstrate that our framework even outperforms existing semi-supervised and fully supervised shadow detectors requiring only 2% pixels to be labeled. [ABSTRACT FROM AUTHOR]

Gene-based burden tests are a popular and powerful approach for analysis of exome-wide association studies. These approaches combine sets of variants within a gene into a single burden score that is then tested for association. Typically, a range of burden scores are calculated and tested across a range of annotation classes and frequency bins. Correlation between these tests can complicate the multiple testing correction and hamper interpretation of the results. We introduce a method called the sparse burden association test (SBAT) that tests the joint set of burden scores under the assumption that causal burden scores act in the same effect direction. The method simultaneously assesses the significance of the model fit and selects the set of burden scores that best explain the association at the same time. Using simulated data, we show that the method is well calibrated and highlight scenarios where the test outperforms existing gene-based tests. We apply the method to 73 quantitative traits from the UK Biobank, showing that SBAT is a valuable additional gene-based test when combined with other existing approaches. This test is implemented in the REGENIE software. Gene-based burden tests are commonly used in exome-wide association studies. We introduce SBAT (sparse burden association test), which jointly models a set of burden scores under the assumption that the causal burden scores act in the same effect direction. We apply SBAT to 73 quantitative traits in the UK Biobank. [ABSTRACT FROM AUTHOR]

Automatic personality assessment (APA) has immense potential to improve decision-making and human-machine interaction. Numerous techniques for APA have been proposed in existing literature, with prior psychological studies demonstrating convergent validity between personality and trait emotional intelligence (EI). However, to the best of our knowledge, none in APA literature has leveraged this relationship. Further, the primary language for most APA studies is English. To this end, we propose VyaktitvaNirdharan, a multi-modal-multitask learning (MM-MTL) system for predicting personality and EI traits using dyadic conversations in a non-intrusive and near real-time fashion. While most of the previous research has focused on APA for the English language, VyaktitvaNirdharan is also amongst the first to use peer-to-peer conversations in Hindi, a low-resource language. We also show the convergent validity between the predictions of the developed system, confirming that the personality and EI traits estimated by our system preserve the properties of the original traits. Our system can identify the Big-Five and EI traits with average F1-scores of 0.88, and 0.82, respectively. The validity of our findings is further supported by a user study involving 7 stakeholders, including recruiters and clinical psychologists, conducted through a prototype. The proposed system demonstrates promising results and has the potential to aid in improving decision-making in recruitment, mental health wellness and enhance human-machine interaction. [ABSTRACT FROM AUTHOR]

Although video affective content analysis has great potential in many applications, it has not been thoroughly studied due to limited datasets. In this article, we construct a large-scale video affective dataset with danmu (VAD). It consists of 19,267 elaborately segmented video clips from user-generated videos. The VAD dataset is annotated by the crowdsourcing platform with discrete valence, arousal, and primary emotions, as well as the comparison of valence and arousal between two consecutive video clips. Unlike previous datasets, including only video clips, our proposed dataset also provides danmu, which is the real-time comment from users as they watch a video. Danmu provides extra information for video affective content analysis. As a preliminary assessment of the usability of our dataset, an analysis of inter-annotator consistency for each label is conducted using weighted Fleiss’ Kappa, regular Fleiss’ Kappa, intraclass correlation coefficient, and percent consensus. Besides, we also perform a statistical analysis of labels and danmu. Finally, video affective content analysis is conducted on our dataset and three typical methods (i.e., TFN, MulT, and MISA) are leveraged to provide benchmarks. We also demonstrate that danmu can significantly improve the performance of the video affective content analysis task on some labels. Our dataset is available for research purposes. [ABSTRACT FROM AUTHOR]

The current paradigm of joint detection and tracking still requires a large amount of instance-level trajectory annotation, which incurs high annotation costs. Moreover, treating embedding training as a classification problem would lead to difficulties in model fitting. In this paper, we propose a new self-supervised multi-object tracking based on the real-time joint detection and embedding (JDE) framework, which we termed as self-supervised multi-object tracking (SS-MOT). In SS-MOT, the short-term temporal correlations between objects within and across adjacent video frames are both considered as self-supervised constraints, where the distances between different objects are enlarged while the distances between same object of adjacent frames are brought closer. In addition, short trajectories are formed by matching pairs of adjacent frames using a matching algorithm, and these matched pairs are treated as positive samples. The distances between positive samples are then minimized for futher the feature representation of the same object. Therefore, our method can be trained on videos without instance-level annotations. We apply our approach to state-of-the-art JDE models, such as FairMOT, Cstrack, and SiamMOT, and achieve comparable results to these supevised methods on the widely used MOT17 and MOT20 challenges. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) plays a crucial role in various applications, including human–computer interaction and affective computing. However, the joint training of an FER network with multiple datasets is a promising strategy to enhance its performance. Nevertheless, widespread annotation inconsistencies and class imbalances among FER datasets pose significant challenges to this approach. This paper proposes a new multi-dataset joint training method, Sample Selection and Paired Augmentation Joint Training (SSPA-JT), to address these challenges. SSPA-JT models annotation inconsistency as a label noise problem and selects clean samples from auxiliary datasets to expand the overall dataset size while maintaining consistent annotation standards. Additionally, a dynamic matching algorithm is developed to pair clean samples of the tail class with noisy samples, which enriches the tail classes with diverse background information. Experimental results demonstrate that SSPA-JT achieved superior or comparable performance compared with the existing methods by addressing both annotation inconsistencies and class imbalance during multi-dataset joint training. It achieved state-of-the-art performance on RAF-DB and CAER-S datasets with accuracies of 92.44% and 98.22%, respectively, reflecting improvements of 0.2% and 3.65% over existing methods. [ABSTRACT FROM AUTHOR]

Named Entity Recognition (NER) is considered an important subtask in information extraction that aims to identify Named Entities (NM) within a given text and classify them into predefined categories (e.g., person, location, organization, and miscellaneous). The use of an appropriate annotation scheme is crucial to label multi-word NEs and enhance recognition performance. This study investigates the effects of using different annotation schemes on NER systems for the Arabic language. The impact of seven annotation schemes, namely IO, IOB, IOE, IOBE, IOBS, IOES, and IOBES, on Arabic NER is examined by applying conditional random fields, multinomial Naive Bayes, and support vector machine classifiers. The experimental results reveal the importance of selecting an optimal annotation scheme and show that annotating NEs based on the simple IO scheme yields a higher performance in terms of precision, recall, and F-measure compared to the other schemes. [ABSTRACT FROM AUTHOR]

Despite most state-of-the-art models having achieved amazing performance in Visual Question Answering (VQA), they usually utilize biases to answer the question. Recently, some studies synthesize counterfactual training samples to help the model to mitigate the biases. However, these synthetic samples need extra annotations and often contain noises. Moreover, these methods simply add synthetic samples to the training data to train the model with the cross-entropy loss, which cannot make the best use of synthetic samples to mitigate the biases. In this article, to mitigate the biases in VQA more effectively, we propose a Hierarchical Counterfactual Contrastive Learning (HCCL) method. Firstly, to avoid introducing noises and extra annotations, our method automatically masks the unimportant features in original pairs to obtain positive samples and create mismatched question-image pairs as negative samples. Then our method uses feature-level and answer-level contrastive learning to make the original sample close to positive samples in the feature space, while away from negative samples in both feature and answer spaces. In this way, the VQA model can learn the robust multimodal features and focus on both visual and language information to produce the answer. Our HCCL method can be adopted in different baselines, and the experimental results on VQA v2, VQA-CP, and GQA-OOD datasets show that our method is effective in mitigating the biases in VQA, which improves the robustness of the VQA model. [ABSTRACT FROM AUTHOR]

Text-to-Video Retrieval is a typical cross-modal retrieval task that has been studied extensively under a conventional supervised setting. Recently, some works have sought to extend the problem to a weakly supervised formulation, which can be more consistent with real-life scenarios and more efficient in annotation cost. In this context, a new task called Partially Relevant Video Retrieval (PRVR) is proposed, which aims to retrieve videos that are partially relevant to a given textual query, i.e., the videos containing at least one semantically relevant moment. Formulating the task as a Multiple Instance Learning (MIL) ranking problem, prior arts rely on heuristics algorithms such as a simple greedy search strategy and deal with each query independently. Although these early explorations have achieved decent performance, they may not fully utilize the bag-level label and only consider the local optimum, which could result in suboptimal solutions and inferior final retrieval performance. To address this problem, in this paper, we propose to exploit the relationships between instances to boost retrieval performance. Based on this idea, we creatively put forward: (1) a new matching scheme for pairing queries and their related moments in the video; and (2) a new loss function to facilitate cross-modal alignment between two views of an instance. Extensive validations on three publicly available datasets have demonstrated the effectiveness of our solution and verified our hypothesis that modeling instance-level relationships is beneficial in the MIL ranking setting. Our code will be publicly available at https://github.com/xjtupanda/BGM-Net. [ABSTRACT FROM AUTHOR]