Search

Multimodal named entity recognition (MNER) is an emerging foundational task in natural language processing. However, existing methods have two main limitations: 1) previous methods have focused on the visual representation of the entire image or target objects. However, they overlook the fine-grained semantic correspondence between entities and visual target objects, or ignore the visual cues of the overall scene and background details in the image. 2) Existing methods have not effectively overcome the semantic gap between different modalities due to the heterogeneity between text and images. To address these issues, we propose a novel multimodal heterogeneous graph entity-level fusion method for MNER (HGMVG) to achieve cross-modal feature interaction from coarse to fine between text and images under the guidance of visual information at different granularities, which can improve the accuracy of named entity recognition. Specifically, to resolve the first issue, we cascade cross-modal semantic interaction information between text and vision at different visual granularities to obtain a comprehensive and effective multimodal representation. For the second issue, we describe the precise semantic correspondences between entity-level words and visual target objects via multimodal heterogeneous graphs, and utilize heterogeneous graph attention networks to achieve cross-modal fine-grained semantic interactions. We conduct extensive experiments on two publicly available Twitter datasets, and the experimental results demonstrate that HGMVG outperforms the current state-of-the-art models in the MNER task. [ABSTRACT FROM AUTHOR]

The Multimodal Named Entity Recognition (MNER) task enhances the text representations and improves the accuracy and robustness of named entity recognition by leveraging visual information from images. However, previous methods have two limitations: (i) the semantic mismatch between text and image modalities makes it challenging to establish accurate internal connections between words and visual representations. Besides, the limited number of characters in social media posts leads to semantic and contextual ambiguity, further exacerbating the semantic mismatch between modalities. (ii) Existing methods employ cross-modal attention mechanisms to facilitate interaction and fusion between different modalities, overlooking fine-grained correspondences between semantic units of text and images. To alleviate these issues, we propose a graph neural network approach for MNER (GNN-MNER), which promotes fine-grained alignment and interaction between semantic units of different modalities. Specifically, to mitigate the issue of semantic mismatch between modalities, we construct corresponding graph structures for text and images, and leverage graph convolutional networks to augment text and visual representations. For the second issue, we propose a multimodal interaction graph to explicitly represent the fine-grained semantic correspondences between text and visual objects. Based on this graph, we implement deep-level feature fusion between modalities utilizing graph attention networks. Compared with existing methods, our approach is the first to extend graph deep learning throughout the MNER task. Extensive experiments on the Twitter multimodal datasets validate the effectiveness of our GNN-MNER. [ABSTRACT FROM AUTHOR]

Continuous data scale growth increases energy consumption and operating cost that cannot be ignored in cloud storage systems. Previous studies have shown that analyzing the characteristics of I/O access and mining data features is effective for reasonable data distribution in storage systems. The granularity and criterion of classification are the key factors in determining the data distribution. To decrease energy consumption and operating cost, this paper puts forward a fine-grained framework of the climatic-season-based energy-aware in cloud storage system called CSEA. The framework concludes the following three aspects: (i) data feature mining. CSEA discovers potential data features by analyzing data access to provide help with data classification. (ii) K-means clustering algorithm. CSEA uses an unsupervised data classification algorithm in machine learning to divide data into categories based on seasonal characteristics by gathering real I/O access. (iii) data distribution of fine-grained. On the basis of seasonal features, CSEA fuses regional features to further refine the data distribution granularity to save on energy consumption and operating cost. Simulation experiments using extended CloudSimDisk and the constructed mathematical models indicate that CSEA reduces the energy consumption and operating cost compared with the single data classification standard and coarse-grained data distribution. [ABSTRACT FROM AUTHOR]

The 2D human pose estimation plays an important role in human‐computer interaction and action recognition. Although the method based on high‐resolution network has superior performance, there is still room for improvement in terms of speed and lightweight. Here, a LiteDEKR, a 2D pose estimation method that combines lightweight and accuracy, is proposed by designing a lightweight network based on DEKR and constructing two scientifically valid loss functions. The method, constructs a multi‐instance bias regression loss that matches the true distribution of keypoint bias, improves the accuracy of bias regression, and constructs a keypoint similarity loss with the object keypoint similarity index of keypoints as the optimization objective to achieve end‐to‐end training of the network. In addition, this paper has designed a lightweight DEKR, using LitePose as the backbone network. With the optimization of the above two loss functions, LiteDEKR not only achieves lightweight but also has high accuracy. Comparative experiments on the COCO and CrowdPose datasets show that compared to the current state‐of‐the‐art Contextual Instance Decoupling, LiteDEKR achieves a similar accuracy with only 10% of its network complexity. It also shows better robustness to low‐resolution input images. [ABSTRACT FROM AUTHOR]

Visual information implied by the images in multimodal relation extraction (MRE) usually contains details that are difficult to describe in text sentences. Integrating textual and visual information is the mainstream method to enhance the understanding and extraction of relations between entities. However, existing MRE methods neglect the semantic gap caused by data heterogeneity. Besides, some approaches map the relations between target objects in image scene graphs to text, but massive invalid visual relations introduce noise. To alleviate the above problems, we propose a novel multimodal relation extraction method based on cooperative enhancement of dual-channel visual semantic information (CE-DCVSI). Specifically, to mitigate the semantic gap between modalities, we realize fine-grained semantic alignment between entities and target objects through multimodal heterogeneous graphs, aligning feature representations of different modalities into the same semantic space using the heterogeneous graph Transformer, thus promoting the consistency and accuracy of feature representations. To eliminate the effect of useless visual relations, we perform multi-scale feature fusion between different levels of visual information and textual representations to increase the complementarity between features, improving the comprehensiveness and robustness of the multimodal representation. Finally, we utilize the information bottleneck principle to filter out invalid information from the multimodal representation to mitigate the negative impact of irrelevant noise. The experiments demonstrate that the method achieves 86.08% of the F1 score on the publicly available MRE dataset, which outperforms other baseline methods. • Capturing fine-grained and multi-scale multimodal representations. • Multimodal heterogeneous graphs illuminate fine-grained alignment relations between modalities. • The heterogeneous graph Transformer adjusts the interaction strength of cross-modal features. • eliminating redundant and useless information through information bottlenecks. • Experimental results demonstrate the effectiveness and robustness of our method. [ABSTRACT FROM AUTHOR]

Entity alignment (EA) aims to find entities that point to the same object in multiple knowledge graphs (KGs), i.e., finding equivalent entity pairs. In recent years, with the emergence of temporal knowledge graphs (TKGs), the TKG alignment has gained wide attention. Many existing methods embed temporal information into a vector space of low dimension and perform entity alignment by incorporating temporal embedding. However, most existing entity alignment methods ignore the relevance of relations in TKGs, which will make the entity alignment less accurate. To solve this problem, we propose a novel temporal knowledge graph alignment approach based on relation association and probabilistic calibration, namely, TKGA-RP. Specifically, to deal with the relevance between relations, we first construct the forward and reverse inverse relations with the beginning and the end of time. And then, we propose a novel way to capture the relational association (RA) of forward and inverse relations by considering the interacted entities. In addition, we constructed relational orthogonal matrices and temporal orthogonal matrices to capture the temporal awareness and relational awareness of the surrounding entities. The final entity representations are generated by graph neural networks. Finally, we calibrate the embedding distance between entities by probabilistic calibration (PC) to improve the performance. Experimental results on two TKG datasets show that our model achieves better performance than existing mainstream methods. [ABSTRACT FROM AUTHOR]

In recommendation systems, users often click on some items that are distinct from historically clicked items. This verifies the existence of interest gaps between the historical interests reflected by historical behaviors and current interests. The gaps pose key challenges for interest modeling in recommendation systems, i.e., learn user current interests from historical behaviors. To bridge interest gaps, various solutions have been proposed from a model perspective, and mainly focus on capturing interest diversity and interest evolution process (the changes in user interests over time). Different from existing solutions, our key idea is smoothing the historical interests from a data perspective, so that interest gaps can be smoothed and bridged. To realize this idea, we propose two types of data-driven smoothing approaches based on the principle of collaborative filtering, i.e., click-centric smoothing and user-centric smoothing. The approaches can be easily applied to a number of existing solutions and enhance their performances. To ensure that the smoothed interests completely and accurately cover user current interests (coverage and accuracy), the click-centric smoothing leverages the relevant clicks of a click to smooth the interests on the click, and the user-centric smoothing leverages the relevant clicks of a user click sequence to smooth the interests learned from the click sequence. The "smooth" is to enhance the coverage, and "relevant" is to enhance the accuracy. To search the relevant clicks, we develop an iterative Bayesian inference over a click-flow graph, i.e., a new implementation of collaborative filtering. To further enhance the accuracy, we propose new constrained attention networks to generate high-quality smoothing factors from these relevant clicks. Both offline and online experiments show the data-driven smoothing approaches significantly outperform the baselines. The user-centric smoothing has been deployed on the e-commerce platform of Meituan and achieved 1.51% elevation of Click-through & Conversion Rate and 0.53% elevation of Click-through Rate. • We are the first to apply the smoothing mechanism to bridge interest gaps in the interest modeling task. • We propose two types of data-driven smoothing approaches based on the assumption of collaborative filtering. • A novel iterative Bayesian inference framework on a click-flow graph is designed to search relevant clicks. • New constrained attention networks are proposed to generate high-quality smoothing factors. • Both offline and online experiments show that our models significantly outperform the SOTA baselines. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence, the application of intelligent algorithms to low-power embedded chips has become a new research topic today. Based on this, this study optimizes the YOLOv2 algorithm by tailoring and successfully deploys it on the K210 chip to train the face object detection algorithm model separately. The intelligent fan with YOLOv2 model deployed in K210 chip can detect the target of the character and obtain the position and size of the character in the machine coordinates. Based on the obtained information of character coordinate position and size, the fan's turning Angle and the size of air supply are intelligently perceived. The experimental results show that the intelligent fan design method proposed here is a new embedded chip intelligent method of cutting and improving the YOLOv2 algorithm. It innovatively designed solo tracking, crowd tracking, and intelligent ranging algorithms, which perform well in human perception of solo tracking and crowd tracking and automatic air volume adjustment, improve the accuracy of air delivery and user comfort, and also provide good theoretical and practical support for the combination of AI and embedded in other fields. [ABSTRACT FROM AUTHOR]

The overload of data always threatens the reliability of storage system. The RAID-6 storage system provides higher reliability and flexible scalability. RAID-6 scaling can rapidly relieve the insufficient storage capacity in a short time. Therefore, this paper proposes Horizontal Data migration Scaling (HDS), an efficient RAID-6 scaling scheme, for HDP Code. First, it only migrates a small amount of data from the old disk to the new disk to regain I/O load balancing among all disks including old and new. Second, it optimizes the update order of anti-diagonal parity data to reduce the cost of parity data update. By numerical results and real experimental data analysis, this paper compares the performance of HDS to Round-Robin and Semi-RR. Compared with Round-Robin and Semi-RR, the final analysis results indicate: (1) HDS reduces the data migration by 59.9 ∼ 83.3%; (2) HDS decreases the total cost of XOR operations by 36.84 ∼ 71.43% and 66.04 ∼ 76.92%; (3) HDS improves the total scaling time by 43.78 ∼ 61.83% and 16.39 ∼ 48.89% under offline. [ABSTRACT FROM AUTHOR]

Caused by massive data, large-scale data centers suffer from insufficient storage capacity. This increases the risk of component damage and disk failure. To enhance storage capacity of data centers, scaling has turned out to be one of the most popular choices. RAID-6 is deployed extensively that has superior reliability, availability, and scalability compared to other RAID levels. However, higher reliability also means expensive parity data update cost. To address this issue, this paper proposes an efficient RAID-6 scaling scheme, HS6, for H-Code. The basic idea of HS6 is reducing the total parity data update cost and saving the total scaling time by optimizing data migration and parity data update. The properties of HS6 can be summarized as follows: (1) it utilizes horizontal data migration to eliminate the horizontal parity data update, and (2) it makes full use of original parity data to cut down the total parity data update cost. Numerical results and experimental data analysis indicate that: (1) HS6 decreases the data migration rate by 76.92% ∼ 95.31%, 55.56% ∼ 84.46%, and 14.29% ∼ 16.67% compared to Round-Robin, Semi-RR, and HCS, (2) HS6 saves the total scaling time by 62.05% ∼ 76.75%, 53.24% ∼ 73.94%, and 3.68% ∼ 6.79% against Round-Robin, Semi-RR, and HCS, respectively, under offline and outperforms Round-Robin by 59.99% ∼ 79.02%, Semi-RR by 53.04% ∼ 76.41%, and HCS by 0.56% ∼ 23.78% in total scaling time under four workloads, and (3) HS6 maintains almost the same user average response time with Round-Robin, Semi-RR, and HCS during scaling and after scaling. [ABSTRACT FROM AUTHOR]

Thanks to excellent reliability, availability, flexibility and scalability, r edundant a rrays of i ndependent (or i nexpensive) d isks (RAID) are widely deployed in large-scale data centers. RAID scaling effectively relieves the storage pressure of the data center and increases both the capacity and I/O parallelism of storage systems. To regain load balancing among all disks including old and new, some data usually are migrated from old disks to new disks. Owing to unique parity layouts of erasure codes, traditional scaling approaches may incur high migration overhead on RAID-6 scaling. This paper proposes an efficient approach based Short-Code for RAID-6 scaling. The approach exhibits three salient features: first, SS6 introduces |$\tau $| to determine where new disks should be inserted. Second, SS6 minimizes migration overhead by delineating migration areas. Third, SS6 reduces the XOR calculation cost by optimizing parity update. The numerical results and experiment results demonstrate that (i) SS6 reduces the amount of data migration and improves the scaling performance compared with Round-Robin and Semi-RR under offline, (ii) SS6 decreases the total scaling time against Round-Robin and Semi-RR under two real-world I/O workloads (iii) the user average response time of SS6 is better than the other two approaches during scaling and after scaling. [ABSTRACT FROM AUTHOR]

Nowadays, dust exposure pollution is receiving a lot of attention due to its significant impact on public health. To investigate the impact of dust particle size and human respiratory strength on respiratory particle deposition patterns, data was collected through on‐site surveys. The study analyzed the equivalent respiratory strength, dust environment characteristics, and bronchial particle escape and deposition patterns of humans in fully mechanized mining faces at various operating times. This was done using ergonomic energy consumption simulation experiments and a fluid–solid interaction method of CFD‐DEM. The findings revealed that as humans worked continuously for 5, 15, 30, 45, and 60 min, their respiratory intensity corresponded to 8, 18, 30, 42, and 50 L/min, respectively. According to the field investigation and particle size analysis, the particle size distribution of 1~5, 5~10, 10~20, 20~30, and 30~40 μm particles accounted for 36%, 26%, 15%, 11%, and 10%, respectively. In general, the deposition rate of dust was highest in the main bronchus of the respiratory tract, followed by the trachea area. Particles ranging from 5 to 10 μm in size were observed to have a higher likelihood of escaping from the tertiary bronchioles and entering the secondary bronchial regions. Conversely, particles larger than 20 μm exhibited a deposition rate of up to 80% in the tertiary bronchial regions. It was noted that the bronchial deposition rate of particles of varying sizes increased with respiratory strength, with smaller particles showing greater sensitivity to changes in respiratory strength in terms of the deposition fraction. Among the different particle sizes, the deposition rate of 5–10 μm particles exhibited the most variation with increasing respiration intensity, ranging up to 17%. [ABSTRACT FROM AUTHOR]

Recently, head detection has been widely used in target detection, which has a great application value for improving security prevention and control in public places, as well as enhancing target tracking and identification in national defense, criminal investigation, and other fields. However, detecting small targets accurately at long distances is very difficult, and current methods often lack optimization of multi‐resolution features. Therefore, the authors propose a one‐stage detection network CFNet (cross‐layer feature fusion and fusion weight attention network), in which a fusion weight attention mechanism module (FWAM) is proposed to give different weights to the fused features in order to distinguish the importance of different features. The module increases the weights of features that contain strong information so that the fused features are focused on feature points that are beneficial for optimal head detection. Meanwhile, a cross‐layer feature fusion module is proposed to fuse information from different resolution feature maps to compensate for the decrease in detection accuracy caused by the omission of information features at low resolution, and a connection network for contextual information fusion is constructed, while weight parameter value settings are introduced to optimize the detection effect after fusion of different resolution features. In order to better reflect the effectiveness of the network, the experiments are performed on the SCUT‐HEAD PartA dataset and the Brainwash dataset; the results show that the network the authors proposed is better than the existing comparison methods, which proves the robustness and effectiveness of the network. [ABSTRACT FROM AUTHOR]

Nowadays, e-commerce platforms of hotels have become a new trend to help people book hotels online. Interest modeling aims to automatically construct user interests that are critical for e-commerce platforms. Although interest modeling has achieved noticeable successes in product recommendations, it has not been explored in hotel search, especially in the reranking stage. This paper studies the interest modeling task for reranking hotels, and identifies two key challenges—high behavior sparsity and large interest gap. To address the challenges, we propose the deep click interest network (DCIN). High behavior sparsity keeps us from extracting rich semantics to characterize user interest preference. Accordingly, we propose the deep click concept to model the multi-view semantics of a click and then enrich the semantics of both a single click and a click sequence, so that the difficulty caused by the behavior sparsity can be addressed. To address the interest gap challenge, DCIN models user interests with two cascaded units: (i) Mutual-attention Interest Calibration Unit (ICU), which uses the candidate hotel to calibrate the embedding of every click, since the candidate hotel is selected by the ranking stage and can represent user current interests to some extent. So the gap between the calibrated embedding and user current interests can be diminished; (ii) Multi-attention Interest Aggregation Unit (IAU), which estimates the weight of a calibrated embedding from multiple perspectives, i.e., context, user feedback and interest consistency. So the calibrated embeddings, which are relevant to the current query context and important to the user, dominate user current interests, and thus the interest gap can be further diminished. Offline experiments over two large datasets and online A/B testing over the platform of Meituan-Hotel show that DCIN significantly outperforms the baselines and improved baselines. Notably, DCIN has been deployed in Meituan-Hotel, resulting in an increase of 2.40% in Click Through Conversion Rate (CTCVR) and that of 1.09% in Click Through Rate (CTR). • We study interest modeling for reranking hotels and identify two key challenges. • Deep click concept is formally defined to model the multi-view semantics of a click. • Mutual attention calibration unit is designed to calibrate the embedding of a click. • Multi-attention aggregation unit is developed to estimate the weights of clicks from multiple perspectives. • Both offline and online experiments show our model outperforms the SOTA baselines. [ABSTRACT FROM AUTHOR]

Question model has been widely concerned as the cornerstone of constructing Visual Question Answering (VQA) models. Existing question models attempt to exploit word context to extract multi-level concepts for modeling multi-level questions. However, they still have many defects. For example, most question models utilize simple fusion methods to fuse shallow modules and extract parameter-unshared low-level concepts, leading to poor modeling of multi-level questions; although some question models use deep bidirectional Transformer encoder in external knowledge transfer and BERT for multi-level questions, their complexity is still high. To solve these issues, we propose a novel low-complex multi-level contextual question model, termed Context-aware Multi-level Question Embedding Fusion (CMQEF). We formalize its concepts and theories, deduce its modeling process, optimization process and feature extraction process, analyze its low complexity and high expressiveness, and prove that it defines a new way to solve parameter non-sharing for extracting parameter-shared multi-level concepts and optimize the tradeoff between expressiveness and complexity in question models. Extensive experiments on VQAv2 and VQA-CPv2 validate that comparing with the state-of-the-art, our CMQEF outperforms it on SANs and UpDn, reduces the language priors of SANs and UpDn, and has preferable interpretability and applicability. Our code is available at https://github.com/lsdruc/CMQEF-for-VQA. • Bi-NGRU replaces higher layers to extract high-level concepts and reduce complexity. • Intra- and inter-question fusion method replaces BERT to solve parameter non-sharing. • Our CMQEF outperforms the state-of-the-art question models on VQAv2. • CMQEF extracts more language context knowledge to reduce the language priors of VQA. • Our CMQEF is more interpretable. [ABSTRACT FROM AUTHOR]

Trackless, rubber-tired, diesel-powered vehicles are widely used in the mining industry. In recent years, the use of high-performance catalysts to remove carbon monoxide (CO) from diesel vehicle exhaust has attracted much attention. In this paper, a series of manganese-vanadium doped ZSM-5 molecular sieve catalysts were prepared by an impregnation method. The structures of the catalysts were characterized and experiments were conducted to evaluate their performance. The CO catalytic oxidation performance experiments revealed that Mn 0.6 -V 0.4 -ZSM-5 had the best performance, with a conversion rate of 97.5 % at 300 °C. The catalyst was also tested for water resistance and stability. Kinetic measurements were performed and the fitted activation energy was 22.68 kJ/mol. Characterization studies revealed that the surface defects formed by Mn and V doping provided a large number of active sites and oxygen vacancies for CO oxidation, increased the number of adsorbed oxygen species on the surface, and enhanced the redox capacity through the interaction of Mn4+ and V5+ to improve the catalytic oxidation performance of the catalyst. The adsorption configurations, adsorption energy (E bin), and formation energy of oxygen vacancies of CO at different adsorption sites and different active fractions were calculated with density functional theory. The E bin of CO on Mn 2 V 2 O 7 , the active fraction of Mn 0.6 -V 0.4 -ZSM-5, was −1.29 eV, which was the most stable adsorption configuration. This configuration had good oxygen mobility, and was conducive to CO catalysis, confirming the characterization and performance experiments. [ABSTRACT FROM AUTHOR]