Search

Pre-trained language model (PTM) has been shown to yield powerful text representations for dense passage retrieval task. The Masked Language Modeling (MLM) is a major sub-task of the pre-training process. However, we found that the conventional random masking strategy tend to select a large number of tokens that have limited effect on the passage retrieval task (e,g. stop-words and punctuation). By noticing the term importance weight can provide valuable information for passage retrieval, we hereby propose alternative retrieval oriented masking (dubbed as ROM) strategy where more important tokens will have a higher probability of being masked out, to capture this straightforward yet essential information to facilitate the language model pre-training process. Notably, the proposed new token masking method will not change the architecture and learning objective of original PTM. Our experiments verify that the proposed ROM enables term importance information to help language model pre-training thus achieving better performance on multiple passage retrieval benchmarks., Search LM part of the "AliceMind SLM + HLAR" method in MS MARCO Passage Ranking Leaderboard Submission

Passage retrieval is a fundamental task in information retrieval (IR) research, which has drawn much attention recently. In the English field, the availability of large-scale annotated dataset (e.g, MS MARCO) and the emergence of deep pre-trained language models (e.g, BERT) has resulted in a substantial improvement of existing passage retrieval systems. However, in the Chinese field, especially for specific domains, passage retrieval systems are still immature due to quality-annotated dataset being limited by scale. Therefore, in this paper, we present a novel multi-domain Chinese dataset for passage retrieval (Multi-CPR). The dataset is collected from three different domains, including E-commerce, Entertainment video and Medical. Each dataset contains millions of passages and a certain amount of human annotated query-passage related pairs. We implement various representative passage retrieval methods as baselines. We find that the performance of retrieval models trained on dataset from general domain will inevitably decrease on specific domain. Nevertheless, a passage retrieval system built on in-domain annotated dataset can achieve significant improvement, which indeed demonstrates the necessity of domain labeled data for further optimization. We hope the release of the Multi-CPR dataset could benchmark Chinese passage retrieval task in specific domain and also make advances for future studies., Comment: SIGIR 2022 Resource Track

Named Entity Recognition (NER) from speech is among Spoken Language Understanding (SLU) tasks, aiming to extract semantic information from the speech signal. NER from speech is usually made through a two-step pipeline that consists of (1) processing the audio using an Automatic Speech Recognition (ASR) system and (2) applying an NER tagger to the ASR outputs. Recent works have shown the capability of the End-to-End (E2E) approach for NER from English and French speech, which is essentially entity-aware ASR. However, due to the many homophones and polyphones that exist in Chinese, NER from Chinese speech is effectively a more challenging task. In this paper, we introduce a new dataset AISEHLL-NER for NER from Chinese speech. Extensive experiments are conducted to explore the performance of several state-of-the-art methods. The results demonstrate that the performance could be improved by combining entity-aware ASR and pretrained NER tagger, which can be easily applied to the modern SLU pipeline. The dataset is publicly available at github.com/Alibaba-NLP/AISHELL-NER.

Self-augmentation has received increasing research interest recently to improve named entity recognition (NER) performance in low-resource scenarios. Token substitution and mixup are two feasible heterogeneous self-augmentation techniques for NER that can achieve effective performance with certain specialized efforts. Noticeably, self-augmentation may introduce potentially noisy augmented data. Prior research has mainly resorted to heuristic rule-based constraints to reduce the noise for specific self-augmentation methods individually. In this paper, we revisit these two typical self-augmentation methods for NER, and propose a unified meta-reweighting strategy for them to achieve a natural integration. Our method is easily extensible, imposing little effort on a specific self-augmentation method. Experiments on different Chinese and English NER benchmarks show that our token substitution and mixup method, as well as their integration, can achieve effective performance improvement. Based on the meta-reweighting mechanism, we can enhance the advantages of the self-augmentation techniques without much extra effort., Comment: Accepted by NAACL 2022

Deep pre-trained language models (e,g. BERT) are effective at large-scale text retrieval task. Existing text retrieval systems with state-of-the-art performance usually adopt a retrieve-then-reranking architecture due to the high computational cost of pre-trained language models and the large corpus size. Under such a multi-stage architecture, previous studies mainly focused on optimizing single stage of the framework thus improving the overall retrieval performance. However, how to directly couple multi-stage features for optimization has not been well studied. In this paper, we design Hybrid List Aware Transformer Reranking (HLATR) as a subsequent reranking module to incorporate both retrieval and reranking stage features. HLATR is lightweight and can be easily parallelized with existing text retrieval systems so that the reranking process can be performed in a single yet efficient processing. Empirical experiments on two large-scale text retrieval datasets show that HLATR can efficiently improve the ranking performance of existing multi-stage text retrieval methods., Comment: Work in progress. HLAR part of the "AliceMind SLM + HLAR" method in MS MARCO Passage Ranking Submission

Medullary thyroid carcinoma (MTC) is relatively rare, and has the main feature of calcitonin (Ct) secretion. However, a few cases of MTC with negative serum calcitonin have been reported in the literature, so the diagnosis and follow up of Ct-negative MTCs are still a challenge. Here we present three cases of Ct-negative MTCs, illustrating the rarity of the disease and challenges in managing it, together with a review of the literature of 39 MTCs with negative serum Ct.

As an effective approach to tune pre-trained language models (PLMs) for specific tasks, prompt-learning has recently attracted much attention from researchers. By using \textit{cloze}-style language prompts to stimulate the versatile knowledge of PLMs, prompt-learning can achieve promising results on a series of NLP tasks, such as natural language inference, sentiment classification, and knowledge probing. In this work, we investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot and zero-shot scenarios. We first develop a simple and effective prompt-learning pipeline by constructing entity-oriented verbalizers and templates and conducting masked language modeling. Further, to tackle the zero-shot regime, we propose a self-supervised strategy that carries out distribution-level optimization in prompt-learning to automatically summarize the information of entity types. Extensive experiments on three fine-grained entity typing benchmarks (with up to 86 classes) under fully supervised, few-shot and zero-shot settings show that prompt-learning methods significantly outperform fine-tuning baselines, especially when the training data is insufficient.

The applications of carvacrol are limited due to its poor stability and water solubility, and high volatility; however, ovalbumin can be used to encapsulate hydrophobic molecules, improve their aqueous solubility, and reduce their volatility. In this study, we fabricated ovalbumin-carvacrol nanoparticles (OCGns) under different pH (2, 5, 7, and 9) conditions using a gel embedding method and investigated their physicochemical and antibacterial properties. Rheological experiments revealed that the G' of ovalbumin gels (OGs) prepared under different pH conditions were OG-2 > OG-7 > OG-9 > OG-5. Carvacrol addition reduced the tight structure of ovalbumin and carvacrol under pH 5 and 7 conditions, with hardness first decreasing and then increasing, but increasing under pH 2 and 9 conditions. Fluorescence and infrared spectroscopy indicated complex formation, with carvacrol increasing the average diameter of nanoparticles prepared at pH 2, 5, 7, and 9. Encapsulation reached 89.34 and 91.86% at pH 2 and 9, respectively; however, inhibition experiments revealed that the minimum inhibitory concentration of OCGn-2 against Gram-positive Bacillus cereus and Salmonella (0.08 and 0.16 mg mL-1, respectively) was lower than that of OCGn-9 (both 0.28 mg mL-1). Moreover, OCGn-2 possessed a better dense gel structure and a higher stability, encapsulation rate, and antibacterial activity, suggesting that pH affects gel microstructure and thus the encapsulation efficiency and bacteriostatic properties of the prepared nanoparticles. These results contribute to our knowledge of the design and fabrication of polymeric nanoparticle delivery systems for bioactive compounds with beneficial properties.

Based on the field measurement of the end resistance of the support during the initial weighting of the basic roof and the macroscopic mine pressure behavior during the weighting period of 101,22211,103 and 301 fully mechanized caving face in Changchun Xing Coal Mine, the mine pressure law of the working face is summarized and compared, and the relationship between the working face length and the working resistance of the support ( the weighting strength ) and the macroscopic mine pressure behavior is obtained. In the range of face length 126-230 m, with the increase of face length, the end-of-cycle resistance of the support gradually increases and the dynamic load coefficient of the support gradually increases when the coefficient of the support gradually increases, and the strata behavior of the working face changes from strong to very strong. When the face length is short ( 126-140.5m ), the hanging top area is too large to cause hurricanes when the working face is pressed, which threatens and damages the personal safety and equipment of the working face staff. Based on the above research, the problem of optimizing the surface length is proposed, and the surface length is determined to be within the range of 140-230 m according to the measured results.

Recently, a variety of probing tasks are proposed to discover linguistic properties learned in contextualized word embeddings. Many of these works implicitly assume these embeddings lay in certain metric spaces, typically the Euclidean space. This work considers a family of geometrically special spaces, the hyperbolic spaces, that exhibit better inductive biases for hierarchical structures and may better reveal linguistic hierarchies encoded in contextualized representations. We introduce a Poincare probe, a structural probe projecting these embeddings into a Poincare subspace with explicitly defined hierarchies. We focus on two probing objectives: (a) dependency trees where the hierarchy is defined as head-dependent structures; (b) lexical sentiments where the hierarchy is defined as the polarity of words (positivity and negativity). We argue that a key desideratum of a probe is its sensitivity to the existence of linguistic structures. We apply our probes on BERT, a typical contextualized embedding model. In a syntactic subspace, our probe better recovers tree structures than Euclidean probes, revealing the possibility that the geometry of BERT syntax may not necessarily be Euclidean. In a sentiment subspace, we reveal two possible meta-embeddings for positive and negative sentiments and show how lexically-controlled contextualization would change the geometric localization of embeddings. We demonstrate the findings with our Poincare probe via extensive experiments and visualization. Our results can be reproduced at https://github.com/FranxYao/PoincareProbe., ICLR 2021 Camera ready

Recognizing relations between entities is a pivotal task of relational learning. Learning relation representations from distantly-labeled datasets is difficult because of the abundant label noise and complicated expressions in human language. This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data that are effective in different settings, including supervised, distantly supervised, and few-shot learning. Instead of solely relying on the supervision from noisy labels, we propose to learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Prototypes are representations in the feature space abstracting the essential semantics of relations between entities in sentences. We learn prototypes based on objectives with clear geometric interpretation, where the prototypes are unit vectors uniformly dispersed in a unit ball, and statement embeddings are centered at the end of their corresponding prototype vectors on the surface of the ball. This approach allows us to learn meaningful, interpretable prototypes for the final classification. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art models. We further demonstrate the robustness of the encoder and the interpretability of prototypes with extensive experiments., Comment: Accepted by ICLR 2021

High grain-Cd-accumulating rice variety Yongyou 9 was planted in Cd-contaminated farmland in Taizhou City, Zhejiang Province, China to study the effects of 5-aminolevulinic acid (ALA) and 24-epibrassinolide (EBR) on Cd accumulation in brown rice. Results showed that the exogenous ALA and EBR had no significant effects on agronomic traits, soil pH and total Cd content in soil, but had some effects on the available Cd content in soil, and significantly influenced the Cd accumulation in the different parts of rice. Results also showed that 100 mg/L exogenous ALA significantly reduced the Cd accumulation in brown rice to blow the food safety standard (0.2 mg/kg), and also significantly reduced the Cd contents in the roots and culm of rice. However, 200 mg/L exogenous ALA treatment increased the Cd content in brown rice remarkably. In addition, 0.15 mg/L EBR treatment increased Cd accumulation in roots, culm, leaves and brown rice notably, whereas 0.30 mg/L exogenous EBR treatment reduced the Cd accumulation in brown rice properly, but it was not significant. Therefore, proper concentration of ALA can effectively reduce the Cd accumulation in brown rice, which can be used as an effective technical method for the safe production of rice in Cd polluted farmland. Keywords: rice, Cd-contaminated soil, 5-aminolevulinic acid, 24-epibrassinolide, Cd accumulation

Carvacrol is an effective natural antimicrobial and antioxidant agent; however, its poor aqueous solubility and high volatility limit its application in food systems. Ovalbumin can encapsulate hydrophobic molecules, thereby improving aqueous solubility and reducing volatility. The aim of this study was to explore the potential of an ovalbumin nanocarrier to improve the application range of carvacrol. Protein structure and scanning electron microscopy showed that carvacrol increased gel hardness and tackiness at pH 5, and OVA gel formed a uniform granular shape that was beneficial for reconstitution. The dominant force in OVA gel changed from electrostatic to hydrophobic interaction, indicating that carvacrol could hydrophobically bind to the gel. The ovalbumin-carvacrol (OVA-Car) complex was prepared using an oil-in-water method. The particle size was 132 ± 10 nm, and the encapsulation efficiency was 51.41%. The polydispersity index was 0.355, indicating the general stability of the composite. Fluorescence spectroscopy and differential thermogram studies indicated complex formation. Antibacterial property testing against Bacillus cereus and Salmonella indicated that the OVA-Car complex inhibited bacterial growth at lower concentrations than free carvacrol. Minimum inhibitory concentration and minimum bactericidal concentration for Bacillus cereus (0.0968 and 0.3875 mg/mL) and Salmonella (0.1937 and 0.3875 mg/mL), respectively, were 2–4 times higher than those of free carvacrol. The time sterilization curve indicated that pathogenic bacterial growth did not reach the control level. The OVA-Car complex has potential application in food systems owing to its storage stability and improved antimicrobial activity.

Data owners’ outsourced data on cloud data storage servers by the deduplication technique can reduce not only their own storage cost but also cloud’s. This paradigm also introduces new security issues such as the potential threat of data lost or corrupted. Data integrity verification is utilized to safeguard these data integrity. However, the cloud deduplication storage only focuses on file/chunk level to store one copy of the same data hosted by different data owners, and is not concerned with the same part of different data, e.g., a series of version files. We propose an integrity verification algorithm of different version files. The algorithm establishes the generic storage model of different version control methods to improve the universality of data verification. Then, the methods of verification tags and proofs generating are improved based on the index pointers corresponding to the storage relationship in the version groups and chained keys. Finally, the random diffusion extraction based on the random data sampling in the version group is proposed to improve the verification efficiency. The results of theoretical and experimental analysis indicate that the algorithm can achieve fast and large-scale verification for different version data.

This paper mainly studies data-based parameter calibration of the computed tomography (CT) system. By establishing a least squares algorithm model, CT system parameters were calibrated. First, the attenuation coefficient is calculated from the known data, and the relationship between the received information and the pitch of the X-ray transmission template is obtained by using the Lambert-beers principle. Then, by using the geometric relationship between the ray and the two-dimensional graph, the relationship model between the pitch and the variable to be sought is established. On this basis, we can get the multivariate linear equations which present the received information and the variables to be solved. Finally, the least square method is used to establish the target programming model which is based on the minimum square sum of the fitting value and the actual value of the received information. By using the genetic algorithm, the multiple linear equations are solved. Meanwhile, the deflection center and receiver unit spacing can be obtained.

A series of side-chain liquid crystalline polysiloxanes containing either cholesteryl or cyanobiphenyl mesogenic groups were synthesized by the hydrosilylation reaction of a polysiloxane containing Si−H group with a vinyl-terminated compound containing the mesogenic group. Their chemical structures were confirmed by IR,1H NMR, and elemental analysis, Their phase behaviors were investigated by differential scanning calorimetry, optical polarizing microscope, and X-ray diffraction technique. The results show that all these polysiloxanes except P V display smectic or nematic mesonorphism or both. The dependence of phase transition temperature on the rigidity of mesogenic unit. the length of spacer group, and the flexibility of\(\rlap{--} (Si(CH_3 )_2 O\rlap{--} )\) chain is also described.

Food-grade microemulsions containing oleic acid, ethanol, Tween 20, and water were formulated as a carrier system for tea seed oil (Camellia oleifera Abel.). The effect of ethanol on the phase behavior of the microemulsion system was clearly reflected in pseudo-ternary diagrams. The solubilization capacity and solubilization efficiency of tea seed oil dispersions were measured along the dilution line at a 70/30 surfactant/oil mass ratio with Tween 20 as the surfactant and oleic acid and ethanol (1:3, w/w) as the oil phase. The dispersed phase of the microemulsion (1.5% weight ratio of tea seed oil to the total amount of oil, surfactant, and tea seed oil) could be fully diluted with water without phase separation. Differential scanning calorimetry and viscosity measurements indicated that both the carrier and solubilized systems underwent a similar microstructure transition upon dilution. The dispersion phases gradually inverted from the water-in-oil phase (< 35% water) to the bicontinuous phase (40-45% water) and finally to the oil-in-water phase (> 45% water) along the dilution line.

Synthetic aperture radar interferometry was used to study the Maergaichaka fault where Manyi earthquake occured on Nov. 8, 1997 in Tibet, China. With a more appropriate base line, we present the coseismic interferometric fringe. With the prior knowledge of strike-slip movements, the decomposition of the displacement vector in the direction of the fault strike indicates that the horizontal displacement amount to 5.03 meters near the epicenter of the Manyi earthquake, which is more consistent with the field observation (4.5 meters) than the previous work (Peltzer et al., 1999; Shan Xinjian et al., 2002). We also modeled the rock physical parameter using Okada elastic model of half space, which was characterized by a high Poisson ratio. The high Poisson ratio of the upper crust indicate the structural back grounding that it is more possible that the interior crustal shortening play an more important role in the development of Tibet plateau than the doubling the continental crust by replacement of the Tibetan upper mantle by underthrust of Indian continental crust

Purpose: This study is in an attempt to assess the diagnostic accuracy of infrared lightscanning by comparing with that of mammography. Methods: A total of 104 patients had been examined by both mammography and infrared lightscanning before surgery. All patients were divided into two groups: cancer and non cancer. The diagnostic accuracy of these two modalities were calculated. Results: Of 104 patients, 43 had breast cancer and 61 had benign lesions, the sensitivity and specificity for mammography were 84% and 83%; 82% and 77% for infrared lightscanning. The predictive values of positivity for mammography and lightscanning were 80% and 70%; the negative predictive value for these two modalities were both 87%. Conclusion: Infrared lightscanning, being of assistance to mammography, could enhance sensitivity and predictive values of positivity in detecting breast cancer, especially, in mammographically dense breast.

A new serial chiral side-chain liquid crystal polysiloxanes containing L-Tyr structures were synthesized at room temperature. Both of them exhibited Sc phase at room temperature.

Optimizing surface passivation Unproductive charge recombination at surface defects can limit the efficiency of hybrid perovskite solar cells, but these defects can be passivated by the binding of small molecules. Wang et al. studied three such small molecules—theophylline, caffeine, and theobromine—that bear both carbonyl and amino groups. For theophylline, hydrogen bonding of the amino hydrogen to surface iodide optimized the carbonyl interaction with a lead antisite defect and improved the efficiency of a perovskite cell from 21 to 22.6%. Science , this issue p. 1509