Search

Chain of thought finetuning (cot-finetuning) aims to endow small language models (SLM) with reasoning ability to improve their performance towards specific tasks by allowing them to imitate the reasoning procedure of large language models (LLM) beyond simply predicting the answers. Most existing cot-finetuning methods adopt a pre-thinking mechanism, allowing the SLM to generate a rationale before providing an answer. This mechanism enables SLM to analyze and think about complex questions, but it also makes answer correctness highly sensitive to minor errors in rationale. Therefore, we propose a robust post-thinking mechanism to generate answers before rationale. Thanks to this answer-first setting, 1) the answer can escape from the adverse effects caused by minor errors in the rationale; 2) the rationale serves as an error amplifier to the answer, which makes the SLM focus on learning hard samples; 3) the inferring efficiency can also benefit from the setting since users can stop the generation right after answers are outputted when inference is conducted. However, although the post-thinking mechanism brings many advantages and improves the overall performance of SLM on specific tasks, it may lose the ability to think about the questions and decompose complex questions into simple sub-questions compared to pre-thinking mechanism. Therefore, a plug-and-play adaptive-thinking mechanism is proposed with the aid of the soft prompt tuning to integrate the merits of the pre-thinking mechanism and post-thinking mechanism, in which a perception module is introduced to adaptively prompt SLM answer or think first based on perceiving the complexity of the questions. Extensive experiments are conducted across 12 reasoning tasks and 2 representative language models to demonstrate the effectiveness of the proposed mechanism.

Eliminating examination bias accurately is pivotal to apply click-through data to train an unbiased ranking model. However, most examination-bias estimators are limited to the hypothesis of Position-Based Model (PBM), which supposes that the calculation of examination bias only depends on the rank of the document. Recently, although some works introduce information such as clicks in the same query list and contextual information when calculating the examination bias, they still do not model the impact of document representation on search engine result pages (SERPs) that seriously affects one's perception of document relevance to a query when examining. Therefore, we propose a Multi-Feature Integration Model (MFIM) where the examination bias depends on the representation of document except the rank of it. Furthermore, we mine a key factor slipoff counts that can indirectly reflects the influence of all perception-bias factors. Real world experiments on Baidu-ULTR dataset demonstrate the superior effectiveness and robustness of the new approach. The source code is available at \href{https://github.com/lixsh6/Tencent_wsdm_cup2023/tree/main/pytorch_unbias}{https://github.com/lixsh6/Tencent\_wsdm\_cup2023}

Pre-trained language models have achieved great success in various large-scale information retrieval tasks. However, most of pretraining tasks are based on counterfeit retrieval data where the query produced by the tailored rule is assumed as the user's issued query on the given document or passage. Therefore, we explore to use large-scale click logs to pretrain a language model instead of replying on the simulated queries. Specifically, we propose to use user behavior features to pretrain a debiased language model for document ranking. Extensive experiments on Baidu desensitization click logs validate the effectiveness of our method. Our team on WSDM Cup 2023 Pre-training for Web Search won the 1st place with a Discounted Cumulative Gain @ 10 (DCG@10) score of 12.16525 on the final leaderboard.

We have combined atomic layer lithography and template stripping to produce a new class of substrates for surface-enhanced infrared absorption (SEIRA) spectroscopy. Our structure consists of a buried and U-shaped metal-insulator-metal waveguide whose folded vertical arms efficiently couple normally incident light. The insulator is formed by atomic layer deposition (ALD) of Al2O3 and precisely defines the gap size. The buried nanocavities are protected from contamination by a silicon template until ready for use and exposed by template stripping on demand. The exposed nanocavity generates strong infrared resonances, tightly confines infrared radiation into a gap that is as small as 3 nm ({\lambda}/3300), and creates a dense array of millimeter-long hotspots. After partial removal of the insulators, the gaps are backfilled with benzenethiol molecules, generating distinct Fano resonances due to strong coupling with gap plasmons, and a SEIRA enhancement factor of 10^5 is observed for a 3 nm gap. Because of the wafer-scale manufacturability, single-digit-nanometer control of the gap size via ALD, and long-term storage enabled by template stripping, our buried plasmonic nanocavity substrates will benefit broad applications in sensing and spectroscopy.

Global context information is vital in visual understanding problems, especially in pixel-level semantic segmentation. The mainstream methods adopt the self-attention mechanism to model global context information. However, pixels belonging to different classes usually have weak feature correlation. Modeling the global pixel-level correlation matrix indiscriminately is extremely redundant in the self-attention mechanism. In order to solve the above problem, we propose a hierarchical context network to differentially model homogeneous pixels with strong correlations and heterogeneous pixels with weak correlations. Specifically, we first propose a multi-scale guided pre-segmentation module to divide the entire feature map into different classed-based homogeneous regions. Within each homogeneous region, we design the pixel context module to capture pixel-level correlations. Subsequently, different from the self-attention mechanism that still models weak heterogeneous correlations in a dense pixel-level manner, the region context module is proposed to model sparse region-level dependencies using a unified representation of each region. Through aggregating fine-grained pixel context features and coarse-grained region context features, our proposed network can not only hierarchically model global context information but also harvest multi-granularity representations to more robustly identify multi-scale objects. We evaluate our approach on Cityscapes and the ISPRS Vaihingen dataset. Without Bells or Whistles, our approach realizes a mean IoU of 82.8% and overall accuracy of 91.4% on Cityscapes and ISPRS Vaihingen test set, achieving state-of-the-art results.

We study the physical-layer security of a multiple source-destination (SD) pairs coexisting wireless network in the face of an eavesdropper, where an eavesdropper intends to wiretap the signal transmitted by the SD pairs. In order to protect the wireless transmission against eavesdropping, we propose a cooperation framework relying on two stages. Specifically, an SD pair is selected to access the total allocated spectrum using an appropriately designed scheme at the beginning of the first stage. The other source nodes (SNs) simultaneously transmit their data to the SN of the above-mentioned SD pair relying on an orthogonal way during the first stage. Then, the SN of the chosen SD pair transmits the data packets containing its own messages and the other SNs' messages to its dedicated destination node (DN) in the second stage, which in turn will forward all the other DNs' data to the application center via the core network. We conceive a specific SD pair selection scheme, termed as the transmit antenna selection aided source-destination pair selection (TAS-SDPS). We derive the secrecy outage probability (SOP) expressions for the TAS-SDPS, as well as for the conventional round-robin source-destination pair selection (RSDPS) and non-cooperative (Non-coop) schemes for comparison purposes. Furthermore, we carry out the secrecy diversity gain analysis in the high main-to-eavesdropper ratio (MER) region, showing that the TAS-SDPS scheme is capable of achieving the maximum attainable secrecy diversity order.