Search

The use of large language models (LLMs) is expanding rapidly, and open-source versions are becoming available, offering users safer and more adaptable options. These models enable users to protect data privacy by eliminating the need to provide data to third parties and can be customized for specific tasks. In this study, we compare the performance of various language models on the Sustainable Development Goal (SDG) mapping task, using the output of GPT-4o as the baseline. The selected open-source models for comparison include Mixtral, LLaMA 2, LLaMA 3, Gemma, and Qwen2. Additionally, GPT-4o-mini, a more specialized version of GPT-4o, was included to extend the comparison. Given the multi-label nature of the SDG mapping task, we employed metrics such as F1 score, precision, and recall with micro-averaging to evaluate different aspects of the models' performance. These metrics are derived from the confusion matrix to ensure a comprehensive evaluation. We provide a clear observation and analysis of each model's performance by plotting curves based on F1 score, precision, and recall at different thresholds. According to the results of this experiment, LLaMA 2 and Gemma still have significant room for improvement. The other four models do not exhibit particularly large differences in performance. The outputs from all seven models are available on Zenodo: https://doi.org/10.5281/zenodo.12789375.

In recent years, Natural Language Processing (NLP) has played a significant role in various Artificial Intelligence (AI) applications such as chatbots, text generation, and language translation. The emergence of large language models (LLMs) has greatly improved the performance of these applications, showing astonishing results in language understanding and generation. However, they still show some disadvantages, such as hallucinations and lack of domain-specific knowledge, that affect their performance in real-world tasks. These issues can be effectively mitigated by incorporating knowledge graphs (KGs), which organise information in structured formats that capture relationships between entities in a versatile and interpretable fashion. Likewise, the construction and validation of KGs present challenges that LLMs can help resolve. The complementary relationship between LLMs and KGs has led to a trend that combines these technologies to achieve trustworthy results. This work collected 28 papers outlining methods for KG-powered LLMs, LLM-based KGs, and LLM-KG hybrid approaches. We systematically analysed and compared these approaches to provide a comprehensive overview highlighting key trends, innovative techniques, and common challenges. This synthesis will benefit researchers new to the field and those seeking to deepen their understanding of how KGs and LLMs can be effectively combined to enhance AI applications capabilities.

Deep unfolding methods have made impressive progress in restoring 3D hyperspectral images (HSIs) from 2D measurements through convolution neural networks or Transformers in spectral compressive imaging. However, they cannot efficiently capture long-range dependencies using global receptive fields, which significantly limits their performance in HSI reconstruction. Moreover, these methods may suffer from local context neglect if we directly utilize Mamba to unfold a 2D feature map as a 1D sequence for modeling global long-range dependencies. To address these challenges, we propose a novel Dual Hyperspectral Mamba (DHM) to explore both global long-range dependencies and local contexts for efficient HSI reconstruction. After learning informative parameters to estimate degradation patterns of the CASSI system, we use them to scale the linear projection and offer noise level for the denoiser (i.e., our proposed DHM). Specifically, our DHM consists of multiple dual hyperspectral S4 blocks (DHSBs) to restore original HSIs. Particularly, each DHSB contains a global hyperspectral S4 block (GHSB) to model long-range dependencies across the entire high-resolution HSIs using global receptive fields, and a local hyperspectral S4 block (LHSB) to address local context neglect by establishing structured state-space sequence (S4) models within local windows. Experiments verify the benefits of our DHM for HSI reconstruction. The source codes and models will be available at https://github.com/JiahuaDong/DHM., Comment: 13 pages, 6 figures

Room-temperature superconductivity represents a significant scientific milestone, with the initial report of LK-99, a copper-substituted lead apatite $\mathrm{Pb}_{10-x}\mathrm{Cu}_{x}(\mathrm{PO}_{4})_{6}\mathrm{O}$, offering a potential breakthrough. However, other researchers have encountered numerous challenges in replicating the original experimental results. In recent studies, Wang et al. successfully observed signs of a possible superconducting phase, such as smaller resistance and stronger diamagnetism, upon doping S into the samples. This indicates that the introduction of S is of significant importance for achieving an appropriate structure. To further investigate the role of S, we have considered the $\mathrm{Pb}_{10-x}\mathrm{Cu}_{x}(\mathrm{PO}_{4})_{6}\mathrm{S}$, systematically discussing its thermodynamic stability, as well as the influence of S on the distribution, concentration, and electronic properties of Cu. We find that $\mathrm{Pb}_{10-x}\mathrm{Cu}_{x}(\mathrm{PO}_{4})_{6}\mathrm{S}$ maintains thermodynamic stability, with S primarily influencing the distribution of Cu. The critical element dictating the electronic characteristics of the material post-synthesis is Cu, while the impact of S on the electronic properties is relatively minor. Our work provides valuable insights into the synthesis of potential apatite based room-temperature superconductors and the role of S in facilitating Cu doping., Comment: 8 pages, 4 figures

Two entangled qubits emerge as an essential resource for quantum control, which are normally quantum confined with atomic precision. It seems inhibitive that in the macroscopic scope collective qubit pairs manifest long coherence and quantum entanglement, especially at high temperature. Here, we report this exotic ensemble effect in solid-state sintering lead apatite samples with copper substitution, which have been repeatedly duplicated with superior stability and low cost. An extraordinarily low-field absorption signal of cw electron paramagnetic resonance (EPR) spectroscopy stems from low-mass conduction electrons implying the coherence of cuprate radicals can be long-termly protected. The pulsed EPR experiments exhibit triplet Rabi oscillation from paired cuprate diradicals with the coherence time exceeding 1 microsecond at 85K. We believe these appealing effects are sufficiently promising to be applied for scalable quantum control and computation., Comment: 16 pages, 4 figures

Aiming to restore the original intensity of shadow regions in an image and make them compatible with the remaining non-shadow regions without a trace, shadow removal is a very challenging problem that benefits many downstream image/video-related tasks. Recently, transformers have shown their strong capability in various applications by capturing global pixel interactions and this capability is highly desirable in shadow removal. However, applying transformers to promote shadow removal is non-trivial for the following two reasons: 1) The patchify operation is not suitable for shadow removal due to irregular shadow shapes; 2) shadow removal only needs one-way interaction from the non-shadow region to the shadow region instead of the common two-way interactions among all pixels in the image. In this paper, we propose a novel cross-region transformer, namely CRFormer, for shadow removal which differs from existing transformers by only considering the pixel interactions from the non-shadow region to the shadow region without splitting images into patches. This is achieved by a carefully designed region-aware cross-attention operation that can aggregate the recovered shadow region features conditioned on the non-shadow region features. Extensive experiments on ISTD, AISTD, SRD, and Video Shadow Removal datasets demonstrate the superiority of our method compared to other state-of-the-art methods.

A new feature of the chaotic signal generated by chaotic shape-forming filter (CSF) is uncovered in this work. We find that, the autocorrelation function (ACF) of the transmitting signal generated by CSF keeps the same as that of the base function of CSF, no matter what information is encoded. We derive the analytical equation to describe the relation between the ACF of the received signal and the wireless channel parameters using the ACF of the transmitted signal as prior knowledge revealed by the finding in this work. This new property can be utilized together with different wireless communication systems to improve the system performance. Specially, to demonstrate the improvement, channel state information (CSI) is identified using the chaotic baseband wireless communication as a paradigm. Two significant benefits by using the new property are 1) the CSI can be identified without the probe information known to the receiver as done in the conventional wireless communication systems, which improves the bandwidth efficiency, especially in the time-varying channel; 2) the correlation operation is insensitive to the channel noise, which improves the identification accuracy as compared to the commonly used methods., Comment: 5 pages,4 figures

The outbreak of the novel Coronavirus Disease 2019 (COVID-19) has lasted for nearly two years and caused unprecedented impacts on people's daily life around the world. Even worse, the emergence of the COVID-19 Delta variant once again puts the world in danger. Fortunately, many countries and companies have started to develop coronavirus vaccines since the beginning of this disaster. Till now, more than 20 vaccines have been approved by the World Health Organization (WHO), bringing light to people besieged by the pandemic. The promotion of COVID-19 vaccination around the world also brings a lot of discussions on social media about different aspects of vaccines, such as efficacy and security. However, there does not exist much research work to systematically analyze public opinion towards COVID-19 vaccines. In this study, we conduct an in-depth analysis of tweets related to the coronavirus vaccine on Twitter to understand the trending topics and their corresponding sentimental polarities regarding the country and vaccine levels. The results show that a majority of people are confident in the effectiveness of vaccines and are willing to get vaccinated. In contrast, the negative tweets are often associated with the complaints of vaccine shortages, side effects after injections and possible death after being vaccinated. Overall, this study exploits popular NLP and topic modeling methods to mine people's opinions on the COVID-19 vaccines on social media and to analyse and visualise them objectively. Our findings can improve the readability of the noisy information on social media and provide effective data support for the government and policy makers.

Effective representation learning is critical for short text clustering due to the sparse, high-dimensional and noise attributes of short text corpus. Existing pre-trained models (e.g., Word2vec and BERT) have greatly improved the expressiveness for short text representations with more condensed, low-dimensional and continuous features compared to the traditional Bag-of-Words (BoW) model. However, these models are trained for general purposes and thus are suboptimal for the short text clustering task. In this paper, we propose two methods to exploit the unsupervised autoencoder (AE) framework to further tune the short text representations based on these pre-trained text models for optimal clustering performance. In our first method Structural Text Network Graph Autoencoder (STN-GAE), we exploit the structural text information among the corpus by constructing a text network, and then adopt graph convolutional network as encoder to fuse the structural features with the pre-trained text features for text representation learning. In our second method Soft Cluster Assignment Autoencoder (SCA-AE), we adopt an extra soft cluster assignment constraint on the latent space of autoencoder to encourage the learned text representations to be more clustering-friendly. We tested two methods on seven popular short text datasets, and the experimental results show that when only using the pre-trained model for short text clustering, BERT performs better than BoW and Word2vec. However, as long as we further tune the pre-trained representations, the proposed method like SCA-AE can greatly increase the clustering performance, and the accuracy improvement compared to use BERT alone could reach as much as 14\%., Comment: To be published in 22nd International Conference on Web Information Systems Engineering (WISE2021)