Search

Human cognition significantly influences expressed behavior and is intrinsically tied to authentic personality traits. Personality assessment plays a pivotal role in various fields, including psychology, education, social media, etc. However, traditional self-report questionnaires can only provide data based on what individuals are willing and able to disclose, thereby lacking objective. Moreover, automated measurements and peer assessments demand significant human effort and resources. In this paper, given the advantages of the Virtual Reality (VR) technique, we develop a VR simulator --- PersonalityScanner, to stimulate cognitive processes and simulate daily behaviors based on an immersive and interactive simulation environment, in which participants carry out a battery of engaging tasks that formulate a natural story of first-day at work. Through this simulator, we collect a synchronous multi-modal dataset with ten modalities, including first/third-person video, audio, text, eye tracking, facial microexpression, pose, depth data, log, and inertial measurement unit. By systematically examining the contributions of different modalities on revealing personality, we demonstrate the superior performance and effectiveness of PersonalityScanner.

Qing-Hui Li,1,2 Qiu-Yan Zhao,1 Wei-Jing Yang,1 Ai-Fang Jiang,2 Chun-E Ren,2 Yu-Han Meng2 1School of Clinical Medicine, Shandong Second Medical University, Weifang, Shandong, 261021, People’s Republic of China; 2Center of Reproductive Medicine, Affiliated Hospital of Shandong Second Medical University, Weifang, Shandong, 261000, People’s Republic of ChinaCorrespondence: Yu-Han Meng, Center of Reproductive Medicine, Affiliated Hospital of Shandong Second Medical University, No. 2428.Yuhe Road, Kuiwen District, Weifang, Shandong, 261031, People’s Republic of China, Tel/Fax +86 536-3081389, Email yhmeng6@163.comAbstract: Recurrent spontaneous abortion (RSA) is defined as two or more consecutive pregnancy failures, which brings tremendous stress to women of childbearing age and seriously affects family well-being. However, the reason in about 50% of cases remains unknown and is defined as unexplained recurrent spontaneous abortion (URSA). The immunological perspective in URSA has attracted widespread attention in recent years. The embryo is regarded as a semi-allogeneic graft to the mother. A successful pregnancy requires transition to an immune environment conducive to embryo survival at the maternal–fetal interface. As an important member of regulatory immunity, regulatory T (Treg) cells play a key role in regulating immune tolerance at the maternal–fetal interface. This review will focus on the phenotypic plasticity and lineage stability of Treg cells to illustrate its relationship with URSA.Keywords: immune homeostasis, Treg cells, phenotype, maternal–fetal tolerance

The micronewton thruster is one of the essential technologies for providing an ultrastable satellite platform for scientific experiments in space. Particularly, space gravitational wave detection requires a microthruster with continuously adjustable thrust, high resolution, low noise, and fast response. To meet these requirements, we designed a two-dimensional (2D) nozzle and manufactured it using micro-electro-mechanical systems (MEMS). Compared to the nozzles machined with conventional methods, the nozzles etched by the MEMS can easily reach a micron size and form arrays to create mutual backups and also provide a way to widen the thrust range by switching the number of openings. Based on a single 2D nozzle, a cold gas microthruster prototype is constructed with nitrogen gas as the propellant. The thrust of the prototype is tested by a pendulum, the range can span 0-156 mu N, the thrust resolution is better than 0.1 mu N, and the thrust noise reaches 0.1 mu N/Hz1/2 at 0.1 Hz. The result demonstrates that the scheme has the potential to be applied to gravitational wave detection missions.

Heterogeneous Graph Neural Networks (HGNNs) have consistently demonstrated exceptional performance in the context of Heterogeneous Information Networks (HINs). However, the majority of existing methods are constrained by their applicability to semi-supervised learning scenarios, which entail substantial labor and resources. Recently, the advent of contrastive learning has catalyzed the development of self-supervised HGNN approaches. In this study, we introduce NHGMI, a novel unsupervised method for heterogeneous graph representation learning, rooted in the principles of mutual information maximization. NHGMI comprehensively harnesses the inherent characteristics of HINs and adeptly amalgamates diverse semantic information emanating from multiple contrasting perspectives. In addition to inter-view contrasts, NHGMI also incorporates intra-view contrasts. Diverging from conventional contrastive learning approaches, NHGMI meticulously selects contrasting samples based on similarity metrics, thereby achieving a noise-free contrastive learning paradigm. Furthermore, we propose an extension of NHGMI that focuses on the generation of negative samples. We conduct extensive experiments across various real-world datasets, and NHGMI consistently exhibits marked superiority when compared to relevant methods, even surpassing semi-supervised approaches. The source code for NHGMI is publicly available at https://github.com/Frederick-the-Fox/NHGMI. © 2024 Elsevier B.V.

Advances in coastal modeling and computation provide the opportunity to examine non-hydrostatic and compressible fluid effects at very small scales, but the cost of these new capabilities and the accuracy of these models versus trusted non-hydrostatic codes has yet to be determined. Here the Coastal and Regional Ocean COmmunity model (CROCO, v1.2) and the National Center for Atmospheric Research large-eddy simulation (NCAR-LES) model are compared, with a focus on their simulation accuracy and computational efficiency. These models differ significantly in numerics and capabilities, so they are run on common classic problems of surface-forced, boundary-layer turbulence. In terms of accuracy, we compare turbulence statistics, including the effect of the explicit subgrid-scale (SGS) parameterization, the effect of the second (dilatational) viscosity, and the sensitivity to the speed of sound, which is used as part of the CROCO compressible turbulence formulation. To gauge how far CROCO is from the NCAR-LES, we first compare the NCAR-LES with two other non-hydrostatic Boussinesq approximation LES codes (PALM and Oceananigans), defining the notion and magnitude of accuracy for the LES and CROCO comparison. To judge efficiency of CROCO, strong and weak scaling simulation sets vary different problem sizes and workloads per processor, respectively. Additionally, the effects of 2D decomposition of CROCO and NCAR-LES and supercomputer settings are tested. In summary, the accuracy comparison between CROCO and the NCAR-LES is similar to the NCAR-LES compared to other LES codes. However, the additional capabilities of CROCO (e.g., nesting, non-uniform grid, and realism of ocean configuration in general) and its weakly compressible formulation come with roughly an order of magnitude of additional costs, despite efforts to reduce them by adjusting the second viscosity and speed of sound as far as accuracy allows. However, a new variant of the non-hydrostatic CROCO formulation is cu

Study Objectives: Excessive daytime sleepiness (EDS) is a major symptom of obstructive sleep apnea (OSA). Traditional polysomnographic (PSG) measures only partially explain EDS in OSA. This study analyzed traditional and novel PSG characteristics of two different measures of EDS among patients with OSA. Methods: Sleepiness was assessed using the Epworth Sleepiness Scale (>10 points defined as "risk of dozing") and a measure of general sleepiness (feeling sleepy >= 3 times/week defined as "feeling sleepy"). Four sleepiness phenotypes were identified: "non-sleepy," "risk of dozing only," "feeling sleepy only," and "both at risk of dozing and feeling sleepy." Results: Altogether, 2083 patients with OSA (69% male) with an apnea-hypopnea index (AHI) >= 5 events/hour were studied; 46% were "non-sleepy," 26% at "risk of dozing only," 7% were "feeling sleepy only," and 21% reported both. The two phenotypes at "risk of dozing" had higher AHI, more severe hypoxemia (as measured by oxygen desaturation index, minimum and average oxygen saturation [SpO(2)], time spent < 90% SpO(2), and hypoxic impacts) and they spent less time awake, had shorter sleep latency, and higher heart rate response to arousals than "non-sleepy" and "feeling sleepy only" phenotypes. While statistically significant, effect sizes were small. Sleep stages, frequency of arousals, wake after sleep onset and limb movement did not differ between sleepiness phenotypes after adjusting for confounders. Conclusions: In a large international group of patients with OSA, PSG characteristics were weakly associated with EDS. The physiological measures differed among individuals characterized as "risk of dozing" or "non-sleepy," while "feeling sleepy only" did not differ from "non-sleepy" individuals.

We model a multiphase and high-volume recruitment process as a large-scale dynamic program. The success of the process is measured by a reward, which is the total assessment score of accepted candidates minus the penalty cost of the number of accepted candidates in the end deviating from a preset hiring target. For a recruiter, two questions are important: How many offers should be made in each phase? And how does the number of phases affect the reward? We consider an upper bound, which is obtained when the information about all candidates is available at the beginning, and a lower bound, which is obtained when the recruiter sets the number of offers to make in each phase before assessing candidates. We show that when the volume (i.e., arrival rate of candidates and the target) is large, the upper bound, the lower bound, and the optimal policy all converge to the same limit. Motivated by the convergence results, we design four easily computable heuristics that are all asymptotically optimal when the volume is large. With simple yet effective heuristics in hand, we can compute the number of offers to make in each phase and examine the impact of the number of phases in the process on the reward. We apply our modeling framework and heuristics to the recruitment process of graduate students in a business program. Our study is the first to model a high-volume recruitment process as a dynamic program and test it in a case study.

In this study, we established a new methodology for preparing 5-nucleotidase (5-NT) with the aim of enhancing our understanding of its enzyme activity and laying a basis for regulating the content of umami-enhancing nucleotides in pork. 5-NT was prepared with Sephadex gel filtration and reverse-phase high-performance liquid chromatography, and its enzymatic properties and catalytic activity were evaluated. The results show that the molecular weight of the prepared 5-NT was 57 kDa, the optimal catalytic temperature was 40 °C, and the optimal pH was 8. Zn2+, and sucrose showed inhibitory effects on the activity of 5-NT, while K+, Na+, Ca2+, Mg2+, glucose, fructose, and trehalose promoted the activity of the studied compound. The prepared 5-NT exhibited higher catalytic activity and selectivity against IMP compared with its commercial counterpart, while its catalytic activity against XMP was not significant (p > 0.05). In brief, we established a new methodology for preparing 5-NT, enhancing our understanding of its enzyme activity and providing a solid basis for regulating the content of umami-enhancing nucleotides in pork through the control of endogenous 5-NT activity.

Glycyrrhizic acid (GA) is a natural edible chiral triterpene saponin that can undergo hierarchical self-assembly in water to form supramolecular hydrogels. Customizing the properties of GA supramolecular hydrogels is crucial for their wide food and biomedical applications, and the incorporation of specific metal ions can be an attractive way for this purpose. Herein, this study investigates the effects of metal ions with different valence states and concentrations on the linear and nonlinear rheological properties and network structures of GA supramolecular hydrogels. Monovalent metal ions (Na+ and K+), exhibiting weak binding affinity to GA, rely on the electrostatic screening effects at high concentrations (e.g., 50 mM) to enhance the GA interfibrillar interactions and thus the formation of a more compact and ordered gel network, which displays a pronounced nonlinear rheological behavior with a typical transition from elastic to viscous response. Polyvalent metal ions (Ca2+, Zn2+, and Al3+), owing to their greater charge density and stronger binding with GA, significantly enhance the network strength of the hydrogels at low concentrations (e.g., 5 mM). Nonetheless, at high ion concentrations (e.g., 50 mM), the GA-Mn+ forms discrete aggregated network structures due to excessively strong cation-carboxylate complexation, leading to the irregular nonlinear rheological responses and the lower resistance to large deformations. These findings can deepen our understanding of the highly tunable rheological behaviors and network structures of GA hydrogels, which demonstrates a new possibility for the design and development of responsive natural supramolecular hydrogels with controlled properties through the strategy of metal ion complexation.

Proper cell fate determination relies on precise spatial and temporal genome-wide cooperation between regulatory elements (REs) and their targeted genes. However, the lengths of REs defined using different methods vary, which indicates that there is sequence redundancy and that the context of the genome may be unintelligible. We developed a method called MAE-seq (Massive Active Enhancers by Sequencing) to experimentally identify functional REs at a 25-bp scale. In this study, MAE-seq was used to identify 626879, 541617 and 554826 25-bp enhancers in mouse embryonic stem cells (mESCs), C2C12 and HEK 293T, respectively. Using ∼1.6 trillion 25 bp DNA fragments and screening 12 billion cells, we identified 626879 as active enhancers in mESCs as an example. Comparative analysis revealed that most of the histone modification datasets were annotated by MAE-Seq loci. Furthermore, 33.85% (212195) of the identified enhancers were identified as de novo ones with no epigenetic modification. Intriguingly, distinct chromatin states dictate the requirement for dissimilar cofactors in governing novel and known enhancers. Validation results show that these 25-bp sequences could act as a functional unit, which shows identical or similar expression patterns as the previously defined larger elements, Enhanced resolution facilitated the identification of numerous cell-specific enhancers and their accurate annotation as super enhancers. Moreover, we characterized novel elements capable of augmenting gene activity. By integrating with high-resolution Hi-C data, over 55.64% of novel elements may have a distal association with different targeted genes. For example, we found that the Cdh1 gene interacts with one novel and two known REs in mESCs. The biological effects of these interactions were investigated using CRISPR-Cas9, revealing their role in coordinating Cdh1 gene expression and mESC proliferation. Our study presents an experimental approach to refine the REs at 25-bp resolution, adva

Yang Liu,1,2,&ast; Hui Yang,1,2,&ast; Xueshu Fu,1,2 Luting Zhong,1,2 Ping Xu,1,2 Fang Fang,1,2 Ying Liu,1,2 Qing Li,1,2 Ya’nan Yan,1,2 Shanchuang Wei,1,2 Junqing Wang,1,2 Chunhua Zhang1– 3 1Department of Gynaecology, Affiliated Hospital of YangZhou University Huai ‘an Maternal and Child Health Care Center, Huai’an, Jiangsu, People’s Republic of China; 2Department of Gynaecology, The Huai’an Maternity and Child Clinical College of Xuzhou Medical University, Huai’an, Jiangsu, People’s Republic of China; 3Department of Integrated Traditional Chinese & Western Clinical Medicine, Macau University of Science and Technology, Macau, People’s Republic of China&ast;These authors contributed equally to this workCorrespondence: Chunhua Zhang, Department of Gynaecology, The Huai’an Maternity and Child Clinical College of Xuzhou Medical University, Huai’an, Jiangsu, People’s Republic of China, Tel +86-13952326117, Email hafyzch@126.comBackground: Multiple primary cancer (MPC) refers to the presence of more than one cancer in an individual. Triple primary malignancies are uncommon.Case: We report the case of a 50-year-old postmenopausal woman in our gynecology department, diagnosed with endometrial cancer, ovarian cancer, and unilateral breast cancer. She carried germline mutations in BRCA2, PALB2, and RECQL4, along with a somatic pathogenic variant in TP53. Endometrial cancer patients harboring germline pathogenic variants in BRCA2 exhibit a heightened risk of ovarian and breast cancer. BRCA2 is known to play a role in the development of ovarian and breast cancer, while PALB2 is identified as a gene associated with breast cancer susceptibility. RECQL4 has been linked to breast cancer, cervical cancer, and other tumors.Conclusion: Genetic testing may be imperative for identifying MPC in endometrial cancer patients. For individuals with BRCA2 and other gene pathogenic variants, routine examination and monitoring of the endometrium, ovaries, b

While various parameterizations of vertical turbulent fluxes at different levels of complexity have been proposed, each has its own limitations. For example, simple first-order closure schemes such as the K-Profile Parameterization (KPP) lack energetic constraints; two-equation models like k-e $k-\textbackslash{}varepsilon $ directly solve an equation for the turbulent kinetic energy but do not account for non-diffusive fluxes, and high-order closures that include the high-order transport terms are computationally expensive. To address these, we extend the Assumed-Distribution Higher-Order Closure (ADC) framework originally proposed for the atmospheric boundary layer and apply it to the ocean surface boundary layer. By assuming a probability distribution function relationship between the vertical velocity and tracers, all second-order and higher-order moments are exactly constructed and turbulence closure is achieved in the ADC scheme. In addition, this ADC parameterization has full energetic constraints and includes non-diffusive fluxes without the computational cost of a full higher-order closure scheme. We have tested the ADC scheme against a combination of large eddy simulation (LES), KPP, and k-e $k-\textbackslash{}varepsilon $ for surface buoyancy-driven convective mixing and found that the ADC scheme is robust with different vertical resolutions and compares well to the LES results. The upper ocean (order of few tens of meters depth from the surface) has a substantial influence on our climate and weather systems. Specifically, upper ocean mixing processes play a key role in modulating global heat budget in the ocean and atmosphere by mixing heat deeper into the ocean or warming the atmosphere above. Accurate representation of the effects of these mixing processes on the global climate and in ocean models is crucial for understanding our current and changing climate. However, current mixing schemes used in these models have shown significant biases. We present

In eukaryotes, DNA compacts into chromatin through nucleosomes1,2. Replication of the eukaryotic genome must be coupled to the transmission of the epigenome encoded in the chromatin3,4. Here we report cryo-electron microscopy structures of yeast (Saccharomyces cerevisiae) replisomes associated with the FACT (facilitates chromatin transactions) complex (comprising Spt16 and Pob3) and an evicted histone hexamer. In these structures, FACT is positioned at the front end of the replisome by engaging with the parental DNA duplex to capture the histones through the middle domain and the acidic carboxyl-terminal domain of Spt16. The H2A–H2B dimer chaperoned by the carboxyl-terminal domain of Spt16 is stably tethered to the H3–H4 tetramer, while the vacant H2A–H2B site is occupied by the histone-binding domain of Mcm2. The Mcm2 histone-binding domain wraps around the DNA-binding surface of one H3–H4 dimer and extends across the tetramerization interface of the H3–H4 tetramer to the binding site of Spt16 middle domain before becoming disordered. This arrangement leaves the remaining DNA-binding surface of the other H3–H4 dimer exposed to additional interactions for further processing. The Mcm2 histone-binding domain and its downstream linker region are nested on top of Tof1, relocating the parental histones to the replisome front for transfer to the newly synthesized lagging-strand DNA. Our findings offer crucial structural insights into the mechanism of replication-coupled histone recycling for maintaining epigenetic inheritance.

With the increasing popularity of smart homes, more and more devices need to connect to home networks. Traditional home networks mainly rely on centralized networking, where an excessive number of devices in the centralized topology can increase the pressure on the central router, potentially leading to decreased network performance metrics such as communication latency. To address the latency performance issues brought about by centralized networks, this paper proposes a new network system called DHNet, and designs an algorithm for clustering networking and communication based on vector routing. Communication within clusters in a simulated virtual environment achieves a latency of approximately 0.7 milliseconds. Furthermore, by directly using the first non-"lo" network card address of a device as the protocol's network layer address, the protocol avoids the several tens of milliseconds of access latency caused by DHCP. The integration of service discovery functionality into the network layer protocol is achieved through a combination of "server-initiated service push" and "client request + server reply" methods. Compared to traditional application-layer DNS passive service discovery, the average latency is reduced by over 50%. The PVH protocol is implemented in the user space using the Go programming language, with implementation details drawn from Google's gVisor project. The code has been ported from x86\_64 Linux computers to devices such as OpenWrt routers and Android smartphones. The PVH protocol can communicate through "tunnels" to provide IP compatibility, allowing existing applications based on TCP/IP to communicate using the PVH protocol without requiring modifications to their code.

Anomaly-based network intrusion detection systems (A-NIDS) use unsupervised models to detect unforeseen attacks. However, existing A-NIDS solutions suffer from low throughput, lack of interpretability, and high maintenance costs. Recent in-network intelligence (INI) exploits programmable switches to offer line-rate deployment of NIDS. Nevertheless, current in-network NIDS are either model-specific or only apply to supervised models. In this paper, we propose Genos, a general in-network framework for unsupervised A-NIDS by rule extraction, which consists of a Model Compiler, a Model Interpreter, and a Model Debugger. Specifically, observing benign data are multimodal and usually located in multiple subspaces in the feature space, we utilize a divide-and-conquer approach for model-agnostic rule extraction. In the Model Compiler, we first propose a tree-based clustering algorithm to partition the feature space into subspaces, then design a decision boundary estimation mechanism to approximate the source model in each subspace. The Model Interpreter interprets predictions by important attributes to aid network operators in understanding the predictions. The Model Debugger conducts incremental updating to rectify errors by only fine-tuning rules on affected subspaces, thus reducing maintenance costs. We implement a prototype using physical hardware, and experiments demonstrate its superior performance of 100 Gbps throughput, great interpretability, and trivial updating overhead., Comment: accepted by IEEE International Conference on Computer Communications (INFOCOM 2024)

Open-vocabulary 3D scene understanding presents a significant challenge in computer vision, withwide-ranging applications in embodied agents and augmented reality systems. Previous approaches haveadopted Neural Radiance Fields (NeRFs) to analyze 3D scenes. In this paper, we introduce SemanticGaussians, a novel open-vocabulary scene understanding approach based on 3D Gaussian Splatting. Our keyidea is distilling pre-trained 2D semantics into 3D Gaussians. We design a versatile projection approachthat maps various 2Dsemantic features from pre-trained image encoders into a novel semantic component of 3D Gaussians, withoutthe additional training required by NeRFs. We further build a 3D semantic network that directly predictsthe semantic component from raw 3D Gaussians for fast inference. We explore several applications ofSemantic Gaussians: semantic segmentation on ScanNet-20, where our approach attains a 4.2% mIoU and 4.0%mAcc improvement over prior open-vocabulary scene understanding counterparts; object part segmentation,sceneediting, and spatial-temporal segmentation with better qualitative results over 2D and 3D baselines,highlighting its versatility and effectiveness on supporting diverse downstream tasks., Comment: Project page: see https://semantic-gaussians.github.io

This paper presents a pilot study that explores the application of active learning, traditionally studied in the context of discriminative models, to generative models. We specifically focus on image synthesis personalization tasks. The primary challenge in conducting active learning on generative models lies in the open-ended nature of querying, which differs from the closed form of querying in discriminative models that typically target a single concept. We introduce the concept of anchor directions to transform the querying process into a semi-open problem. We propose a direction-based uncertainty sampling strategy to enable generative active learning and tackle the exploitation-exploration dilemma. Extensive experiments are conducted to validate the effectiveness of our approach, demonstrating that an open-source model can achieve superior performance compared to closed-source models developed by large companies, such as Google's StyleDrop. The source code is available at https://github.com/zhangxulu1996/GAL4Personalization.

This paper presents a pilot study aimed at introducing multi-agent debate into multimodal reasoning. The study addresses two key challenges: the trivialization of opinions resulting from excessive summarization and the diversion of focus caused by distractor concepts introduced from images. These challenges stem from the inductive (bottom-up) nature of existing debating schemes. To address the issue, we propose a deductive (top-down) debating approach called Blueprint Debate on Graphs (BDoG). In BDoG, debates are confined to a blueprint graph to prevent opinion trivialization through world-level summarization. Moreover, by storing evidence in branches within the graph, BDoG mitigates distractions caused by frequent but irrelevant concepts. Extensive experiments validate BDoG, achieving state-of-the-art results in Science QA and MMBench with significant improvements over previous methods., Comment: Work in progress

Neuro-symbolic reinforcement learning (NS-RL) has emerged as a promising paradigm for explainable decision-making, characterized by the interpretability of symbolic policies. NS-RL entails structured state representations for tasks with visual observations, but previous methods are unable to refine the structured states with rewards due to a lack of efficiency. Accessibility also remains to be an issue, as extensive domain knowledge is required to interpret symbolic policies. In this paper, we present a framework for learning structured states and symbolic policies jointly, whose key idea is to distill vision foundation models into a scalable perception module and refine it during policy learning. Moreover, we design a pipeline to generate language explanations for policies and decisions using large language models. In experiments on nine Atari tasks, we verify the efficacy of our approach, and we also present explanations for policies and decisions., Comment: ICML 2024. Project page: https://ins-rl.github.io

We explore how reconciling several foundation models (large language models and vision-language models) with a novel unified memory mechanism could tackle the challenging video understanding problem, especially capturing the long-term temporal relations in lengthy videos. In particular, the proposed multimodal agent VideoAgent: 1) constructs a structured memory to store both the generic temporal event descriptions and object-centric tracking states of the video; 2) given an input task query, it employs tools including video segment localization and object memory querying along with other visual foundation models to interactively solve the task, utilizing the zero-shot tool-use ability of LLMs. VideoAgent demonstrates impressive performances on several long-horizon video understanding benchmarks, an average increase of 6.6% on NExT-QA and 26.0% on EgoSchema over baselines, closing the gap between open-sourced models and private counterparts including Gemini 1.5 Pro., Comment: Project page: videoagent.github.io; First two authors contributed equally

Fashion analysis refers to the process of examining and evaluating trends, styles, and elements within the fashion industry to understand and interpret its current state, generating fashion reports. It is traditionally performed by fashion professionals based on their expertise and experience, which requires high labour cost and may also produce biased results for relying heavily on a small group of people. In this paper, to tackle the Fashion Report Generation (FashionReGen) task, we propose an intelligent Fashion Analyzing and Reporting system based the advanced Large Language Models (LLMs), debbed as GPT-FAR. Specifically, it tries to deliver FashionReGen based on effective catwalk analysis, which is equipped with several key procedures, namely, catwalk understanding, collective organization and analysis, and report generation. By posing and exploring such an open-ended, complex and domain-specific task of FashionReGen, it is able to test the general capability of LLMs in fashion domain. It also inspires the explorations of more high-level tasks with industrial significance in other domains. Video illustration and more materials of GPT-FAR can be found in https://github.com/CompFashion/FashionReGen.

Large Language Models (LLMs) have demonstrated exceptional performance in biochemical tasks, especially the molecule caption translation task, which aims to bridge the gap between molecules and natural language texts. However, previous methods in adapting LLMs to the molecule-caption translation task required extra domain-specific pre-training stages, suffered weak alignment between molecular and textual spaces, or imposed stringent demands on the scale of LLMs. To resolve the challenges, we propose In-Context Molecule Adaptation (ICMA), as a new paradigm allowing LLMs to learn the molecule-text alignment from context examples via In-Context Molecule Tuning. Specifically, ICMA incorporates the following three stages: Hybrid Context Retrieval, Post-retrieval Re-ranking, and In-context Molecule Tuning. Initially, Hybrid Context Retrieval utilizes BM25 Caption Retrieval and Molecule Graph Retrieval to retrieve informative context examples. Additionally, we also propose Post-retrieval Re-ranking with Sequence Reversal and Random Walk to further improve the quality of retrieval results. Finally, In-Context Molecule Tuning unlocks the in-context molecule learning capability of LLMs with retrieved examples and adapts the parameters of LLMs for the molecule-caption translation task. Experimental results demonstrate that ICMT can empower LLMs to achieve state-of-the-art or comparable performance without extra training corpora and intricate structures, showing that LLMs are inherently in-context molecule learners.

Non-fullerene acceptors (NFAs) have recently emerged as pivotal materials for enhancing the efficiency of organic solar cells (OSCs). To further advance OSC efficiency, precise control over the energy levels of NFAs is imperative, necessitating the development of a robust computational method for accurate energy level predictions. Unfortunately, conventional computational techniques often yield relatively large errors, typically ranging from 0.2 to 0.5 electronvolts (eV), when predicting energy levels. In this study, the authors present a novel method that not only expedites energy level predictions but also significantly improves accuracy , reducing the error margin to 0.06 eV. The method comprises two essential components. The first component involves data cleansing, which systematically eliminates problematic experimental data and thereby minimizes input data errors. The second component introduces a molecular description method based on the electronic properties of the sub-units comprising NFAs. The approach simplifies the intricacies of molecular computation and demonstrates markedly enhanced prediction performance compared to the conventional density functional theory (DFT) method. Our methodology will expedite research in the field of NFAs, serving as a catalyst for the development of similar computational approaches to address challenges in other areas of material science and molecular research. An accurate and rapid computational method is developed to predict the energy levels of nonfullerene acceptors, yielding smaller prediction errors than previous computation methods. The accurate prediction is enabled by the combination of a data-cleansing protocol that can effectively eliminate problematic experimental and a sub-unit-based molecular description method that greatly simplifies the complexity of molecular representations and hence the relevant molecular computation. image

Vehicle-to-everything (V2X) is a popular topic in the field of Autonomous Driving in recent years. Vehicle-infrastructure cooperation (VIC) becomes one of the important research area. Due to the complexity of traffic conditions such as blind spots and occlusion, it greatly limits the perception capabilities of single-view roadside sensing systems. To further enhance the accuracy of roadside perception and provide better information to the vehicle side, in this paper, we constructed holographic intersections with various layouts to build a large-scale multi-sensor holographic vehicle-infrastructure cooperation dataset, called HoloVIC. Our dataset includes 3 different types of sensors (Camera, Lidar, Fisheye) and employs 4 sensor-layouts based on the different intersections. Each intersection is equipped with 6-18 sensors to capture synchronous data. While autonomous vehicles pass through these intersections for collecting VIC data. HoloVIC contains in total on 100k+ synchronous frames from different sensors. Additionally, we annotated 3D bounding boxes based on Camera, Fisheye, and Lidar. We also associate the IDs of the same objects across different devices and consecutive frames in sequence. Based on HoloVIC, we formulated four tasks to facilitate the development of related research. We also provide benchmarks for these tasks., Comment: Accept to CVPR 2024, Benchmark Website: https://holovic.net

Existing galvanometer-based laser scanning systems are challenging to apply in multi-scale 3D reconstruction because of the difficulty in achieving a balance between high reconstruction accuracy and a wide reconstruction range. This paper presents a novel method that synchronizes laser scanning by switching the field-of-view (FOV) of a camera using multi-galvanometers. In addition to the advanced hardware setup, we establish a comprehensive mathematical model of the system by modeling dynamic camera, dynamic laser, and their combined interaction. We then propose a high-precision and flexible calibration method by constructing an error model and minimizing the objective function. Finally, we evaluate the performance of the proposed system by scanning standard components. The evaluation results demonstrate that the accuracy of the proposed 3D reconstruction system achieves 0.3 mm when the measurement range is extended to 1100 mm $\times$ 1300 mm $\times$ 650 mm. With the same reconstruction accuracy, the reconstruction range is expanded by a factor of 25, indicating that the proposed method simultaneously allows for high-precision and wide-range 3D reconstruction in industrial applications., Comment: 9 pages,6 figures, Journal

In an era of information explosion, recommender systems are vital tools to deliver personalized recommendations for users. The key of recommender systems is to forecast users' future behaviors based on previous user-item interactions. Due to their strong expressive power of capturing high-order connectivities in user-item interaction data, recent years have witnessed a rising interest in leveraging Graph Neural Networks (GNNs) to boost the prediction performance of recommender systems. Nonetheless, classic Matrix Factorization (MF) and Deep Neural Network (DNN) approaches still play an important role in real-world large-scale recommender systems due to their scalability advantages. Despite the existence of GNN-acceleration solutions, it remains an open question whether GNN-based recommender systems can scale as efficiently as classic MF and DNN methods. In this paper, we propose a Linear-Time Graph Neural Network (LTGNN) to scale up GNN-based recommender systems to achieve comparable scalability as classic MF approaches while maintaining GNNs' powerful expressiveness for superior prediction accuracy. Extensive experiments and ablation studies are presented to validate the effectiveness and scalability of the proposed algorithm. Our implementation based on PyTorch is available., Comment: 12 pages, 5 figures, accepted by The Web Conference 2024

High-quality sentence embeddings are fundamental in many natural language processing (NLP) tasks, such as semantic textual similarity (STS) and retrieval-augmented generation (RAG). Nevertheless, most existing methods leverage fixed-length embeddings from full-layer language models, which lack the scalability to accommodate the diverse available resources across various applications. Viewing this gap, we propose a novel sentence embedding model $\mathrm{Espresso}$ $\mathrm{Sentence}$ $\mathrm{Embeddings}$ (ESE) with two learning processes. First, the learn-to-express process encodes more salient representations to lower layers. Second, the learn-to-compress process compacts essential features into the initial dimensions using Principal Component Analysis (PCA). This way, ESE can scale model depth via the former process and embedding size via the latter. Extensive experiments on STS and RAG suggest that ESE can effectively produce high-quality embeddings with less model depth and embedding size, enhancing embedding inference efficiency.

As network security receives widespread attention, encrypted traffic classification has become the current research focus. However, existing methods conduct traffic classification without sufficiently considering the common characteristics between data samples, leading to suboptimal performance. Moreover, they train the packet-level and flow-level classification tasks independently, which is redundant because the packet representations learned in the packet-level task can be exploited by the flow-level task. Therefore, in this paper, we propose an effective model named a Contrastive Learning Enhanced Temporal Fusion Encoder (CLE-TFE). In particular, we utilize supervised contrastive learning to enhance the packet-level and flow-level representations and perform graph data augmentation on the byte-level traffic graph so that the fine-grained semantic-invariant characteristics between bytes can be captured through contrastive learning. We also propose cross-level multi-task learning, which simultaneously accomplishes the packet-level and flow-level classification tasks in the same model with one training. Further experiments show that CLE-TFE achieves the best overall performance on the two tasks, while its computational overhead (i.e., floating point operations, FLOPs) is only about 1/14 of the pre-trained model (e.g., ET-BERT). We release the code at https://github.com/ViktorAxelsen/CLE-TFE, Comment: The code is available at https://github.com/ViktorAxelsen/CLE-TFE

Bioinformatics has witnessed a paradigm shift with the increasing integration of artificial intelligence (AI), particularly through the adoption of foundation models (FMs). These AI techniques have rapidly advanced, addressing historical challenges in bioinformatics such as the scarcity of annotated data and the presence of data noise. FMs are particularly adept at handling large-scale, unlabeled data, a common scenario in biological contexts due to the time-consuming and costly nature of experimentally determining labeled data. This characteristic has allowed FMs to excel and achieve notable results in various downstream validation tasks, demonstrating their ability to represent diverse biological entities effectively. Undoubtedly, FMs have ushered in a new era in computational biology, especially in the realm of deep learning. The primary goal of this survey is to conduct a systematic investigation and summary of FMs in bioinformatics, tracing their evolution, current research status, and the methodologies employed. Central to our focus is the application of FMs to specific biological problems, aiming to guide the research community in choosing appropriate FMs for their research needs. We delve into the specifics of the problem at hand including sequence analysis, structure prediction, function annotation, and multimodal integration, comparing the structures and advancements against traditional methods. Furthermore, the review analyses challenges and limitations faced by FMs in biology, such as data noise, model explainability, and potential biases. Finally, we outline potential development paths and strategies for FMs in future biological research, setting the stage for continued innovation and application in this rapidly evolving field. This comprehensive review serves not only as an academic resource but also as a roadmap for future explorations and applications of FMs in biology., Comment: 27 pages, 3 figures, 2 tables

The development of Neural Radiance Fields (NeRFs) has provided a potent representation for encapsulating the geometric and appearance characteristics of 3D scenes. Enhancing the capabilities of NeRFs in open-vocabulary 3D semantic perception tasks has been a recent focus. However, current methods that extract semantics directly from Contrastive Language-Image Pretraining (CLIP) for semantic field learning encounter difficulties due to noisy and view-inconsistent semantics provided by CLIP. To tackle these limitations, we propose OV-NeRF, which exploits the potential of pre-trained vision and language foundation models to enhance semantic field learning through proposed single-view and cross-view strategies. First, from the single-view perspective, we introduce Region Semantic Ranking (RSR) regularization by leveraging 2D mask proposals derived from SAM to rectify the noisy semantics of each training view, facilitating accurate semantic field learning. Second, from the cross-view perspective, we propose a Cross-view Self-enhancement (CSE) strategy to address the challenge raised by view-inconsistent semantics. Rather than invariably utilizing the 2D inconsistent semantics from CLIP, CSE leverages the 3D consistent semantics generated from the well-trained semantic field itself for semantic field training, aiming to reduce ambiguity and enhance overall semantic consistency across different views. Extensive experiments validate our OV-NeRF outperforms current state-of-the-art methods, achieving a significant improvement of 20.31% and 18.42% in mIoU metric on Replica and Scannet, respectively. Furthermore, our approach exhibits consistent superior results across various CLIP configurations, further verifying its robustness.

3D Gaussian Splatting has emerged as an alternative 3D representation for novel view synthesis, benefiting from its high-quality rendering results and real-time rendering speed. However, the 3D Gaussians learned by 3D-GS have ambiguous structures without any geometry constraints. This inherent issue in 3D-GS leads to a rough boundary when segmenting individual objects. To remedy these problems, we propose SAGD, a conceptually simple yet effective boundary-enhanced segmentation pipeline for 3D-GS to improve segmentation accuracy while preserving segmentation speed. Specifically, we introduce a Gaussian Decomposition scheme, which ingeniously utilizes the special structure of 3D Gaussian, finds out, and then decomposes the boundary Gaussians. Moreover, to achieve fast interactive 3D segmentation, we introduce a novel training-free pipeline by lifting a 2D foundation model to 3D-GS. Extensive experiments demonstrate that our approach achieves high-quality 3D segmentation without rough boundary issues, which can be easily applied to other scene editing tasks.

Large-scale pre-trained vision models (PVMs) have shown great potential for adaptability across various downstream vision tasks. However, with state-of-the-art PVMs growing to billions or even trillions of parameters, the standard full fine-tuning paradigm is becoming unsustainable due to high computational and storage demands. In response, researchers are exploring parameter-efficient fine-tuning (PEFT), which seeks to exceed the performance of full fine-tuning with minimal parameter modifications. This survey provides a comprehensive overview and future directions for visual PEFT, offering a systematic review of the latest advancements. First, we provide a formal definition of PEFT and discuss model pre-training methods. We then categorize existing methods into three categories: addition-based, partial-based, and unified-based. Finally, we introduce the commonly used datasets and applications and suggest potential future research challenges. A comprehensive collection of resources is available at https://github.com/synbol/Awesome-Parameter-Efficient-Transfer-Learning., Comment: 9 pages, 3 figures, 2 tables

Weight quantization is an effective technique to compress deep neural networks for their deployment on edge devices with limited resources. Traditional loss-aware quantization methods commonly use the quantized gradient to replace the full-precision gradient. However, we discover that the gradient error will lead to an unexpected zig-zagging-like issue in the gradient descent learning procedures, where the gradient directions rapidly oscillate or zig-zag, and such issue seriously slows down the model convergence. Accordingly, this paper proposes a one-step forward and backtrack way for loss-aware quantization to get more accurate and stable gradient direction to defy this issue. During the gradient descent learning, a one-step forward search is designed to find the trial gradient of the next-step, which is adopted to adjust the gradient of current step towards the direction of fast convergence. After that, we backtrack the current step to update the full-precision and quantized weights through the current-step gradient and the trial gradient. A series of theoretical analysis and experiments on benchmark deep models have demonstrated the effectiveness and competitiveness of the proposed method, and our method especially outperforms others on the convergence performance., Comment: 9 pages, 13 figures,accepted by AAAI-24

Simultaneously achieving 3D reconstruction and new view synthesis for indoor environments has widespread applications but is technically very challenging. State-of-the-art methods based on implicit neural functions can achieve excellent 3D reconstruction results, but their performances on new view synthesis can be unsatisfactory. The exciting development of neural radiance field (NeRF) has revolutionized new view synthesis, however, NeRF-based models can fail to reconstruct clean geometric surfaces. We have developed a dual neural radiance field (Du-NeRF) to simultaneously achieve high-quality geometry reconstruction and view rendering. Du-NeRF contains two geometric fields, one derived from the SDF field to facilitate geometric reconstruction and the other derived from the density field to boost new view synthesis. One of the innovative features of Du-NeRF is that it decouples a view-independent component from the density field and uses it as a label to supervise the learning process of the SDF field. This reduces shape-radiance ambiguity and enables geometry and color to benefit from each other during the learning process. Extensive experiments demonstrate that Du-NeRF can significantly improve the performance of novel view synthesis and 3D reconstruction for indoor environments and it is particularly effective in constructing areas containing fine geometries that do not obey multi-view color consistency., Comment: 20 pages, 8 figures

Despite much progress, achieving real-time high-fidelity head avatar animation is still difficult and existing methods have to trade-off between speed and quality. 3DMM based methods often fail to model non-facial structures such as eyeglasses and hairstyles, while neural implicit models suffer from deformation inflexibility and rendering inefficiency. Although 3D Gaussian has been demonstrated to possess promising capability for geometry representation and radiance field reconstruction, applying 3D Gaussian in head avatar creation remains a major challenge since it is difficult for 3D Gaussian to model the head shape variations caused by changing poses and expressions. In this paper, we introduce PSAvatar, a novel framework for animatable head avatar creation that utilizes discrete geometric primitive to create a parametric morphable shape model and employs 3D Gaussian for fine detail representation and high fidelity rendering. The parametric morphable shape model is a Point-based Morphable Shape Model (PMSM) which uses points instead of meshes for 3D representation to achieve enhanced representation flexibility. The PMSM first converts the FLAME mesh to points by sampling on the surfaces as well as off the meshes to enable the reconstruction of not only surface-like structures but also complex geometries such as eyeglasses and hairstyles. By aligning these points with the head shape in an analysis-by-synthesis manner, the PMSM makes it possible to utilize 3D Gaussian for fine detail representation and appearance modeling, thus enabling the creation of high-fidelity avatars. We show that PSAvatar can reconstruct high-fidelity head avatars of a variety of subjects and the avatars can be animated in real-time ($\ge$ 25 fps at a resolution of 512 $\times$ 512 )., Comment: 13 pages, 10 figures

3D vision-language grounding, which focuses on aligning language with the 3D physical environment, stands as a cornerstone in the development of embodied agents. In comparison to recent advancements in the 2D domain, grounding language in 3D scenes faces several significant challenges: (i) the inherent complexity of 3D scenes due to the diverse object configurations, their rich attributes, and intricate relationships; (ii) the scarcity of paired 3D vision-language data to support grounded learning; and (iii) the absence of a unified learning framework to distill knowledge from grounded 3D data. In this work, we aim to address these three major challenges in 3D vision-language by examining the potential of systematically upscaling 3D vision-language learning in indoor environments. We introduce the first million-scale 3D vision-language dataset, SceneVerse, encompassing about 68K 3D indoor scenes and comprising 2.5M vision-language pairs derived from both human annotations and our scalable scene-graph-based generation approach. We demonstrate that this scaling allows for a unified pre-training framework, Grounded Pre-training for Scenes (GPS), for 3D vision-language learning. Through extensive experiments, we showcase the effectiveness of GPS by achieving state-of-the-art performance on all existing 3D visual grounding benchmarks. The vast potential of SceneVerse and GPS is unveiled through zero-shot transfer experiments in the challenging 3D vision-language tasks. Project website: https://scene-verse.github.io., Comment: 21 pages

ChatGPT explores a strategic blueprint of question answering (QA) in delivering medical diagnosis, treatment recommendations, and other healthcare support. This is achieved through the increasing incorporation of medical domain data via natural language processing (NLP) and multimodal paradigms. By transitioning the distribution of text, images, videos, and other modalities from the general domain to the medical domain, these techniques have expedited the progress of medical domain question answering (MDQA). They bridge the gap between human natural language and sophisticated medical domain knowledge or expert manual annotations, handling large-scale, diverse, unbalanced, or even unlabeled data analysis scenarios in medical contexts. Central to our focus is the utilizing of language models and multimodal paradigms for medical question answering, aiming to guide the research community in selecting appropriate mechanisms for their specific medical research requirements. Specialized tasks such as unimodal-related question answering, reading comprehension, reasoning, diagnosis, relation extraction, probability modeling, and others, as well as multimodal-related tasks like vision question answering, image caption, cross-modal retrieval, report summarization, and generation, are discussed in detail. Each section delves into the intricate specifics of the respective method under consideration. This paper highlights the structures and advancements of medical domain explorations against general domain methods, emphasizing their applications across different tasks and datasets. It also outlines current challenges and opportunities for future medical domain research, paving the way for continued innovation and application in this rapidly evolving field., Comment: 50 pages, 3 figures, 3 tables

Finetuning large language models (LLMs) has been empirically effective on a variety of downstream tasks. Existing approaches to finetuning an LLM either focus on parameter-efficient finetuning, which only updates a small number of trainable parameters, or attempt to reduce the memory footprint during the training phase of the finetuning. Typically, the memory footprint during finetuning stems from three contributors: model weights, optimizer states, and intermediate activations. However, existing works still require considerable memory and none can simultaneously mitigate memory footprint for all three sources. In this paper, we present Quantized Side Tuing (QST), which enables memory-efficient and fast finetuning of LLMs by operating through a dual-stage process. First, QST quantizes an LLM's model weights into 4-bit to reduce the memory footprint of the LLM's original weights; QST also introduces a side network separated from the LLM, which utilizes the hidden states of the LLM to make task-specific predictions. Using a separate side network avoids performing backpropagation through the LLM, thus reducing the memory requirement of the intermediate activations. Furthermore, QST leverages several low-rank adaptors and gradient-free downsample modules to significantly reduce the trainable parameters, so as to save the memory footprint of the optimizer states. Experiments show that QST can reduce the total memory footprint by up to 2.3 $\times$ and speed up the finetuning process by up to 3 $\times$ while achieving competent performance compared with the state-of-the-art. When it comes to full finetuning, QST can reduce the total memory footprint up to 7 $\times$.

3D landmark detection plays a pivotal role in various applications such as 3D registration, pose estimation, and virtual try-on. While considerable success has been achieved in 2D human landmark detection or pose estimation, there is a notable scarcity of reported works on landmark detection in unordered 3D point clouds. This paper introduces a novel challenge, namely 3D landmark detection on human point clouds, presenting two primary contributions. Firstly, we establish a comprehensive human point cloud dataset, named HPoint103, designed to support the 3D landmark detection community. This dataset comprises 103 human point clouds created with commercial software and actors, each manually annotated with 11 stable landmarks. Secondly, we propose a Dual Cascade Point Transformer (D-CPT) model for precise point-based landmark detection. D-CPT gradually refines the landmarks through cascade Transformer decoder layers across the entire point cloud stream, simultaneously enhancing landmark coordinates with a RefineNet over local regions. Comparative evaluations with popular point-based methods on HPoint103 and the public dataset DHP19 demonstrate the dramatic outperformance of our D-CPT. Additionally, the integration of our RefineNet into existing methods consistently improves performance.

In the wake of the rapid deployment of large-scale low-Earth orbit satellite constellations, exploiting the full computing potential of Commercial Off-The-Shelf (COTS) devices in these environments has become a pressing issue. However, understanding this problem is far from straightforward due to the inherent differences between the terrestrial infrastructure and the satellite platform in space. In this paper, we take an important step towards closing this knowledge gap by presenting the first measurement study on the thermal control, power management, and performance of COTS computing devices on satellites. Our measurements reveal that the satellite platform and COTS computing devices significantly interplay in terms of the temperature and energy, forming the main constraints on satellite computing. Further, we analyze the critical factors that shape the characteristics of onboard COTS computing devices. We provide guidelines for future research on optimizing the use of such devices for computing purposes. Finally, we have released the datasets to facilitate further study in satellite computing.

Forecasting the trend of stock prices is an enduring topic at the intersection of finance and computer science. Periodical updates to forecasters have proven effective in handling concept drifts arising from non-stationary markets. However, the existing methods neglect either emerging patterns in recent data or recurring patterns in historical data, both of which are empirically advantageous for future forecasting. To address this issue, we propose meta-learning with dynamic adaptation (MetaDA) for the incremental learning of stock trends, which periodically performs dynamic model adaptation utilizing the emerging and recurring patterns simultaneously. We initially organize the stock trend forecasting into meta-learning tasks and train a forecasting model following meta-learning protocols. During model adaptation, MetaDA efficiently adapts the forecasting model with the latest data and a selected portion of historical data, which is dynamically identified by a task inference module. The task inference module first extracts task-level embeddings from the historical tasks, and then identifies the informative data with a task inference network. MetaDA has been evaluated on real-world stock datasets, achieving state-of-the-art performance with satisfactory efficiency.

Huanhuan Zhu, Chen Chen, Luhan Geng, Qing Li, Chengning Zhang, Lin Wu, Bo Zhang, Suyan Duan,&ast; Changying Xing,&ast; Yanggang Yuan&ast; Department of Nephrology, the First Affiliated Hospital of Nanjing Medical University, Nanjing Medical University, Nanjing, People’s Republic of China&ast;These authors contributed equally to this workCorrespondence: Yanggang Yuan, Department of Nephrology, the First Affiliated Hospital of Nanjing Medical University, Nanjing Medical University, 300 Guangzhou Road, Nanjing, Jiangsu Province, 210029, People’s Republic of China, Tel/Fax +86-25-6813-6462, Email ygyuan@njmu.edu.cnBackground: Mounting evidence suggests that mitochondrial dysfunction contributes to lupus nephritis (LN) pathogenesis. Mitochondrial pyruvate carrier 1 (MPC1) and mitochondrial pyruvate carrier 2 (MPC2) mediating pyruvate transport from the cytoplasm to the mitochondrial matrix, determines the cell survival and cellular energy supply. Here, we aimed to investigate the association of mitochondrial pyruvate carrier expression with the clinical and histological features in LN.Methods: Patients with biopsy-proven proliferative LN (class III and class IV, n=18) and membranous LN (class V, n=18) were included. Expression of MPC1 and MPC2 were examined by immunohistochemistry. MPC protein levels in the two groups were evaluated by the Student’s t-test. Correlation analysis between MPC levels and clinicopathological features was performed by Spearman’s rank correlation.Results: Both MPC1 and MPC2 were exclusively expressed in renal tubules of enrolled LN. Significantly lower MPC1 and MPC2 were observed in patients with proliferative LN compared to membranous LN. In addition, the MPC1 and MPC2 were negatively correlated with SLEDAI-2K score, renal function, and renal pathology activity index.Conclusion: Both MPC1 and MPC2 were localized in renal tubules, and decreased MPC content was more pronounced in proliferative LN than membranous LN. MPC l

Single image reflection removal is inherently ambiguous, as both the reflection and transmission components requiring separation may follow natural image statistics. Existing methods attempt to address the issue by using various types of low-level and physics-based cues as sources of reflection signals. However, these cues are not universally applicable, since they are only observable in specific capture scenarios. This leads to a significant performance drop when test images do not align with their assumptions. In this paper, we aim to explore a novel flexible interactive reflection removal approach that leverages various forms of sparse human guidance, such as points and bounding boxes, as auxiliary high-level prior to achieve robust reflection removal. However, incorporating the raw user guidance naively into the existing reflection removal network does not result in performance gains. To this end, we innovatively transform raw user input into a unified form -- reflection masks using an Interactive Segmentation Foundation Model. Such a design absorbs the quintessence of the foundational segmentation model and flexible human guidance, thereby mitigating the challenges of reflection separations. Furthermore, to fully utilize user guidance and reduce user annotation costs, we design a mask-guided reflection removal network, comprising our proposed self-adaptive prompt block. This block adaptively incorporates user guidance as anchors and refines transmission features via cross-attention mechanisms. Extensive results on real-world images validate that our method demonstrates state-of-the-art performance on various datasets with the help of flexible and sparse user guidance. Our code and dataset will be publicly available here https://github.com/ShawnChenn/FlexibleReflectionRemoval.

Multivariate time series prediction is widely used in daily life, which poses significant challenges due to the complex correlations that exist at multi-grained levels. Unfortunately, the majority of current time series prediction models fail to simultaneously learn the correlations of multivariate time series at multi-grained levels, resulting in suboptimal performance. To address this, we propose a Multi-Grained Correlations-based Prediction (MGCP) Network, which simultaneously considers the correlations at three granularity levels to enhance prediction performance. Specifically, MGCP utilizes Adaptive Fourier Neural Operators and Graph Convolutional Networks to learn the global spatiotemporal correlations and inter-series correlations, enabling the extraction of potential features from multivariate time series at fine-grained and medium-grained levels. Additionally, MGCP employs adversarial training with an attention mechanism-based predictor and conditional discriminator to optimize prediction results at coarse-grained level, ensuring high fidelity between the generated forecast results and the actual data distribution. Finally, we compare MGCP with several state-of-the-art time series prediction algorithms on real-world benchmark datasets, and our results demonstrate the generality and effectiveness of the proposed model.

Large language models (LLMs) can elicit social bias during generations, especially when inference with toxic prompts. Controlling the sensitive attributes in generation encounters challenges in data distribution, generalizability, and efficiency. Specifically, fine-tuning and retrieval demand extensive unbiased corpus, while direct prompting requires meticulously curated instructions for correcting the output in multiple rounds of thoughts but poses challenges on memory and inference latency. In this work, we propose the Expert-Guided Extinction of Toxic Tokens for Debiased Generation (EXPOSED) to eliminate the undesired harmful outputs for LLMs without the aforementioned requirements. EXPOSED constructs a debiasing expert based on the abundant toxic corpus to expose and elicit the potentially dangerous tokens. It then processes the output to the LLMs and constructs a fair distribution by suppressing and attenuating the toxic tokens. EXPOSED is evaluated on fairness benchmarks over three LLM families. Extensive experiments demonstrate that compared with other baselines, the proposed EXPOSED significantly reduces the potential social bias while balancing fairness and generation performance.

Entity alignment (EA) aims to merge two knowledge graphs (KGs) by identifying equivalent entity pairs. While existing methods heavily rely on human-generated labels, it is prohibitively expensive to incorporate cross-domain experts for annotation in real-world scenarios. The advent of Large Language Models (LLMs) presents new avenues for automating EA with annotations, inspired by their comprehensive capability to process semantic information. However, it is nontrivial to directly apply LLMs for EA since the annotation space in real-world KGs is large. LLMs could also generate noisy labels that may mislead the alignment. To this end, we propose a unified framework, LLM4EA, to effectively leverage LLMs for EA. Specifically, we design a novel active learning policy to significantly reduce the annotation space by prioritizing the most valuable entities based on the entire inter-KG and intra-KG structure. Moreover, we introduce an unsupervised label refiner to continuously enhance label accuracy through in-depth probabilistic reasoning. We iteratively optimize the policy based on the feedback from a base EA model. Extensive experiments demonstrate the advantages of LLM4EA on four benchmark datasets in terms of effectiveness, robustness, and efficiency. Codes are available via https://github.com/chensyCN/llm4ea_official.

A unified model for 3D vision-language (3D-VL) understanding is expected to take various scene representations and perform a wide range of tasks in a 3D scene. However, a considerable gap exists between existing methods and such a unified model, due to the independent application of representation and insufficient exploration of 3D multi-task training. In this paper, we introduce PQ3D, a unified model capable of using Promptable Queries to tackle a wide range of 3D-VL tasks, from low-level instance segmentation to high-level reasoning and planning. This is achieved through three key innovations: (1) unifying various 3D scene representations (i.e., voxels, point clouds, multi-view images) into a shared 3D coordinate space by segment-level grouping, (2) an attention-based query decoder for task-specific information retrieval guided by prompts, and (3) universal output heads for different tasks to support multi-task training. Tested across ten diverse 3D-VL datasets, PQ3D demonstrates impressive performance on these tasks, setting new records on most benchmarks. Particularly, PQ3D improves the state-of-the-art on ScanNet200 by 1.8% (AP), ScanRefer by 5.4% (acc@0.5), Multi3DRefer by 11.7% (F1@0.5), and Scan2Cap by 13.4% (CIDEr@0.5). Moreover, PQ3D supports flexible inference with individual or combined forms of available 3D representations, e.g., solely voxel input., Comment: Project page: https://pq3d.github.io

This paper investigates the dynamics of a deep neural network (DNN) learning interactions. Previous studies have discovered and mathematically proven that given each input sample, a well-trained DNN usually only encodes a small number of interactions (non-linear relationships) between input variables in the sample. A series of theorems have been derived to prove that we can consider the DNN's inference equivalent to using these interactions as primitive patterns for inference. In this paper, we discover the DNN learns interactions in two phases. The first phase mainly penalizes interactions of medium and high orders, and the second phase mainly learns interactions of gradually increasing orders. We can consider the two-phase phenomenon as the starting point of a DNN learning over-fitted features. Such a phenomenon has been widely shared by DNNs with various architectures trained for different tasks. Therefore, the discovery of the two-phase dynamics provides a detailed mechanism for how a DNN gradually learns different inference patterns (interactions). In particular, we have also verified the claim that high-order interactions have weaker generalization power than low-order interactions. Thus, the discovered two-phase dynamics also explains how the generalization power of a DNN changes during the training process.

Although large language models (LLMs) have achieved significant success, their vulnerability to adversarial perturbations, including recent jailbreak attacks, has raised considerable concerns. However, the increasing size of these models and their limited access make improving their robustness a challenging task. Among various defense strategies, randomized smoothing has shown great potential for LLMs, as it does not require full access to the model's parameters or fine-tuning via adversarial training. However, randomized smoothing involves adding noise to the input before model prediction, and the final model's robustness largely depends on the model's performance on these noise corrupted data. Its effectiveness is often limited by the model's sub-optimal performance on noisy data. To address this issue, we propose to leverage the multitasking nature of LLMs to first denoise the noisy inputs and then to make predictions based on these denoised versions. We call this procedure self-denoised smoothing. Unlike previous denoised smoothing techniques in computer vision, which require training a separate model to enhance the robustness of LLMs, our method offers significantly better efficiency and flexibility. Our experimental results indicate that our method surpasses existing methods in both empirical and certified robustness in defending against adversarial attacks for both downstream tasks and human alignments (i.e., jailbreak attacks). Our code is publicly available at https://github.com/UCSB-NLP-Chang/SelfDenoise, Comment: Accepted by NAACL 2024. Jiabao, Bairu, Zhen, Guanhua contributed equally. This is an updated version of the paper: arXiv:2307.07171