Search

The operation of distribution networks is currently facing potential safety and quality defects that pose significant hazards. One solution to strengthen management, reduce manual workload, and improve efficiency and quality is by applying deep detection networks for dynamic defect detection in distribution network engineering. To start, defects in distribution network engineering are classified. Then, advanced deep detection networks and their applications in dynamic defect detection are researched and analyzed, along with a review of existing research. Key issues and their solutions for deep detection network application in dynamic defect detection in distribution network engineering are summarized. Finally, future research directions are explored to provide valuable references for future studies.

Aims: To develop a model to predict the risk of moderate to severe cancer-related fatigue (CRF) in colorectal cancer patients after chemotherapy. Methods: The study population was colorectal cancer patients who received chemotherapy from September 2021 to June 2022 in a grade 3 and first-class hospital. Demographic, clinical, physiological, psychological, and socioeconomic factors were collected 1 to 2 days before chemotherapy. Patients were followed for 1 to 2 days after chemotherapy to assess fatigue using the Piper Fatigue Scale. A random sampling method was used to select 181 patients with moderate to severe CRF as the case group. The risk set sampling method was used to select 181 patients with mild or no CRF as the control group. Logistic regression, back-propagation artificial neural network (BP-ANN) and decision tree models were constructed and compared. Results: A total of 362 patients consisting of 241 derivation samples and 121 validation samples were enrolled. Comparing the three models, the prediction effect of BP-ANN was the best, with a receiver operating characteristic curve (ROC) of 0.83. Internal and external verification indicated the accuracy of prediction was 70.4% and 80.8%, respectively. Significant predictors identified were surgery, complications, hypokalaemia, albumin, neutrophil percentage, pain (VAS score), Activities of Daily Living (ADL) score, sleep quality (PSQI score), anxiety (HAD-A score), depression (HAD-D score) and nutrition (PG-SGA score). Conclusions: BP-ANN was the best model, offering theoretical guidance for clinicians to formulate a tool to identify patients at high risk of moderate to severe CRF. Impact： · A prediction model can be developed to predict the risk of moderate to severe cancer-related fatigue in colorectal cancer patients after chemotherapy. · The BP-ANN model offers theoretical guidance for a clinically predictable tool to assist nurses in identifying and supporting patients at high risk of moderate to severe CRF. · There are 11 risk factors for moderate to severe CRF in patients with colorectal cancer after chemotherapy, and the BP-ANN is the best prediction model with strong predictive performance.

The replacement of small cations with bulkier organic cations containing long alkyl chains or benzene rings to form a thin two-dimensional (2D) perovskite passivation layer on three-dimensional (3D) perovskite (2D/3D) has become a promising strategy for improving both the efficiency and stability of perovskite solar cells (PSCs). The 2D layer defines the interfacial chemistry and physics at the 2D/3D bilayer and endows the 2D/3D structure with better chemical and thermal stability. Herein, 2D/3D (CF3-PEA)2FA0.85MA0.15Pb2I7/FA0.85MA0.15PbI3 planar heterojunction perovskite was produced using a facile interfacial ion exchange process. The 2D (CF3-PEA)2FA0.85MA0.15Pb2I7 capping layer can not only passivate the FA0.85MA0.15PbI3 film but also act as super-hydrophobic layer to inhibit water diffusion and significantly enhance the stability. The 2D capping layer can also establish a unique graded band structure at the perovskite/Spiro-OMeTAD interface and lead to p-type doping for Spiro-OMeTAD layer which is beneficial for efficient charge transport. Optimized PSCs based on this 2D/3D heterojunction yield a champion power conversion efficiency (PCE) of 23.1% and improved stability. The device maintains 84% output for 2400 h aging under ambient environmental conditions without encapsulation, and maintains 81% for 200 h under illumination with encapsulation. This work will inspire the design of more fluorinated 2D perovskite interfaces for advanced photovoltaics and beyond.

Transfer-based attacks generate adversarial examples on the surrogate model, which can mislead other black-box models without any access, making it promising to attack real-world applications. Recently, several works have been proposed to boost adversarial transferability, in which the surrogate model is usually overlooked. In this work, we identify that non-linear layers (e.g., ReLU, max-pooling, etc.) truncate the gradient during backward propagation, making the gradient w.r.t.input image imprecise to the loss function. We hypothesize and empirically validate that such truncation undermines the transferability of adversarial examples. Based on these findings, we propose a novel method called Backward Propagation Attack (BPA) to increase the relevance between the gradient w.r.t. input image and loss function so as to generate adversarial examples with higher transferability. Specifically, BPA adopts a non-monotonic function as the derivative of ReLU and incorporates softmax with temperature to smooth the derivative of max-pooling, thereby mitigating the information loss during the backward propagation of gradients. Empirical results on the ImageNet dataset demonstrate that not only does our method substantially boost the adversarial transferability, but it also is general to existing transfer-based attacks., Comment: 14 pages

Recent deep learning models have attracted substantial attention in infant brain analysis. These models have performed state-of-the-art performance, such as semi-supervised techniques (e.g., Temporal Ensembling, mean teacher). However, these models depend on an encoder-decoder structure with stacked local operators to gather long-range information, and the local operators limit the efficiency and effectiveness. Besides, the $MRI$ data contain different tissue properties ($TPs$) such as $T1$ and $T2$. One major limitation of these models is that they use both data as inputs to the segment process, i.e., the models are trained on the dataset once, and it requires much computational and memory requirements during inference. In this work, we address the above limitations by designing a new deep-learning model, called 3D-DenseUNet, which works as adaptable global aggregation blocks in down-sampling to solve the issue of spatial information loss. The self-attention module connects the down-sampling blocks to up-sampling blocks, and integrates the feature maps in three dimensions of spatial and channel, effectively improving the representation potential and discriminating ability of the model. Additionally, we propose a new method called Two Independent Teachers ($2IT$), that summarizes the model weights instead of label predictions. Each teacher model is trained on different types of brain data, $T1$ and $T2$, respectively. Then, a fuse model is added to improve test accuracy and enable training with fewer parameters and labels compared to the Temporal Ensembling method without modifying the network architecture. Empirical results demonstrate the effectiveness of the proposed method., Comment: This manuscript contains 14 pages, 7 figures. We have submitted the manuscript to Journal of IEEE Transactions on Medical Imaging (TMI) in June 2023

Meta-structures are widely used to define which subset of neighbors to aggregate information in heterogeneous information networks (HINs). In this work, we investigate existing meta-structures, including meta-path and meta-graph, and observe that they are initially designed manually with fixed patterns and hence are insufficient to encode various rich semantic information on diverse HINs. Through reflection on their limitation, we define a new concept called meta-multigraph as a more expressive and flexible generalization of meta-graph, and propose a stable differentiable search method to automatically optimize the meta-multigraph for specific HINs and tasks. As the flexibility of meta-multigraphs may propagate redundant messages, we further introduce a complex-to-concise (C2C) meta-multigraph that propagates messages from complex to concise along the depth of meta-multigraph. Moreover, we observe that the differentiable search typically suffers from unstable search and a significant gap between the meta-structures in search and evaluation. To this end, we propose a progressive search algorithm by implicitly narrowing the search space to improve search stability and reduce inconsistency. Extensive experiments are conducted on six medium-scale benchmark datasets and one large-scale benchmark dataset over two representative tasks, i.e., node classification and recommendation. Empirical results demonstrate that our search methods can automatically find expressive meta-multigraphs and C2C meta-multigraphs, enabling our model to outperform state-of-the-art heterogeneous graph neural networks., Comment: 17 pages, 10 figures. arXiv admin note: text overlap with arXiv:2211.14752

Pose estimation of the human body and hands is a fundamental problem in computer vision, and learning-based solutions require a large amount of annotated data. In this work, we improve the efficiency of the data annotation process for 3D pose estimation problems with Active Learning (AL) in a multi-view setting. AL selects examples with the highest value to annotate under limited annotation budgets (time and cost), but choosing the selection strategy is often nontrivial. We present a framework to efficiently extend existing single-view AL strategies. We then propose two novel AL strategies that make full use of multi-view geometry. Moreover, we demonstrate additional performance gains by incorporating pseudo-labels computed during the AL process, which is a form of self-training. Our system significantly outperforms simulated annotation baselines in 3D body and hand pose estimation on two large-scale benchmarks: CMU Panoptic Studio and InterHand2.6M. Notably, on CMU Panoptic Studio, we are able to reduce the turn-around time by 60% and annotation cost by 80% when compared to the conventional annotation process., IEEE WACV 2023 algorithms track. Code: https://github.com/facebookresearch/multi_view_active_learning

As a relaxation of the clique, a k-plex of a graph is a vertex set that each vertex is not connected with at most k vertices of this set. Given an undirected graph, the Maximum k-plex Problem (MkP) aims to find its largest k-plex. Branch and bound algorithms are a type of well-studied and effective method for exact MkP solving, whose performance depends heavily on the quality of the upper bounds. In this paper, we investigate the relaxation properties of k-plex and propose an effective upper bound called Relaxed Graph color Bound (RGB) for the MkP. To describe and calculate RGB, we propose a new quasi-independent set structure that focuses on the number of conflict vertices. We combine RGB with two of the state-of-the-art branch and bound MkP algorithms, Maplex and KpLeX. Extensive experiments on real-world benchmarks, DIMACS benchmarks, and random graphs show the excellent performance of our proposed method over the state-of-the-art algorithms.

Knowledge distillation is a powerful technique for transferring knowledge from a pre-trained teacher model to a student model. However, the true potential of knowledge transfer has not been fully explored. Existing approaches primarily focus on distilling individual information or instance-level relationships, overlooking the valuable information embedded in token-level relationships, which may be particularly affected by the long-tail effects. To address the above limitations, we propose a novel method called Knowledge Distillation with Token-level Relationship Graph (TRG) that leverages the token-wise relational knowledge to enhance the performance of knowledge distillation. By employing TRG, the student model can effectively emulate higher-level semantic information from the teacher model, resulting in improved distillation results. To further enhance the learning process, we introduce a token-wise contextual loss called contextual loss, which encourages the student model to capture the inner-instance semantic contextual of the teacher model. We conduct experiments to evaluate the effectiveness of the proposed method against several state-of-the-art approaches. Empirical results demonstrate the superiority of TRG across various visual classification tasks, including those involving imbalanced data. Our method consistently outperforms the existing baselines, establishing a new state-of-the-art performance in the field of knowledge distillation.

Existing heterogeneous graph neural networks (HGNNs) have achieved great success in utilizing the rich semantic information in heterogeneous information networks (HINs). However, few works have delved into the utilization of long-range dependencies in HINs, which is extremely valuable as many real-world HINs are sparse, and each node has only a few directly connected neighbors. Although some HGNNs can utilize distant neighbors by stacking multiple layers or leveraging long meta-paths, the exponentially increased number of nodes in the receptive field or the number of meta-paths incurs high computation and memory costs. To address these issues, we investigate the importance of different meta-paths and propose Long-range Dependency based Multi-Layer Perceptron (LDMLP). Specifically, to solve the high-cost problem of leveraging long-range dependencies, LDMLP adopts a search stage to discover effective meta-paths automatically, reducing the exponentially increased number of meta-paths to a constant. To avoid the influence of specific modules on search results, LDMLP utilizes a simple architecture with only multi-layer perceptions in the search stage, improving the generalization of searched meta-paths. As a result, the searched meta-paths not only perform well in LDMLP but also enable other HGNNs like HAN and SeHGNN to perform better. Extensive experiments on eight heterogeneous datasets demonstrate that LDMLP achieves state-of-the-art performance while enjoying high efficiency and generalization, especially on sparse HINs., Comment: 12 pages, 3 figures

Supplementary Material 10

International audience; Branch and Bound (BnB) has been successfully used to solve many combinatorial optimization problems. However, BnB MaxSAT solvers perform poorly when solving real-world and academic optimization problems. They are only competitive for random and some crafted instances. Thus, it is a prevailing opinion in the community that BnB is not really useful for practical MaxSAT solving. We refute this opinion by presenting a new BnB MaxSAT solver, called MaxCDCL, which combines clause learning and an efficient bounding procedure. MaxCDCL is among the top 5 out of a total of 15 exact solvers that participated in the 2020 MaxSAT Evaluation, solving several instances that other solvers cannot solve. Furthermore, MaxCDCL solves the highest number of instances from different MaxSAT Evaluations when combined with the best existing solvers.

The accurate length change of the double ball bar (DBB) is very important for the quantitative calculation of geometric errors. The installation error of the DBB can seriously affect the length change, which was usually considered as the eccentricities and removed mistakenly. This paper proposed a removal method for the installation error in the circular test. Firstly, the comprehensive length change model was established with pre-fitting the geometric errors. Secondly, the Fourier transformation was carried out to solve the installation errors. Meanwhile, the quantitative relationship between the eccentricity and the installation error is built. Then, the essential errors of the proposed methods were analyzed, and the simulation results indicated that the proposed method can achieve a good removal accuracy with an appropriate radius difference. Finally, two circular tests with a radius of 100 mm and 112.936 mm were carried out, and installation error was separated from the geometric errors. As the result, the installation errors were obtained as − 13.9 μm in X direction and 37.7 μm in Y direction, and the amplitude of the length change of the DBB decreases from 69.0 to 21.5 μm, which shows the great effect of the installation error.

Background: Cryoglobulinemia is a syndrome characterized by the presence of cryoglobulins (CGs) in serum, and cardiac involvement is a rare occurrence that can affect treatment and prognosis. This study aimed to explore the clinical characteristics of cryoglobulinemia with cardiac involvement.Methods: 108 patients diagnosed with cryoglobulinemia who were admitted and treated in Peking Union Medical College Hospital (PUMCH) between June 1985 and June 2019 were enrolled in the present study. Clinical characteristics, therapy, and prognosis of patients with cardiac involvement were retrospectively analyzed.Results: The cryoglobulinemia with cardiac involvement was found in 7 patients, thus reaching the incidence of 6.5%. Heart failure was the main cardiac manifestation found in these patients, all with the involvement of external cardiac organs. Laboratory examinations showed significant elevation of N-terminal brain natriuretic peptide precursor (NT-proBNP) and brain natriuretic peptide (BNP) with negative troponin (cTnI). Electrocardiogram (ECG) was generally normal or only showed low-flat and biphasic multi-lead T waves. Echocardiography was performed in 6 patients, all of whom showed enlargement of heart cavity. Five patients had reduced left ventricular myocardial contractible motion with decreased ejection fraction, 3 patients had pericardial effusion, and 1 patient had left ventricular hypertrophy or severe aortic insufficiency. Cardiac magnetic resonance imaging showed delayed myocardial enhancement in 2 patients. One patient underwent a myocardial biopsy, which showed perivasculitis. Condition in 6 patients who received active treatment targeting improved in the early stage. Three patients (3/7, 42.9%) died due to disease progression during follow-up period.Conclusions: Cryoglobulinemia with cardiac involvement is a rare but serious condition that has relatively high risk of death. When patients with cryoglobulinemia without underlying heart disease experience heart failure, chest pain, or elevation of asymptomatic NT-proBNP and BNP, there is a high possibility of cardiac involvement, even if the electrocardiogram and troponin are negative. Further examinations such as echocardiography, cardiac magnetic resonance imaging, and myocardial biopsy examination could contribute to the diagnosis. Cardiac manifestations could be timely reversed after active targeted treatment. NT-proBNP and echocardiography could be used for the monitoring of disease efficacy.

Maximum Common induced Subgraph (MCS) is an important NP-hard problem with wide real-world applications. Branch-and-Bound (BnB) is the basis of a class of efficient algorithms for MCS, consisting in successively selecting vertices to match and pruning when it is discovered that a solution better than the best solution found so far does not exist. The method of selecting the vertices to match is essential for the performance of BnB. In this paper, we propose a new value function and a hybrid selection strategy used in reinforcement learning to define a new vertex selection method, and propose a new BnB algorithm, called McSplitDAL, for MCS. Extensive experiments show that McSplitDAL significantly improves the current best BnB algorithms, McSplit+LL and McSplit+RL. An empirical analysis is also performed to illustrate why the new value function and the hybrid selection strategy are effective.

Existing textual adversarial attacks usually utilize the gradient or prediction confidence to generate adversarial examples, making it hard to be deployed in real-world applications. To this end, we consider a rarely investigated but more rigorous setting, namely hard-label attack, in which the attacker can only access the prediction label. In particular, we find we can learn the importance of different words via the change on prediction label caused by word substitutions on the adversarial examples. Based on this observation, we propose a novel adversarial attack, termed Text Hard-label attacker (TextHacker). TextHacker randomly perturbs lots of words to craft an adversarial example. Then, TextHacker adopts a hybrid local search algorithm with the estimation of word importance from the attack history to minimize the adversarial perturbation. Extensive evaluations for text classification and textual entailment show that TextHacker significantly outperforms existing hard-label attacks regarding the attack performance as well as adversary quality., Comment: Accepted by EMNLP 2022 Findings, Code is available at https://github.com/JHL-HUST/TextHacker

TSP is a classical NP-hard combinatorial optimization problem with many practical variants. LKH is one of the state-of-the-art local search algorithms for the TSP. LKH-3 is a powerful extension of LKH that can solve many TSP variants. Both LKH and LKH-3 associate a candidate set to each city to improve the efficiency, and have two different methods, $\alpha$-measure and POPMUSIC, to decide the candidate sets. In this work, we first propose a Variable Strategy Reinforced LKH (VSR-LKH) algorithm, which incorporates three reinforcement learning methods (Q-learning, Sarsa, Monte Carlo) with LKH, for the TSP. We further propose a new algorithm called VSR-LKH-3 that combines the variable strategy reinforcement learning method with LKH-3 for typical TSP variants, including the TSP with time windows (TSPTW) and Colored TSP (CTSP). The proposed algorithms replace the inflexible traversal operations in LKH and LKH-3 and let the algorithms learn to make a choice at each search step by reinforcement learning. Both LKH and LKH-3, with either $\alpha$-measure or POPMUSIC, can be significantly improved by our methods. Extensive experiments on 236 widely-used TSP benchmarks with up to 85,900 cities demonstrate the excellent performance of VSR-LKH. VSR-LKH-3 also significantly outperforms the state-of-the-art heuristics for TSPTW and CTSP., Comment: arXiv admin note: text overlap with arXiv:2107.06870

The Partitioning Min-Max Weighted Matching (PMMWM) problem is an NP-hard problem that combines the problem of partitioning a group of vertices of a bipartite graph into disjoint subsets with limited size and the classical Min-Max Weighted Matching (MMWM) problem. Kress et al. proposed this problem in 2015 and they also provided several algorithms, among which MP$_{\text{LS}}$ is the state-of-the-art. In this work, we observe there is a time bottleneck in the matching phase of MP$_{\text{LS}}$. Hence, we optimize the redundant operations during the matching iterations, and propose an efficient algorithm called the MP$_{\text{KM-M}}$ that greatly speeds up MP$_{\text{LS}}$. The bottleneck time complexity is optimized from $O(n^3)$ to $O(n^2)$. We also prove the correctness of MP$_{\text{KM-M}}$ by the primal-dual method. To test the performance on diverse instances, we generate various types and sizes of benchmarks, and carried out an extensive computational study on the performance of MP$_{\text{KM-M}}$ and MP$_{\text{LS}}$. The evaluation results show that our MP$_{\text{KM-M}}$ greatly shortens the runtime as compared with MP$_{\text{LS}}$ while yielding the same solution quality.

By incorporating the graph structural information into Transformers, graph Transformers have exhibited promising performance for graph representation learning in recent years. Existing graph Transformers leverage specific strategies, such as Laplacian eigenvectors and shortest paths of the node pairs, to preserve the structural features of nodes and feed them into the vanilla Transformer to learn the representations of nodes. It is hard for such predefined rules to extract informative graph structural features for arbitrary graphs whose topology structure varies greatly, limiting the learning capacity of the models. To this end, we propose an adaptive graph Transformer, termed Multi-Neighborhood Attention based Graph Transformer (MNA-GT), which captures the graph structural information for each node from the multi-neighborhood attention mechanism adaptively. By defining the input to perform scaled-dot product as an attention kernel, MNA-GT constructs multiple attention kernels based on different hops of neighborhoods such that each attention kernel can capture specific graph structural information of the corresponding neighborhood for each node pair. In this way, MNA-GT can preserve the graph structural information efficiently by incorporating node representations learned by different attention kernels. MNA-GT further employs an attention layer to learn the importance of different attention kernels to enable the model to adaptively capture the graph structural information for different nodes. Extensive experiments are conducted on a variety of graph benchmarks, and the empirical results show that MNA-GT outperforms many strong baselines., Comment: 8 pages, 4 figures, 5 tables, submitted to a conference of 2023

Heterogeneous information networks (HINs) are widely employed for describing real-world data with intricate entities and relationships. To automatically utilize their semantic information, graph neural architecture search has recently been developed on various tasks of HINs. Existing works, on the other hand, show weaknesses in instability and inflexibility. To address these issues, we propose a novel method called Partial Message Meta Multigraph search (PMMM) to automatically optimize the neural architecture design on HINs. Specifically, to learn how graph neural networks (GNNs) propagate messages along various types of edges, PMMM adopts an efficient differentiable framework to search for a meaningful meta multigraph, which can capture more flexible and complex semantic relations than a meta graph. The differentiable search typically suffers from performance instability, so we further propose a stable algorithm called partial message search to ensure that the searched meta multigraph consistently surpasses the manually designed meta-structures, i.e., meta-paths. Extensive experiments on six benchmark datasets over two representative tasks, including node classification and recommendation, demonstrate the effectiveness of the proposed method. Our approach outperforms the state-of-the-art heterogeneous GNNs, finds out meaningful meta multigraphs, and is significantly more stable., Comment: 12 pages, 7 figures, 8 tables, accepted by AAAI 2023 conference

Node classification on attributed networks is a semi-supervised task that is crucial for network analysis. By decoupling two critical operations in Graph Convolutional Networks (GCNs), namely feature transformation and neighborhood aggregation, some recent works of decoupled GCNs could support the information to propagate deeper and achieve advanced performance. However, they follow the traditional structure-aware propagation strategy of GCNs, making it hard to capture the attribute correlation of nodes and sensitive to the structure noise described by edges whose two endpoints belong to different categories. To address these issues, we propose a new method called the itshape Propagation with Adaptive Mask then Training (PAMT). The key idea is to integrate the attribute similarity mask into the structure-aware propagation process. In this way, PAMT could preserve the attribute correlation of adjacent nodes during the propagation and effectively reduce the influence of structure noise. Moreover, we develop an iterative refinement mechanism to update the similarity mask during the training process for improving the training performance. Extensive experiments on four real-world datasets demonstrate the superior performance and robustness of PAMT.

Knowledge distillation aims to transfer knowledge to the student model by utilizing the predictions/features of the teacher model, and feature-based distillation has recently shown its superiority over logit-based distillation. However, due to the cumbersome computation and storage of extra feature transformation, the training overhead of feature-based methods is much higher than that of logit-based distillation. In this work, we revisit the logit-based knowledge distillation, and observe that the existing logit-based distillation methods treat the prediction logits only in the instance level, while many other useful semantic information is overlooked. To address this issue, we propose a Class-aware Logit Knowledge Distillation (CLKD) method, that extents the logit distillation in both instance-level and class-level. CLKD enables the student model mimic higher semantic information from the teacher model, hence improving the distillation performance. We further introduce a novel loss called Class Correlation Loss to force the student learn the inherent class-level correlation of the teacher. Empirical comparisons demonstrate the superiority of the proposed method over several prevailing logit-based methods and feature-based methods, in which CLKD achieves compelling results on various visual classification tasks and outperforms the state-of-the-art baselines., Comment: 12 pages, 4 figures, 12 tables

Additional file 1: Supplementary Table 1. Sensitive analysis of BMI and WC with All-cause mortality. Supplementary Table 2. Hazard ratios of All-cause mortality according to BMI or WC for various subgroups.

We consider concept generalization at a large scale in the diverse and natural visual spectrum. Established computational modes (i.e., rule-based or similarity-based) are primarily studied isolated and focus on confined and abstract problem spaces. In this work, we study these two modes when the problem space scales up, and the $complexity$ of concepts becomes diverse. Specifically, at the $representational \ level$, we seek to answer how the complexity varies when a visual concept is mapped to the representation space. Prior psychology literature has shown that two types of complexities (i.e., subjective complexity and visual complexity) (Griffiths and Tenenbaum, 2003) build an inverted-U relation (Donderi, 2006; Sun and Firestone, 2021). Leveraging Representativeness of Attribute (RoA), we computationally confirm the following observation: Models use attributes with high RoA to describe visual concepts, and the description length falls in an inverted-U relation with the increment in visual complexity. At the $computational \ level$, we aim to answer how the complexity of representation affects the shift between the rule- and similarity-based generalization. We hypothesize that category-conditioned visual modeling estimates the co-occurrence frequency between visual and categorical attributes, thus potentially serving as the prior for the natural visual world. Experimental results show that representations with relatively high subjective complexity outperform those with relatively low subjective complexity in the rule-based generalization, while the trend is the opposite in the similarity-based generalization.

The graph Transformer emerges as a new architecture and has shown superior performance on various graph mining tasks. In this work, we observe that existing graph Transformers treat nodes as independent tokens and construct a single long sequence composed of all node tokens so as to train the Transformer model, causing it hard to scale to large graphs due to the quadratic complexity on the number of nodes for the self-attention computation. To this end, we propose a Neighborhood Aggregation Graph Transformer (NAGphormer) that treats each node as a sequence containing a series of tokens constructed by our proposed Hop2Token module. For each node, Hop2Token aggregates the neighborhood features from different hops into different representations and thereby produces a sequence of token vectors as one input. In this way, NAGphormer could be trained in a mini-batch manner and thus could scale to large graphs. Moreover, we mathematically show that as compared to a category of advanced Graph Neural Networks (GNNs), the decoupled Graph Convolutional Network, NAGphormer could learn more informative node representations from the multi-hop neighborhoods. Extensive experiments on benchmark datasets from small to large are conducted to demonstrate that NAGphormer consistently outperforms existing graph Transformers and mainstream GNNs. Code is available at https://github.com/JHL-HUST/NAGphormer., Comment: Accepted by ICLR 2023

We attribute the vulnerability of natural language processing models to the fact that similar inputs are converted to dissimilar representations in the embedding space, leading to inconsistent outputs, and we propose a novel robust training method, termed Fast Triplet Metric Learning (FTML). Specifically, we argue that the original sample should have similar representation with its adversarial counterparts and distinguish its representation from other samples for better robustness. To this end, we adopt the triplet metric learning into the standard training to pull words closer to their positive samples (i.e., synonyms) and push away their negative samples (i.e., non-synonyms) in the embedding space. Extensive experiments demonstrate that FTML can significantly promote the model robustness against various advanced adversarial attacks while keeping competitive classification accuracy on original samples. Besides, our method is efficient as it only needs to adjust the embedding and introduces very little overhead on the standard training. Our work shows great potential of improving the textual robustness through robust word embedding., Comment: Accepted by UAI 2022, code is available at https://github.com/JHL-HUST/FTML

In this paper, a multiple-relaxation-time lattice Boltzmann (LB) approach is developed for the simulation of three-dimensional (3D) liquid-vapor phase change based on the pseudopotential model. In contrast to some existing 3D thermal LB models for liquid-vapor phase change, the present approach has two advantages: for one thing, the current approach does not require calculating the gradient of volumetric heat capacity [i.e., $\nabla \left( {\rho {c_v}} \right)$], and for another, the current approach is constructed based on the seven discrete velocities in three dimensions (D3Q7), making the current thermal LB model more efficient and easy to implement. Also, based on the scheme proposed by Zhou and He [Phys Fluids 9:1591-1598, 1997], a pressure boundary condition for the D3Q19 lattice is proposed to model the multiphase flow in open systems. The current method is then validated by considering the temperature distribution in a 3D saturated liquid-vapor system, the $d^2$ law and the droplet evaporation on a heated surface. It is observed that the numerical results fit well with the analytical solutions, the results of the finite difference method and the experimental data. Our numerical results indicate that the present approach is reliable and efficient in dealing with the 3D liquid-vapor phase change.

We give a simple combinatorial algorithm to deterministically approximately count the number of satisfying assignments of general constraint satisfaction problems (CSPs). Suppose that the CSP has domain size $q=O(1)$, each constraint contains at most $k=O(1)$ variables, shares variables with at most $Δ=O(1)$ constraints, and is violated with probability at most $p$ by a uniform random assignment. The algorithm returns in polynomial time in an improved local lemma regime: \[ q^2\cdot k\cdot p\cdotΔ^5\le C_0\quad\text{for a suitably small absolute constant }C_0. \] Here the key term $Δ^5$ improves the previously best known $Δ^7$ for general CSPs [JPV21b] and $Δ^{5.714}$ for the special case of $k$-CNF [JPV21a, HSW21]. Our deterministic counting algorithm is a derandomization of the very recent fast sampling algorithm in [HWY22]. It departs substantially from all previous deterministic counting Lovász local lemma algorithms which relied on linear programming, and gives a deterministic approximate counting algorithm that straightforwardly derandomizes a fast sampling algorithm, hence unifying the fast sampling and deterministic approximate counting in the same algorithmic framework. To obtain the improved regime, in our analysis we develop a refinement of the $\{2,3\}$-trees that were used in the previous analyses of counting/sampling LLL. Similar techniques can be applied to the previous LP-based algorithms to obtain the same improved regime and may be of independent interests., Accepted to SODA 2023. arXiv admin note: text overlap with arXiv:2204.01520

In recent years, many data augmentation techniques have been proposed to increase the diversity of input data and reduce the risk of overfitting on deep neural networks. In this work, we propose an easy-to-implement and model-free data augmentation method called Local Magnification (LOMA). Different from other geometric data augmentation methods that perform global transformations on images, LOMA generates additional training data by randomly magnifying a local area of the image. This local magnification results in geometric changes that significantly broaden the range of augmentations while maintaining the recognizability of objects. Moreover, we extend the idea of LOMA and random cropping to the feature space to augment the feature map, which further boosts the classification accuracy considerably. Experiments show that our proposed LOMA, though straightforward, can be combined with standard data augmentation to significantly improve the performance on image classification and object detection. And further combination with our feature augmentation techniques, termed LOMA_IF&FO, can continue to strengthen the model and outperform advanced intensity transformation methods for data augmentation., 10 pages, 7 figures, 7 tables, submitted to a conference of 2023

In the forming process of helical surface, the forming cutter profile has a decisive influence on machining accuracy. In this paper, a point-vector envelope (PVE) method is proposed to calculate the forming cutter profile of a helical surface. Firstly, the transverse profile of the helical surface is dispersed into a series of points based on the principle of average, and add corresponding normal vector to the discrete point for obtaining the point-vector (PV), and use the helical motion of PVs to envelope the forming cutter profile. Then, through the coordinate transformation and projection of PVs, the PV groups are formed on the calculation plane. A PVE approximation algorithm is constructed to determine all the envelope points of the PV groups, and the grinding wheel profile is obtained by fitting all the envelope points into curves. Finally, the validity of the proposed digital method is demonstrated in numerical and experimental example of the helical gear form grinding.

Due to the extensive applications of power electronic devices in power grids, many insulation components will be stressed by high voltage PWM, multilevel or even more complex waveforms. In this paper, a high voltage generator able to produce various waveforms is developed. The generator is based on cascade connected multilevel modular converter (MMC) topology, including 20 half bridge MMC cells with the capability of 21 output voltage levels. Half of the series-connected MMC cells are charged positively, while the other half are charged negatively. Each positive and negative half bridge MMC cell is connected alternately to optimize the insulation design. The power supply of the driving circuit for each MMC cell is provided by an individual fly-back converter fed by the cell capacitor to realize a compact design. On the basis of simulations, a multilevel arbitrary waveform generator prototype was constructed, with the maximum peak-to-peak voltage of ∼ 28 kV. A capacitor switching strategy is utilized to solve the voltage-imbalance problem resulted from the difference of the element parameters and other reasons. Finally, it was tested on a typical insulation with a simulated defect.

Hydrogen and oxygen fuel obtained from water electrolysis mainly contains H2 and O2, usually abbreviated to HHO. The compositional characteristics of HHO were analyzed by gas chromatography-mass spectrometry (GC-MS). The influence of HHO on the combustion process in the cylinder was discussed. The change law of the maximum combustion pressure and the variation of the crank angle were studied, and the nonlinear dynamic process of the combustion process was revealed. The research indicated that the cylinder pressure increased after mixing HHO and that the period of flame development and rapid combustion was shortened. With the increase in HHO content, the cycle-by-cycle variations were reduced, combustion stability improved, partial-burning and other abnormal combustion phenomena improved, the phase space trajectory distribution gradually intensified, and the engine combustion process of the cyclical variation increased.

Background Pyropia is an economically advantageous genus of red macroalgae, which has been cultivated in the coastal areas of East Asia for over 300 years. Realizing estimation of macroalgae biomass in a high-throughput way would great benefit their cultivation management and research on breeding and phenomics. However, the conventional method is labour-intensive, time-consuming, manually destructive, and prone to human error. Nowadays, high-throughput phenotyping using unmanned aerial vehicle (UAV)-based spectral imaging is widely used for terrestrial crops, grassland, and forest, but no such application in marine aquaculture has been reported. Results In this study, multispectral images of cultivated Pyropia yezoensis were taken using a UAV system in the north of Haizhou Bay in the midwestern coast of Yellow Sea. The exposure period of P. yezoensis was utilized to prevent the significant shielding effect of seawater on the reflectance spectrum. The vegetation indices of normalized difference vegetation index (NDVI), ratio vegetation index (RVI), difference vegetation index (DVI) and normalized difference of red edge (NDRE) were derived and indicated no significant difference between the time that P. yezoensis was completely exposed to the air and 1 h later. The regression models of the vegetation indices and P. yezoensis biomass per unit area were established and validated. The quadratic model of DVI (Biomass = − 5.550DVI2 + 105.410DVI + 7.530) showed more accuracy than the other index or indices combination, with the highest coefficient of determination (R2), root mean square error (RMSE), and relative estimated accuracy (Ac) values of 0.925, 8.06, and 74.93%, respectively. The regression model was further validated by consistently predicting the biomass with a high R2 value of 0.918, RMSE of 8.80, and Ac of 82.25%. Conclusions This study suggests that the biomass of Pyropia can be effectively estimated using UAV-based spectral imaging with high accuracy and consistency. It also implied that multispectral aerial imaging is potential to assist digital management and phenomics research on cultivated macroalgae in a high-throughput way.

We give a Markov chain based perfect sampler for uniform sampling solutions of constraint satisfaction problems (CSP). Under some mild Lov��sz local lemma conditions where each constraint of the CSP has a small number of forbidden local configurations, our algorithm is accurate and efficient: it outputs a perfect uniform random solution and its expected running time is quasilinear in the number of variables. Prior to our work, perfect samplers are only shown to exist for CSPs under much more restrictive conditions (Guo, Jerrum, and Liu, JACM'19). Our algorithm has two components: 1. A simple perfect sampling algorithm using bounding chains (Huber, STOC'98; Haggstrom and Nelander, Scandinavian Journal of Statistics'99). This sampler is efficient if each variable domain is small. 2. A simple but powerful state tensorization trick to reduce large domains to smaller ones. This trick is a generalization of state compression (Feng, He, and Yin, STOC'21). The crux of our analysis is a simple information percolation argument which allows us to achieve bounds even beyond current best approximate samplers (Jain, Pham, and Vuong, ArXiv'21). Previous related works either use intricate algorithms or need sophisticated analysis or even both. Thus we view the simplicity of both our algorithm and analysis as a strength of our work., 56 pages, 1 table, 5 figures, 9 algorithms

A line of work has shown that natural text processing models are vulnerable to adversarial examples. Correspondingly, various defense methods are proposed to mitigate the threat of textual adversarial examples, eg, adversarial training, input transformations, detection, etc. In this work, we treat the optimization process for synonym substitution based textual adversarial attacks as a specific sequence of word replacement, in which each word mutually influences other words. We identify that we could destroy such mutual interaction and eliminate the adversarial perturbation by randomly substituting a word with its synonyms. Based on this observation, we propose a novel textual adversarial example detection method, termed Randomized Substitution and Vote (RS&V), which votes the prediction label by accumulating the logits of k samples generated by randomly substituting the words in the input text with synonyms. The proposed RS&V is generally applicable to any existing neural networks without modification on the architecture or extra training, and it is orthogonal to prior work on making the classification network itself more robust. Empirical evaluations on three benchmark datasets demonstrate that our RS&V could detect the textual adversarial examples more successfully than the existing detection methods while maintaining the high classification accuracy on benign samples., Accepted by UAI 2022, code is avaliable at https://github.com/JHL-HUST/RSV

Additional file 2: Figure S1. Overview of the single-cell analysis strategy of human HCC. Figure S2. Single-cell mixture WES revealed inter-tumor genetic heterogeneity of HCC. Figure S3. High quality single-cell mutation data were obtained by target sequencing. Figure S4. Single-cell clonal structures of HCC1, HCC2 and HCC9 based on point mutations. Figure S5. scRNA-Seq revealed the constituent cell types of HCC. Figure S6. scRNA-Seq revealed the inter-tumor and intra-tumor heterogeneity of HCC. Figure S7. Schematic representation of major findings in this study.

Local search has been demonstrated as an efficient approach for two practical generalizations of the MaxSAT problem, namely Partial MaxSAT (PMS) and Weighted PMS (WPMS). In this work, we observe that most local search (W)PMS solvers usually flip a single variable per iteration. Such a mechanism may lead to relatively low-quality local optimal solutions, and may limit the diversity of search directions to escape from local optima. To address this issue, we propose a general strategy, called farsighted probabilistic sampling (FPS), to replace the single flipping mechanism so as to boost the local search (W)PMS algorithms. FPS considers the benefit of continuously flipping a pair of variables in order to find higher-quality local optimal solutions. Moreover, FPS proposes an effective approach to escape from local optima by preferring the best to flip among the best sampled single variable and the best sampled variable pair. Extensive experiments demonstrate that our proposed FPS strategy significantly improves the state-of-the-art (W)PMS solvers, and FPS has an excellent generalization capability to various local search MaxSAT solvers., Comment: Accepted by AAAI 2023

The square kernel is a standard unit for contemporary CNNs, as it fits well on the tensor computation for convolution operation. However, the retinal ganglion cells in the biological visual system have approximately concentric receptive fields. Motivated by this observation, we propose to use circular kernel with a concentric and isotropic receptive field as an option for the convolution operation. We first propose a simple yet efficient implementation of the convolution using circular kernels, and empirically show the significant advantages of large circular kernels over the counterpart square kernels. We then expand the operation space of several typical Neural Architecture Search (NAS) methods with the convolutions of large circular kernels. The searched new neural architectures do contain large circular kernels and outperform the original searched models considerably. Our additional analysis also reveals that large circular kernels could help the model to be more robust to the rotated or sheared images due to their better rotation invariance. Our work shows the potential of designing new convolutional kernels for CNNs, bringing up the prospect of expanding the search space of NAS with new variants of convolutions., Comment: 20 pages, 10 figures, submitted to a conference

Multi-Object Tracking (MOT) has achieved aggressive progress and derived many excellent deep learning trackers. Meanwhile, most deep learning models are known to be vulnerable to adversarial examples that are crafted with small perturbations but could mislead the model prediction. In this work, we observe that the robustness on the MOT trackers is rarely studied, and it is challenging to attack the MOT system since its mature association algorithms are designed to be robust against errors during the tracking. To this end, we analyze the vulnerability of popular MOT trackers and propose a novel adversarial attack method called Tracklet-Switch (TraSw) against the complete tracking pipeline of MOT. The proposed TraSw can fool the advanced deep pedestrian trackers (i.e., FairMOT and ByteTrack), causing them fail to track the targets in the subsequent frames by perturbing very few frames. Experiments on the MOT-Challenge datasets (i.e., 2DMOT15, MOT17, and MOT20) show that TraSw can achieve an extraordinarily high success attack rate of over 95% by attacking only four frames on average. To our knowledge, this is the first work on the adversarial attack against the pedestrian MOT trackers. Code is available at https://github.com/JHL-HUST/TraSw .

Additional file 1: Supplementary Figure 1. Flow diagram of participant selection. Supplemental Table 1. The population attributable risk percent and the number needed to screened for all-cause mortality based on LTC count. Supplemental Table 2. Multimorbidity and all-cause mortality: Multiple logistic regression analysis. This table shows that being older, living without partner, and being underweight had a higher risk of mortality. In contrast, participants who were female, overweight, class I obesity, and physically active had a significantly lower adjusted risk of all-cause mortality. Supplemental Table 3. Comparison of LTCs (self-report vs physical examination) in prediction of all-cause mortality. Supplemental Table 4. Multivariable Adjusted odds ratio (95% confidence interval) for multimorbidity and all-cause mortality according to the different classification by BMI. Supplemental Table 5. The most impactful LTCs combinations in stratified logistic regression analysis for mortality (based on LTC count). these three tables show that the sensitivity analysis yielded similar findings as our main results, and the risk of death increased with the increase in LTC count.

Deep learning models are known to be vulnerable to adversarial examples crafted by adding human-imperceptible perturbations on benign images. Many existing adversarial attack methods have achieved great white-box attack performance, but exhibit low transferability when attacking other models. Various momentum iterative gradient-based methods are shown to be effective to improve the adversarial transferability. In what follows, we propose an enhanced momentum iterative gradient-based method to further enhance the adversarial transferability. Specifically, instead of only accumulating the gradient during the iterative process, we additionally accumulate the average gradient of the data points sampled in the gradient direction of the previous iteration so as to stabilize the update direction and escape from poor local maxima. Extensive experiments on the standard ImageNet dataset demonstrate that our method could improve the adversarial transferability of momentum-based methods by a large margin of 11.1% on average. Moreover, by incorporating with various input transformation methods, the adversarial transferability could be further improved significantly. We also attack several extra advanced defense models under the ensemble-model setting, and the enhancements are remarkable with at least 7.8% on average., Comment: 13 pages

Community is a universal structure in various complex networks, and community detection is a fundamental task for network analysis. With the rapid growth of network scale, networks are massive, changing rapidly and could naturally be modeled as graph streams. Due to the limited memory and access constraint in graph streams, existing non-streaming community detection methods are no longer applicable. This raises an emerging need for online approaches. In this work, we consider the problem of uncovering the local community containing a few query nodes in graph streams, termed streaming local community detection. This is a new problem raised recently that is more challenging for community detection and only a few works address this online setting. Correspondingly, we design an online single-pass streaming local community detection approach. Inspired by the "local" property of communities, our method samples the local structure around the query nodes in graph streams, and extracts the target community on the sampled subgraph using our proposed metric called the approximate conductance. Comprehensive experiments show that our method remarkably outperforms the streaming baseline on both effectiveness and efficiency, and even achieves similar accuracy comparing to the state-of-the-art non-streaming local community detection methods that use static and complete graphs.

Additional file 1.

Additional file 1. Supplementary Methods.

In the field of adversarial robustness, there is a common practice that adopts the single-step adversarial training for quickly developing adversarially robust models. However, the single-step adversarial training is most likely to cause catastrophic overfitting, as after a few training epochs it will be hard to generate strong adversarial examples to continuously boost the adversarial robustness. In this work, we aim to avoid the catastrophic overfitting by introducing multi-step adversarial examples during the single-step adversarial training. Then, to balance the large training overhead of generating multi-step adversarial examples, we propose a Multi-stage Optimization based Adversarial Training (MOAT) method that periodically trains the model on mixed benign examples, single-step adversarial examples, and multi-step adversarial examples stage by stage. In this way, the overall training overhead is reduced significantly, meanwhile, the model could avoid catastrophic overfitting. Extensive experiments on CIFAR-10 and CIFAR-100 datasets demonstrate that under similar amount of training overhead, the proposed MOAT exhibits better robustness than either single-step or multi-step adversarial training methods., Comment: 13 pages

Background: Cyclin F (CCNF) dysfunction has been implicated in various forms of cancer, offering a new avenue for understanding the pathogenic mechanisms underlying hepatocellular carcinoma (HCC). We aimed to evaluate the role of CCNF in HCC using publicly available data from The Cancer Genome Atlas (TCGA). Method: We used TCGA data and Gene Expression Omnibus (GEO) data to analyze the differential expression of CCNF between tumor and adjacent tissues and the relationship between CCNF and clinical characteristics. We compared prognosis of patients with HCC with high and low CCNF expression and constructed receiver operating characteristic (ROC) curves. In addition, we also explored the types of gene mutations in relevant groups and conducted Gene Set Enrichment Analysis (GSEA). Results: The expression of CCNF in liver cancer tissues was significantly increased compared with that in adjacent tissues, and patients with high CCNF expression had a worse prognosis than those with low CCNF expression. Patients with high CCNF expression also had more somatic mutations. High expression of CCNF hampers the prognosis independently. The GSEA showed that the "http://www.gsea-msigdb.org/gsea/msigdb/cards/BIOCARTA_WNT_PATHWAY" Wnt pathway, "http://www.gsea-msigdb.org/gsea/msigdb/cards/BIOCARTA_P53_PATHWAY" P53 pathway, "http://www.gsea-msigdb.org/gsea/msigdb/cards/HALLMARK_PI3K_AKT_MTOR_SIGNALING" PI3K/Akt/mTOR pathway, "http://www.gsea-msigdb.org/gsea/msigdb/cards/HALLMARK_NOTCH_SIGNALING" Notch pathway were enriched in patients with the high CCNF expression phenotype. Conclusion: High CCNF expression can be seen as an independent risk factor for poor survival in HCC. Its expression may serve as a target for the diagnosis and treatment of liver cancer.

Oxidative stress is a pathophysiological condition resulting in neurotoxicity, which is possibly associated with neurodegenerative disorders. In this study, the antioxidative effects of the antioxidant astaxanthin (AXT) in combination with huperzine A (HupA), which is used as a cholinesterase inhibitor for the treatment of Alzheimer's disease, were investigated. PC12 cells were treated with either tert‑butyl hydroperoxide (TBHP), or with the toxic version of β‑amyloid, Aβ25‑35, to induce oxidative stress and neurotoxicity. Cell viability, morphology, lactate dehydrogenase (LDH) release, intracellular accumulation of reactive oxygen species (ROS), superoxide dismutase (SOD) activity and malondialdehyde (MDA) content were determined, while neuroprotection was also monitored using an MTT assay. It was found that combining AXT with HupA significantly increased the viability of PC12 cells, prevented membrane damage (as measured by LDH release), attenuated intracellular ROS formation, increased SOD activity and decreased the level of MDA after TBHP exposure when compared to these drugs administered alone. Pretreatment with HupA and AXT decreased toxic damage produced by Aβ25‑35. These data indicated that combining an antioxidant with a cholinesterase inhibitor increases the degree of neuroprotection; with future investigation this could be a potential therapy used to decrease neurotoxicity in the brain.

Anonymization technique has been extensively studied and widely applied for privacy-preserving data publishing. In most previous approaches, a microdata table consists of three categories of attribute: explicit-identifier, quasi-identifier (QI), and sensitive attribute. Actually, different individuals may have different view on the sensitivity of different attributes. Therefore, there is another type of attribute that contains both QI values and sensitive values, namely, semi-sensitive attribute. Based on such observation, we propose a new anonymization technique, called local generalization and bucketization, to prevent identity disclosure and protect the sensitive values on each semi-sensitive attribute and sensitive attribute. The rationale is to use local generalization and local bucketization to divide the tuples into local equivalence groups and partition the sensitive values into local buckets, respectively. The protections of local generalization and local bucketization are independent, so that they can be implemented by appropriate algorithms without weakening other protection, respectively. Besides, the protection of local bucketization for each semi-sensitive attribute and sensitive attribute is also independent. Consequently, local bucketization can comply with various principles in different attributes according to the actual requirements of anonymization. The conducted extensive experiments illustrate the effectiveness of the proposed approach.

We address a new variant of packing problem called the circle bin packing problem (CBPP), which is to find a dense packing of circle items to multiple square bins so as to minimize the number of used bins. To this end, we propose an adaptive large neighborhood search (ALNS) algorithm, which uses our Greedy Algorithm with Corner Occupying Action (GACOA) to construct an initial layout. The greedy solution is usually in a local optimum trap, and ALNS enables multiple neighborhood search that depends on the stochastic annealing schedule to avoid getting stuck in local minimum traps. Specifically, ALNS perturbs the current layout to jump out of a local optimum by iteratively reassigns some circles and accepts the new layout with some probability during the search. The acceptance probability is adjusted adaptively using simulated annealing that fine-tunes the search direction in order to reach the global optimum. We benchmark computational results against GACOA in heterogeneous instances. ALNS always outperforms GACOA in improving the objective function, and in several cases, there is a significant reduction on the number of bins used in the packing., Comment: 13 pages, 6 figures, 6 tables