Search

Pedestrian trajectory prediction is an important technique of autonomous driving. In order to accurately predict the reasonable future trajectory of pedestrians, it is inevitable to consider social interactions among pedestrians and the influence of surrounding scene simultaneously, which can fully represent the complex behavior information and ensure the rationality of predicted trajectories obeyed realistic rules. In this article, we propose one new prediction model named social soft attention graph convolution network (SSAGCN), which aims to simultaneously handle social interactions among pedestrians and scene interactions between pedestrians and environments. In detail, when modeling social interaction, we propose a new social soft attention function, which fully considers various interaction factors among pedestrians. Also, it can distinguish the influence of pedestrians around the agent based on different factors under various situations. For the scene interaction, we propose one new sequential scene sharing mechanism. The influence of the scene on one agent at each moment can be shared with other neighbors through social soft attention; therefore, the influence of the scene is expanded both in spatial and temporal dimensions. With the help of these improvements, we successfully obtain socially and physically acceptable predicted trajectories. The experiments on public available datasets prove the effectiveness of SSAGCN and have achieved state-of-the-art results. The project code is available at https://github.com/WW-Tong/ssagcn_for_path_prediction

Traffic intersections are important scenes that can be seen almost everywhere in the traffic system. Currently, most simulation methods perform well at highways and urban traffic networks. In intersection scenarios, the challenge lies in the lack of clearly defined lanes, where agents with various motion plannings converge in the central area from different directions. Traditional model-based methods are difficult to drive agents to move realistically at intersections without enough predefined lanes, while data-driven methods often require a large amount of high-quality input data. Simultaneously, tedious parameter tuning is inevitable involved to obtain the desired simulation results. In this paper, we present a novel adaptive and planning-aware hybrid-driven method (TraInterSim) to simulate traffic intersection scenarios. Our hybrid-driven method combines an optimization-based data-driven scheme with a velocity continuity model. It guides the agent's movements using real-world data and can generate those behaviors not present in the input data. Our optimization method fully considers velocity continuity, desired speed, direction guidance, and planning-aware collision avoidance. Agents can perceive others’ motion plannings and relative distances to avoid possible collisions. To preserve the individual flexibility of different agents, the parameters in our method are automatically adjusted during the simulation. TraInterSim can generate realistic behaviors of heterogeneous agents in different traffic intersection scenarios in interactive rates. Through extensive experiments as well as user studies, we validate the effectiveness and rationality of the proposed simulation method.

Maize (Zea maysL.) Fusarium ear rot (FER), which is caused by the fungal pathogen Fusarium verticillioides, is a major ear disease affecting maize production in China. Developing resistant maize hybrids is a cost‐effective and environmentally friendly method of reducing yield losses due to FER. Using the nail punch inoculation method, 17 highly resistant and 16 highly susceptible inbred lines were identified in this study, and a large‐scale maize hybrid consisting of 528 F1’s was produced using a half‐diallel cross model. The hybrids were inoculated with F. verticillioidesin 6 year–location environment combinations during 2020–2022. Our results showed that estimates of genotypic, environmental, and genotype × environment interaction variance components were very significant (p< 0.001). All the crosses generated from highly resistant inbred lines exhibited high kernel resistance, whereas those generated from highly susceptible inbred lines mostly exhibited high susceptibility to FER. Crosses between resistant and susceptible inbred lines showed intermediate resistance to FER. The general combining ability (GCA) effect was greater than the specific combining ability effect across multiple environments, indicating that the inheritance of FER resistance was mainly influenced by additive genetic effects. The correlation coefficient between the expected GCA value and hybrid resistance was 0.789, whereas that between the mid‐parent value and hybrid resistance was only 0.644. Inbred performance per se and their corresponding hybrids across multiple environments were significantly correlated (r= 0.823). However, no relationship was observed between the FER resistance of F1’s and that of their parental heterotic groups. In breeding, selecting parents with Fusarium ear rot (FER) resistance is more crucial than selecting the hybrids themselves.The additive effect is the main genetic component of ear rot in maize, whereas the non‐additive effect is minor.Additional inoculation tests for FER resistance are required for specific hybrid combinations.Hybrid resistance can be inferred based on the general combining ability of parental inbred lines.The additive effects of resistance genes in highly inbred lines are minor.

Low-light stereo image enhancement (LLSIE) is a task that improves the visual quality of stereo images captured in low-light environment. The core idea of LLSIE is to exploit the cross-view cues for enhancing the stereo image pairs. However, LLSIE is still a challenging task due to the perspective difference and insufficient information interaction between left and right views. In this paper, we propose a novel Multi-Scale Interaction Network (MSINet) for LLSIE. To be specific, we present a new multi-scale stereo Interaction module (MIM), which can promote the multi-scale cross-view information flow. Firstly, MIM extracts multi-scale features from both the left and right views. Then, the features from two views are fused in different scales, so that the information from stereo image pairs can be fully exchanged. Compared to the existing parallax attention module (PAM), our MIM explores the cross-view interaction in multiple scales with less computational cost more sufficiently. Besides, we also develop a new Spatial Gating Block (SGB) for refining intra-view features. By incorporating the long residual and spatial gating mechanism, SGB is capable of completing dynamic spatial modeling and obtaining strong feature representations. Extensive experiments on the Flickr1024, KITTI 2015 and Middlebury datasets show that the proposed MSINet reaches SOTA performance and obtains better visual results than other compared methods.

Despite significant advancements in Multi-Agent Deep Reinforcement Learning (MADRL) approaches for Traffic Light Control (TLC), effectively coordinating agents in diverse traffic environments remains a challenge. Studies in MADRL for TLC often focus on repeatedly constructing the same intersection models with sparse experience. However, real road networks comprise Multi-Type of Intersections (MTIs) rather than being limited to intersections with four directions. In the scenario with MTIs, each type of intersection exhibits a distinctive topology structure and phase set, leading to disparities in the spaces of state and action. This article introduces Adaptive Multi-agent Deep Mixed Reinforcement Learning (AMDMRL) for addressing tasks with multiple types of intersections in TLC. AMDMRL adopts a two-level hierarchy, where high-level proxies guide low-level agents in decision-making and updating. All proxies are updated by value decomposition to obtain the globally optimal policy. Moreover, the AMDMRL approach incorporates a mixed cooperative mechanism to enhance cooperation among agents, which adopts a mixed encoder to aggregate the information from correlated agents. We conduct comparative experiments involving four traditional and four DRL-based approaches, utilizing three training and four testing datasets. The results indicate that the AMDMRL approach achieves average reductions of 41% than traditional approaches, and 16% compared to DRL-based approaches in traveling time on three training datasets. During testing, the AMDMRL approach exhibits a 37% improvement in reward compared to the MADRL-based approaches.

With smartphones and geo-locating devices widely used around the world, ridesharing, as a main application field of spatial crowdsourcing, has been fast expanding its widespread adoption and potentially brings great benefits to human society and the environment. However, ridesharing has so far not been as popular as expected. A recent survey shows that notable obstacles come from concerns about social comfort and price fairness when riding with strangers. To defeat these obstacles, in this paper, we study a novel problem of utility-aware ride matching (URM) for dynamic ridesharing, where drivers and riders are matched in batches by considering their social comfort and price revenue. While the URM problem is of practical usefulness, we prove that this problem is Nondeterministic Polynomial Hard (NP-hard). To tackle the problem optimally and find exact answers, we present a novel bipartite matching algorithm by integrating an effective Driver-Rider Graph (DR-Graph) index. To balance the effectiveness and efficiency, we propose two efficient algorithms to solve the URM problem with only a small loss of utility. Leveraging a bounded Driver-Rider-group Graph (DRg-Graph) and several useful pruning bounds for matching utility and travel cost, we develop an $\epsilon$ε-refining algorithm to find an $\epsilon$ε-approximation matching utility of the optimal answer. Extensive experiments on real datasets showed that our proposed algorithms achieved effective matching results of social-pricing based utilities and the efficient performance of quick matching under various parameter settings.

Surface defect detection is an important step in ensuring product quality in various manufacturing industries. Existing methods have achieved significant results, but there are still challenges, such as the lack of real-time detection speed, low contrast between defects and background, and insufficient handling of defect details. To address these issues, we introduce a lightweight and efficient method called context-aware adaptive weighted attention network (CAWANet) for real-time surface defect segmentation. To handle the computational resource constraints when dealing with deep features, we introduce context-aware adaptive weighted convolution (CAWAConv), which aims to extract deep features while suppressing noise interference. This allows the model to remain lightweight without compromising its ability to recognize subtle defect characteristics. In addition, during the feature fusion stage, we propose the feature detail improvement (FDI) module to capture more complex defect details. The FDI module enhances the representation of defect-related information, further optimizing the segmentation results. We conducted experimental evaluations of CAWANet on three surface defect detection datasets: the magnetic tile, NEU-Seg, and MSD. The experimental results indicate that our proposed CAWANet strikes a favorable balance between accuracy and inference speed compared to other state-of-the-art methods. Our code will be available at https://github.com/ZGWzzu/CAWANet.

Numerous patterns found in urban phenomena, such as air pollution and human mobility, can be characterized as many directed geospatial networks (geo-networks) that represent spreading processes in urban space. These geo-networks can be analyzed from multiple levels, ranging from the macro-level of summarizing all geo-networks, meso-level of comparing or summarizing parts of geo-networks, and micro-level of inspecting individual geo-networks. Most of the existing visualizations cannot support multilevel analysis well. These techniques work by: 1) showing geo-networks separately with multiple maps leads to heavy context switching costs between different maps; 2) summarizing all geo-networks into a single network can lead to the loss of individual information; 3) drawing all geo-networks onto one map might suffer from the visual scalability issue in distinguishing individual geo-networks. In this study, we propose GeoNetverse, a novel visualization technique for analyzing aggregate geo-networks from multiple levels. Inspired by metro maps, GeoNetverse balances the overview and details of the geo-networks by placing the edges shared between geo-networks in a stacked manner. To enhance the visual scalability, GeoNetverse incorporates a level-of-detail rendering, a progressive crossing minimization, and a coloring technique. A set of evaluations was conducted to evaluate GeoNetverse from multiple perspectives.

Graph Convolutional Networks (GCNs) derive inspiration from recent advances in computer vision, by stacking layers of first-order filters followed by a nonlinear activation function to learn entity or graph embeddings. Although GCNs have been shown to boost the performance of many network analysis tasks, they still face tremendous challenges in learning from Heterogeneous Information Networks (HINs), where relations play a decisive role in knowledge reasoning. What’s more, there are multiaspect representations of entities in HINs, and a filter learned in one aspect do not necessarily apply to another. We address these challenges by proposing the Aspect-Aware Graph Attention Network (AGAT), a model that extends GCNs with alternative learnable filters to incorporate entity and relational information. Instead of focusing on learning the general entity embeddings, AGAT learns the adaptive entity embeddings based on prediction scenario. Experiments of link prediction and semi-supervised classification verify the effectiveness of our algorithm.

Quantum computing is a rapidly evolving field that enables exponential speed-up over classical algorithms. At the heart of this revolutionary technology are quantum circuits, which serve as vital tools for implementing, analyzing, and optimizing quantum algorithms. Recent advancements in quantum computing and the increasing capability of quantum devices have led to the development of more complex quantum circuits. However, traditional quantum circuit diagrams suffer from scalability and readability issues, which limit the efficiency of analysis and optimization processes. In this research, we propose a novel visualization approach for large-scale quantum circuits by adopting semantic analysis to facilitate the comprehension of quantum circuits. We first exploit meta-data and semantic information extracted from the underlying code of quantum circuits to create component segmentations and pattern abstractions, allowing for easier wrangling of massive circuit diagrams. We then develop Quantivine, an interactive system for exploring and understanding quantum circuits. A series of novel circuit visualizations is designed to uncover contextual details such as qubit provenance, parallelism, and entanglement. The effectiveness of Quantivine is demonstrated through two usage scenarios of quantum circuits with up to 100 qubits and a formal user evaluation with quantum experts. A free copy of this paper and all supplemental materials are available at https://osf.io/2m9yh/?view_only=0aa1618c97244f5093cd7ce15f1431f9.

As a new computing paradigm, crowdsourced mobility service is booming with the rapid development of sharing economy. In the typical crowdsourced mobility service, a large number of part-time workers perform the spatial tasks offered by the platform and share the benefits in proportion, thereby, the strategy of task assignment directly affects the level of revenue and fairness among workers. In order to balance the revenue and fairness of workers, in this paper we study a novel type of fairness-aware spatial crowdsourcing problem, namely Fairness-Guaranteed Task Assignment (FGTA), which aims to maximize the total revenue of workers at a certain level of fairness guarantee and that is proved to be NP-hard. To solve this problem, we propose an efficient game-theory based approach for task assignment, which makes use of the best-response framework to iteratively select the best strategy for each worker until a Nash equilibrium is reached. Inspired by the observation that tasks with similar spatial and temporal features can be assigned together to a worker, we propose a spatial-temporal grouping based optimization to further improve the efficiency of task assignment. Furthermore, to improve the quality of Nash equilibrium, we present an effective large neighborhood search based optimization that trains a DQN decision model as destroy operator to accelerate the convergence of optimal task assignment. Finally, extensive experiments conducted on two real-world datasets demonstrate that our proposed approaches achieve better effectiveness and efficiency than the state-of-the-arts.

The adoption of unmanned vehicles in urban logistics has gradually become a trend. It can effectively lower carbon emissions, reduce labor costs, and improve logistics efficiency. In this paper, we investigate a novel problem of unmanned vehicle scheduling (UVS) for large-scale urban logistics, where the logistics platform assigns unmanned vehicles to deliver parcels among stations under the constraints of time, capacity, and electricity to maximize the overall revenue of the logistics platform. Although the UVS problem is of practical usefulness, solving it requires non-trivial efforts, because we have proved that the UVS problem is NP-hard. To solve the UVS problem efficiently, we propose an efficient two-stage processing framework, including task assignment and vehicle reposition. Specifically, in the first stage, we propose an effective preference-aware matching (PAM) algorithm to deal with task assignments between unmanned vehicles and delivery tasks, which considers not only the electricity consumption of unmanned vehicles but also the supply-demand balance between delivery tasks and unmanned vehicles. In the second stage, we propose two vehicle repositioning algorithms based on deep reinforcement learning, termed restricted DQN repositioning algorithm (RDR) and restricted A2C repositioning algorithm (RAR), which can effectively refine the vehicle’s reposition stations based on vehicle supply and demand, electricity supply and demand, charging pile availability and collision avoidance restriction rules at current and neighbor stations, so that the vehicles can be efficiently relocated to stations with over-delivery tasks. Finally, extensive experiments have demonstrated that our proposed algorithms can achieve desirable efficiency and effectiveness.

In soccer, player action evaluation provides a fine-grained method to analyze player performance and plays an important role in improving winning chances in future matches. However, previous studies on action evaluation only provide a score for each action, and hardly support inspecting and comparing player actions integrated with complex match context information such as team tactics and player locations. In this work, we collaborate with soccer analysts and coaches to characterize the domain problems of evaluating player performance based on action scores. We design a tailored visualization of soccer player actions that places the action choice together with the tactic it belongs to as well as the player locations in the same view. Based on the design, we introduce a visual analytics system, Action-Evaluator, to facilitate a comprehensive player action evaluation through player navigation, action investigation, and action explanation. With the system, analysts can find players to be analyzed efficiently, learn how they performed under various match situations, and obtain valuable insights to improve their action choices. The usefulness and effectiveness of this work are demonstrated by two case studies on a real-world dataset and an expert interview.

In geo-related fields such as urban informatics, atmospheric science, and geography, large-scale spatial time (ST) series (i.e., geo-referred time series) are collected for monitoring and understanding important spatiotemporal phenomena. ST series visualization is an effective means of understanding the data and reviewing spatiotemporal phenomena, which is a prerequisite for in-depth data analysis. However, visualizing these series is challenging due to their large scales, inherent dynamics, and spatiotemporal nature. In this study, we introduce the notion of patterns of evolution in ST series. Each evolution pattern is characterized by 1) a set of ST series that are close in space and 2) a time period when the trends of these ST series are correlated. We then leverage Storyline techniques by considering an analogy between evolution patterns and sessions, and finally design a novel visualization called GeoChron, which is capable of visualizing large-scale ST series in an evolution pattern-aware and narrative-preserving manner. GeoChron includes a mining framework to extract evolution patterns and two-level visualizations to enhance its visual scalability. We evaluate GeoChron with two case studies, an informal user study, an ablation study, parameter analysis, and running time analysis.

Plant height is an important agronomic trait that affects high-density tolerance and lodging resistance. However, the regulators and their underlying molecular mechanisms controlling plant height in maize remain understudied. Here, we report that knockout mutants of the calcium-dependent protein kinase gene ZmCPK39(ZmCPK39-KO) exhibit dramatically reduced plant height, characterized by shorter internodes and a slight decrease in node numbers. Furthermore, we identified a ZmCPK39-interacting protein, the knotted-related homeobox (ZmKnox2), and observed that plant height was also significantly reduced in a mutatortransposon-inserted mutant of ZmKnox2(ZmKnox2-Mu). Combined analysis of transcriptomic and metabonomic data indicates that multiple phytohormone signaling and photosynthesis pathways are disrupted in both ZmCPK39-KO and ZmKnox2-Mu mutants. Taken together, these results provide new insights into the function of ZmCPK39and identify potential targets for breeding lodging-resistant and high-density tolerant maize cultivars.

Gray leaf spot (GLS), caused by the fungal pathogens Cercospora zeae-maydisand Cercospora zeina, is a major foliar disease of maize worldwide (Zea maysL.). Here we demonstrate that ZmWAKLencoding cell-wall-associated receptor kinase-like protein is the causative gene at the major quantitative disease resistance locus against GLS. The ZmWAKLYprotein, encoded by the resistance allele, can self-associate and interact with a leucine-rich repeat immune-related kinase ZmWIK on the plasma membrane. The ZmWAKLY/ZmWIK receptor complex interacts with and phosphorylates the receptor-like cytoplasmic kinase (RLCK) ZmBLK1, which in turn phosphorylates its downstream NADPH oxidase ZmRBOH4. Upon pathogen infection, ZmWAKLYphosphorylation activity is transiently increased, initiating immune signaling from ZmWAKLY, ZmWIK, ZmBLK1 to ZmRBOH4, ultimately triggering a reactive oxygen species burst. Our study thus uncovers the role of the maize ZmWAKL–ZmWIK–ZmBLK1–ZmRBOH4 receptor/signaling/executor module in perceiving the pathogen invasion, transducing immune signals, activating defense responses and conferring increased resistance to GLS.

The booming of mobile internet and crowdsourcing technology has offered great opportunities for first and last mile logistics (FLML) service. Unlike the traditional FLML service that separates the parcel collection in the first mile from the parcel delivery in the last mile, a new type of crowdsourced FLML service integrates parcel collection and parcel delivery services as a whole, which can significantly improve the efficiency of FLML service. Briefly, in a crowdsourced FLML service, the platform assigns the customers’ triggered pick-up parcels to the couriers who are delivering drop-off parcels in terms of the real-time status of couriers (e.g., capacity, location, and schedule). Existing works solving the crowdsourced FLML problem only consider the utility maximization for the platform but ignore the incentive to the utilities of couriers. Inspired by this, in this paper, we investigate a novel type of crowdsourced FLML problem, namely Auction-based Crowdsourced FLML (ACF), where the platform assigns the couriers with suitable pick-up parcels based on the preferences of couriers with the goal of maximizing the social welfare of the platform and couriers. To solve the ACF problem, we present a novel auction model named Multi-attribute Reverse Vickrey (MRV), where the couriers bid on parcels according to their preferences for parcels. Based on the MRV model, we present three efficient assignment algorithms to assign parcels to couriers. In addition, we give theoretical analysis for our proposed algorithms. Extensive experiments examine the efficiency and effectiveness of our solutions.

An insightful understanding and relational reasoning of motion behavior are typical components for trajectory prediction to achieve safe planning when navigating in complex scenarios. Due to the differences in behavioral responses of heterogeneous agents and the existence of chain effect in message passing, an effective prediction method is desired to better acquire potential behavioral intention and model motion behavior. In this paper, we construct a trajectory prediction method to represent and encode the behavioral interactions among heterogeneous agents, called as Tree variant with Behavioral Intention Perception (BIP-Tree). Specifically, a dual-behavior interaction module is presented to deeply understand behavioral intention by simultaneously considering the behavioral perception and behavioral response in spatial interaction. The behavioral perception means that individual acquires behavioral features from interactive objects located in its perception range, while the behavioral response means that each agent makes distinctive reactions to different categories of agents (for example, due to different collision risks caused by pedestrian and vehicle, a pedestrian will respond differently to the interactive agents at the same distance). Meanwhile, we also introduce one new tree variant in message passing stage to enhance the acquisition of potential motion feature, denoting traffic agents as nodes and the interactions among them as tree trunks. The interaction message can be delivered along tree trunks from leaf nodes to root node, to further achieves the chain effect of high-order interactions beyond adjacent entities. Our method is evaluated on several public datasets, such as Apolloscape, nuScenes, Argoverse, SDD, INTERACTION, inD, and Waymo. The extensive experimental results demonstrate that our method can predict more plausible and realistic trajectories with multi-modality. Among them, the best performance is achieved on three datasets. More remarkably, compared with state-of-the-arts, our method achieves significant performance and decreases by at least 13.04% on average ADE and 19.42% on average FDE on inD dataset with four intersections. The dataset and code are available at: htpps://github.com/VTP-TL/BIP-Tree.

Developing effective visual analytics systems demands care in characterization of domain problems and integration of visualization techniques and computational models. Urban visual analytics has already achieved remarkable success in tackling urban problems and providing fundamental services for smart cities. To promote further academic research and assist the development of industrial urban analytics systems, we comprehensively review urban visual analytics studies from four perspectives. In particular, we identify 8 urban domains and 22 types of popular visualization, analyze 7 types of computational method, and categorize existing systems into 4 types based on their integration of visualization techniques and computational models. We conclude with potential research directions and opportunities.

With the continued proliferation of mobile Internet and geo-locating technologies, carpooling as a green transport mode is widely accepted and becoming tremendously popular worldwide. In this paper, we focus on a popular carpooling service called ride hitching, which is typically implemented using a publish/subscribe approach. In a ride hitching service, drivers subscribe ride orders published by riders and continuously receive matching ride orders until one is picked. The current systems (e.g., Didi Hitch) adopt a threshold-based approach to filter ride orders. That is, a new ride order will be sent to all subscribing drivers whose planned trips can match the ride order within a pre-defined detour threshold. A limitation of this approach is that it is difficult for drivers to specify a reasonable detour threshold in practice. In addressing this problem, we propose a novel type of top-$k$k subscription queries called Top-$k$k Ride Subscription (TkRS) query, which continuously returns the best $k$k ride orders that match drivers’ trip plans to them. We propose two efficient algorithms to enable the top-$k$k result maintenance. We also design a novel hybrid grid index and a two-level buffer structure to efficiently track the top-$k$k results for all TkRS queries. Finally, extensive experiments on real-life datasets suggest that our proposed algorithms are capable of achieving desirable performance in practical settings.

Multiple-view (MV) representations enabling multi-perspective exploration of large and complex data are often employed on 2D displays. The technique also shows great potential in addressing complex analytic tasks in immersive visualization. However, although useful, the design space of MV representations in immersive visualization lacks in deep exploration. In this paper, we propose a new perspective to this line of research, by examining the effects of view layout for MV representations on situated analytics. Specifically, we disentangle situated analytics in perspectives of situatedness regarding spatial relationship between visual representations and physical referents, and analytics regarding cross-view data analysis including filtering, refocusing, and connecting tasks. Through an in-depth analysis of existing layout paradigms, we summarize design trade-offs for achieving high situatedness and effective analytics simultaneously. We then distill a list of design requirements for a desired layout that balances situatedness and analytics, and develop a prototype system with an automatic layout adaptation method to fulfill the requirements. The method mainly includes a cylindrical paradigm for egocentric reference frame, and a force-directed method for proper view-view, view-user, and view-referent proximities and high view visibility. We conducted a formal user study that compares layouts by our method with linked and embedded layouts. Quantitative results show that participants finished filtering- and connecting-centered tasks significantly faster with our layouts, and user feedback confirms high usability of the prototype system.

Data workers usually seek to understand the semantics of data wrangling scripts in various scenarios, such as code debugging, reusing, and maintaining. However, the understanding is challenging for novice data workers due to the variety of programming languages, functions, and parameters. Based on the observation that differences between input and output tables highly relate to the type of data transformation, we outline a design space including 103 characteristics to describe table differences. Then, we develop Comantics, a three-step pipeline that automatically detects the semantics of data transformation scripts. The first step focuses on the detection of table differences for each line of wrangling code. Second, we incorporate a characteristic-based component and a Siamese convolutional neural network-based component for the detection of transformation types. Third, we derive the parameters of each data transformation by employing a “slot filling” strategy. We design experiments to evaluate the performance of Comantics. Further, we assess its flexibility using three example applications in different domains.

Over the past few decades, a large number of graph layout techniques have been proposed for visualizing graphs from various domains. In this paper, we present a general framework, Taurus, for unifying popular techniques such as the spring-electrical model, stress model, and maxent-stress model. It is based on a unified force representation, which formulates most existing techniques as a combination of quotient-based forces that combine power functions of graph-theoretical and Euclidean distances. This representation enables us to compare the strengths and weaknesses of existing techniques, while facilitating the development of new methods. Based on this, we propose a new balanced stress model (BSM) that is able to layout graphs in superior quality. In addition, we introduce a universal augmented stochastic gradient descent (SGD) optimizer that efficiently finds proper solutions for all layout techniques. To demonstrate the power of our framework, we conduct a comprehensive evaluation of existing techniques on a large number of synthetic and real graphs. We release an open-source package, which facilitates easy comparison of different graph layout methods for any graph input as well as effectively creating customized graph layout techniques.

City express delivery services (a.k.a. last-mile delivery) have become more prominent in recent years. Many logistics giants, such as Amazon, JD, and Cainiao, have deployed intelligent express delivery systems to deal with the growing demand for parcel delivery. Existing works adopt queuing or batch processing approaches to assign parcels to couriers. However, these approaches do not fully consider the distribution of parcels and couriers, leading to poor quality of task assignment. In this paper, we investigate a problem of delivery matching based on revenue maximization in real-time city express delivery services. Given a set of couriers and a stream of parcel collection tasks, our problem aims to assign each collection task to a suitable courier to maximize the overall revenue of the platform. The problem is shown to be NP-hard. To tackle the problem efficiently, we present a time-aware batch matching algorithm to offer high-quality courier-task matching in each sliding window. We further theoretically analyze the matching approximation bound. In addition, we propose an efficient deep reinforcement learning based approach to adaptively determine the sliding window size for better matching results. Finally, extensive experiments demonstrate that our proposed algorithms can achieve desirable effectiveness and efficiency under a wide range of parameter settings.

Most works of image captioning are implemented under the full supervision of paired image–caption data. Limited to expensive cost of data collection, the task of unpaired image captioning has attracted researchers’ attention. In this article, we propose a novel memorial GAN (MemGAN) with the joint semantic optimization for unpaired image captioning. The core idea is to explore implicit semantic correlation between disjointed images and sentences through building a multimodal semantic-aware space (SAS). Concretely, each modality is mapped into a unified multimodal SAS, where SAS includes the semantic vectors of image ${I}$ , visual concepts ${O}$ , unpaired sentence ${S}$ , and the generated caption ${C}$ . We adopt the memory unit based on multihead attention and relational gate as a backbone to preserve and transit crucial multimodal semantics in the SAS for image caption generation and sentence reconstruction. Then, the memory unit is embedded into a GAN framework to exploit the semantic similarity and relevance in SAS, that is, imposing a joint semantic-aware optimization on SAS without supervision clues. To summarize, the proposed MemGAN learns the latent semantic relevance of SAS’s multimodalities in an adversarial manner. Extensive experiments and qualitative results demonstrate the effectiveness of MemGAN, achieving improvements over state of the arts on unpaired image captioning benchmarks.

Face parsing aims to assign pixel-wise semantic labels to different facial components (e.g., hair, brows, and lips) in given face images. However, directly predicting pixel-level labels for each facial component over the whole face image would obtain limited accuracy, especially for tiny facial components. To address this problem, some recent works propose to first crop tiny patches from the whole face image and then predict masks for each facial component. However, such cropping-and-segmenting strategy consists of two independent stages, which cannot be jointly optimized. Besides, as one valuable piece of information for parsing the highly structured facial components, context cues are not elaborately explored by the existing works. To address these issues, we propose a component-level refinement network (CLRNet) for precisely segmenting out each facial component. Specifically, we introduce an attention mechanism to bridge the two independent stages together and form an end-to-end trainable pipeline for face parsing. Furthermore, we incorporate the global context information into the refining process for each cropped facial component patch, providing informative cues for accurate parsing. Extensive experiments are carried out on two benchmark datasets, LFW-PL and HELEN. The results demonstrate the superiority of the proposed CLRNet over other state-of-the-art methods, especially for tiny facial components.

Categorizing aerial photographs with varied weather/lighting conditions and sophisticated geomorphic factors is a key module in autonomous navigation, environmental evaluation, and so on. Previous image recognizers cannot fulfill this task due to three challenges: 1) localizing visually/semantically salient regions within each aerial photograph in a weakly annotated context due to the unaffordable human resources required for pixel-level annotation; 2) aerial photographs are generally with multiple informative attributes (e.g., clarity and reflectivity), and we have to encode them for better aerial photograph modeling; and 3) designing a cross-domain knowledge transferal module to enhance aerial photograph perception since multiresolution aerial photographs are taken asynchronistically and are mutually complementary. To handle the above problems, we propose to optimize aerial photograph’s feature learning by leveraging the low-resolution spatial composition to enhance the deep learning of perceptual features with a high resolution. More specifically, we first extract many BING-based object patches (Cheng et al., 2014) from each aerial photograph. A weakly supervised ranking algorithm selects a few semantically salient ones by seamlessly incorporating multiple aerial photograph attributes. Toward an interpretable aerial photograph recognizer indicative to human visual perception, we construct a gaze shifting path (GSP) by linking the top-ranking object patches and, subsequently, derive the deep GSP feature. Finally, a cross-domain multilabel SVM is formulated to categorize each aerial photograph. It leverages the global feature from low-resolution counterparts to optimize the deep GSP feature from a high-resolution aerial photograph. Comparative results on our compiled million-scale aerial photograph set have demonstrated the competitiveness of our approach. Besides, the eye-tracking experiment has shown that our ranking-based GSPs are over 92% consistent with the real human gaze shifting sequences.

Searching for rare astronomical objects based on spectral data is similar to finding needles in a haystack owing to their rarity and the immense data volume gathered from large astronomical spectroscopic surveys. In this paper, we propose a novel automated approximate nearest neighbor search method based on unsupervised hashing learning for rare spectra retrieval. The proposed method employs a multilayer neural network using autoencoders as the local compact feature extractors. Autoencoders are trained with a non-gradient learning algorithm with graph Laplace regularization. This algorithm also simplifies the tuning of network architecture hyperparameters and the learning control hyperparameters. Meanwhile, the graph Laplace regularization can enhance the robustness by reducing the sensibility to noise. The proposed model is data-driven; thus, it can be viewed as a general-purpose retrieval model. The proposed model is evaluated in experiments and real-world applications where rare O-type stars and their subclass are retrieved from the dataset obtained from the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (Guo Shoujing Telescope). The experimental and application results show that the proposed model outperformed the baseline methods, demonstrating the effectiveness of the proposed method in rare spectra retrieval tasks.

Trajectory prediction is a fundamental and challenging task for numerous applications, such as autonomous driving and intelligent robots. Current works typically treat pedestrian trajectories as a series of 2D point coordinates. However, in real scenarios, the trajectory often exhibits randomness, and has its own probability distribution. Inspired by this observation and other movement characteristics of pedestrians, we propose a simple and intuitive movement description called a trajectory distribution, which maps the coordinates of the pedestrian trajectory to a 2D Gaussian distribution in space. Based on this novel description, we develop a new trajectory prediction method, which we call the social probability method. The method combines trajectory distributions and powerful convolutional recurrent neural networks. Both the input and output of our method are trajectory distributions, which provide the recurrent neural network with sufficient spatial and random information about moving pedestrians. Furthermore, the social probability method extracts spatio-temporal features directly from the new movement description to generate robust and accurate predictions. Experiments on public benchmark datasets show the effectiveness of the proposed method.

In this article, we propose a transformer-based RGB-D egocentric action recognition framework, called Trear. It consists of two modules: 1) interframe attention encoder and 2) mutual-attentional fusion block. Instead of using optical flow or recurrent units, we adopt a self-attention mechanism to model the temporal structure of the data from different modalities. Input frames are cropped randomly to mitigate the effect of the data redundancy. Features from each modality are interacted through the proposed fusion block and combined through a simple yet effective fusion operation to produce a joint RGB-D representation. Empirical experiments on two large egocentric RGB-D data sets: 1) THU-READ and 2) first-person hand action, and one small data set, wearable computer vision systems, have shown that the proposed method outperforms the state-of-the-art results by a large margin.

Autophagy, a crucial pathway for the degradation of proteins in eukaryotic cells, is linked to the development of Alzheimer’s disease (AD), and the accumulated autophagosomes in the cells resulting in the death of cells. Sevoflurane can impair spatial learning and memory in mice with AD and lead to the apoptosis of nerve cells; however, the underlying mechanisms remain unknown. We aim to explore the effects and underlying mechanisms of sevoflurane in APPswe/PS1ΔE9 double-transgenic mice. 51 heterozygous APPswe/PS1ΔE9 double-transgenic mice were involved and divided into three groups, including control group, sham group and sevoflurane group. Morris water maze experiment was used to test the learning and memory abilities of mice, flow cytometry was conducted to detect apoptosis and mitochondrial membrane potential of brain cells in mice, transmission electron microscopy was used to observe the number of autophagosomes at the axon in mice, and western blot was carried out to detect the expression of Bax, Bcl-2, LC3II, P62, KIF3B and DIC proteins of brain cells in mice. In our study, we found that significantly longer escape latencies, fewer crossings of the platform and shorter time spent in the target quadrant of the morris water maze experiment in the sevoflurane group. Flow cytometry showed cellular apoptosis was increased and the membrane potential of the mitochondria was reduced of brain cells in the sevoflurane group. Transmission electron microscopy displayed that there was a remarkable upregulation of autophagosomes at the axon of brain cells in mice after treatment of sevoflurane. Western blot demonstrated that the expression of Bax, LC3II, P62 and KIF3B proteins were elevated, and the expression of Bcl-2 and DIC proteins were reduced in the sevoflurane group. Sevoflurane impaired acquisition learning and memory function, promoted the apoptosis of hippocampal neurons in APPswe/PS1ΔE9 double-transgenic mice, and the mechanism might be related to the activation of autophagy along with the disruption of autophagosomes retrograde transport in axons.

As an interesting and important problem in computer vision, learning-based video saliency detection aims to discover the visually interesting regions in a video sequence. Capturing the information within frame and between frame at different aspects (such as spatial contexts, motion information, temporal consistency across frames, and multiscale representation) is important for this task. A key issue is how to jointly model all these factors within a unified data-driven scheme in an end-to-end fashion. In this article, we propose an end-to-end spatiotemporal deep video saliency detection approach, which captures the information on spatial contexts and motion characteristics. Furthermore, it encodes the temporal consistency information across the consecutive frames by implementing a convolutional long short-term memory (Conv-LSTM) model. In addition, the multiscale saliency properties for each frame are adaptively integrated for final saliency prediction in a collaborative feature-pyramid way. Finally, the proposed deep learning approach unifies all the aforementioned parts into an end-to-end joint deep learning scheme. Experimental results demonstrate the effectiveness of our approach in comparison with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Grain water content (GWC) is a key determinant for mechanical harvesting of maize (Zea mays). In our previous research, we identified a quantitative trait locus, qGWC1, associated with GWC in maize. Here, we examined near-isogenic lines (NILs) NILLand NILHthat differed at the qGWC1locus. Lower GWC in NILLwas primarily attributed to reduced grain water weight (GWW) and smaller fresh grain size, rather than the accumulation of dry matter. The difference in GWC between the NILs became more pronounced approximately 35 d after pollination (DAP), arising from a faster dehydration rate in NILL. Through an integrated analysis of the transcriptome, proteome, and metabolome, coupled with an examination of hormones and their derivatives, we detected a marked decrease in JA, along with an increase in cytokinin, storage forms of IAA (IAA-Glu, IAA-ASP), and IAA precursor IPA in immature NILLkernels. During kernel development, genes associated with sucrose synthases, starch biosynthesis, and zein production in NILL, exhibited an initial up-regulation followed by a gradual down-regulation, compared to those in NILH. This discovery highlights the crucial role of phytohormone homeostasis and genes related to kernel development in balancing GWC and dry matter accumulation in maize kernels.

Despite the competitive prediction performance, recent deep image quality models suffer from the following limitations. First, it is deficiently effective to interpret and quantify the region-level quality, which contributes to global features during deep architecture training. Second, human visual perception is sensitive to compositional features (i.e., the sophisticated spatial configurations among regions), but explicitly incorporating them into a deep model is challenging. Third, the state-of-the-art deep quality models typically use rectangular image patches as inputs, but there is no evidence that these rectangles can reflect arbitrarily shaped objects, such as beaches and jungles. By defining the complet, which is a set of image segments collaboratively characterizing the spatial/geometric distribution of multiple visual elements, we propose a novel quality-modeling framework that involves two key modules: a complet ranking algorithm and a spatially-aware dual aggregation network (SDA-Net). Specifically, to describe the region-level quality features, we build complets to characterize the high-order spatial interactions among the arbitrarily shaped segments in each image. To obtain complets that are highly descriptive to image compositions, a weakly supervised complet ranking algorithm is designed by quantifying the quality of each complet. The algorithm seamlessly encodes three factors: the image-level quality discrimination, weakly supervised constraint, and complet geometry of each image. Based on the top-ranking complets, a novel multi-column convolutional neural network (CNN) called SDA-Net is designed, which supports input segments with arbitrary shapes. The key is a dual-aggregation mechanism that fuses the intracomplet deep features and the intercomplet deep features under a unified framework. Thorough experimental validations on a series of benchmark data sets demonstrated the superiority of our method. [ABSTRACT FROM AUTHOR]

People in crowd scenes often exhibit the characteristic of imbalanced distribution. On the one hand, people size varies largely due to the camera perspective. People far away from the camera look smaller and are likely to occlude each other, whereas people near to the camera look larger and are relatively sparse. On the other hand, the number of people also varies greatly in the same or different scenes. This article aims to develop a novel model that can accurately estimate the crowd count from a given scene with imbalanced people distribution. To this end, we have proposed an effective multi-level convolutional neural network (MLCNN) architecture that first adaptively learns multi-level density maps and then fuses them to predict the final output. Density map of each level focuses on dealing with people of certain sizes. As a result, the fusion of multi-level density maps is able to tackle the large variation in people size. In addition, we introduce a new loss function named balanced loss (BL) to impose relatively BL feedback during training, which helps further improve the performance of the proposed network. Furthermore, we introduce a new data set including 1111 images with a total of 49 061 head annotations. MLCNN is easy to train with only one end-to-end training stage. Experimental results demonstrate that our MLCNN achieves state-of-the-art performance. In particular, our MLCNN reaches a mean absolute error (MAE) of 242.4 on the UCF_CC_50 data set, which is 37.2 lower than the second-best result.

Disk Filtration (DF) Malware can attack air-gapped computers. However, none of the existing technique can detect DF attacks. To address this problem, a method for detecting the DF attacks based on the fourteen Machine Learning (ML) algorithms is proposed in this paper. First, we collect a number of data about Power Spectral Density (PSD) and frequency of the sound wave from the Hard Disk Drive (HDD). Second, the corresponding machine learning models are trained respectively using the collected data. Third, the trained ML models are employed to detect whether a DF attack occurs or not respectively, if given pair of values of PSD and frequency are input. The experimental results show that the max accuracy of detection is greater than or equal to 99.4%.

Visualization and artificial intelligence (AI) are well-applied approaches to data analysis. On one hand, visualization can facilitate humans in data understanding through intuitive visual representation and interactive exploration. On the other hand, AI is able to learn from data and implement bulky tasks for humans. In complex data analysis scenarios, like epidemic traceability and city planning, humans need to understand large-scale data and make decisions, which requires complementing the strengths of both visualization and AI. Existing studies have introduced AI-assisted visualization as AI4VIS and visualization-assisted AI as VIS4AI. However, how can AI and visualization complement each other and be integrated into data analysis processes are still missing. In this paper, we define three integration levels of visualization and AI. The highest integration level is described as the framework of VIS+AI, which allows AI to learn human intelligence from interactions and communicate with humans through visual interfaces. We also summarize future directions of VIS+AI to inspire related studies.

Ribonucleic acid &#x0028 RNA &#x0029 hybridization is widely used in popular RNA simulation software in bioinformatics. However, limited by the exponential computational complexity of combinatorial problems, it is challenging to decide, within an acceptable time, whether a specific RNA hybridization is effective. We hereby introduce a machine learning based technique to address this problem. Sample machine learning &#x0028 ML &#x0029 models tested in the training phase include algorithms based on the boosted tree &#x0028 BT &#x0029 , random forest &#x0028 RF &#x0029 , decision tree &#x0028 DT &#x0029 and logistic regression &#x0028 LR &#x0029, and the corresponding models are obtained. Given the RNA molecular coding training and testing sets, the trained machine learning models are applied to predict the classification of RNA hybridization results. The experiment results show that the optimal predictive accuracies are 96.2&#x0025, 96.6&#x0025, 96.0&#x0025 and 69.8&#x0025 for the RF, BT, DT and LR-based approaches, respectively, under the strong constraint condition, compared with traditional representative methods. Furthermore, the average computation efficiency of the RF, BT, DT and LR-based approaches are 208 679, 269 756, 184 333 and 187 458 times higher than that of existing approach, respectively. Given an RNA design, the BT-based approach demonstrates high computational efficiency and better predictive accuracy in determining the biological effectiveness of molecular hybridization.

A pressing/sintering method was used for the stable solidification of spent cesium adsorbent by adding natural allophane and mordenite. The stable Cs4Al4Si20O48crystalline phase was recrystallized by the reaction of Cs2O, Al2O3, and SiO2above 1000 °C, and the Cs immobilization ratio of approximately 100% was maintained. The solidified products exhibited excellent leaching resistance in distilled water and 0.1 M NaCl. The leaching rate of Cs at 90 °C was in the order of H2O ≈ 0.1 M NaCl ≪ 0.1 M HNO3< 0.1 M NaOH.A press/sintering method was proposed for the stable solidification of spent absorbent for Cs (AMP–Cs/SiO2) using allophane and mordenite. The pyrolysis of Cs3PMo12O40resulted the generation of gaseous Cs2O. At higher temperature, the allophane and mordenite structure begined to collapse gradually. The stable Cs4Al4Si20O48crystalline phase was recrystallized by the reaction of Cs2O, Al2O3, and SiO2above 1000 °C.

Control of flowering time is crucial for reproductive success of cereal crops, and has a significant impact on grain yield as well as adaptation to diverse environmental conditions. Plants integrate signals from both environmental cues and endogenous regulatory pathways to fine-tune flowering time. The CCT domain originally described to a 43-amino acid sequence at the C-terminus of three Arabidopsis proteins, namely CONSTANS (CO), CO-LIKE, and TIMING OF CAB1 (TOC1). The CCT domain-containing genes (CCTgenes), which encode transcription co-factors, are the major genetic determinants that modulate flowering time, and this in turn enables plants to effectively expand their territory to take advantage of favorable habitats. Moreover, certain CCTgenes have pleiotropic effects on morphological traits and confer resistance/tolerance to biotic/abiotic stresses. CCTgenes can be classified into three families, namely COL(CONSTANS-like), PRR(Pseudo-response regulator), and CMF(CCT motif family), based on their non-CCT domains. During domestication, natural and artificial selection resulted in reduced nucleotide diversity of CCTgenes in modern cultivated cereals than their wild types. Here, we review the features and functions of CCTgenes in cereal crops and propose future research to focus on CCTgenes and their utilization in crop breeding.

Plants adaptively change their cell wall composition and structure during their growth, development, and interactions with environmental stresses. Dirigent proteins (DIRs) contribute to environmental adaptations by dynamically reorganizing the cell wall and/or by generating defense compounds. A maize DIR, ZmDRR206, was previously reported to play a dominant role in regulation of storage nutrient accumulation in endosperm during maize kernel development. Here we show that ZmDRR206 mediates maize seedling growth and disease resistance by coordinately regulating biosynthesis of cell wall components for cell-wall integrity (CWI) maintenance. Expression of ZmDRR206was induced in maize seedlings upon pathogen infection. ZmDRR206overexpression in maize resulted in reduced seedling growth and photosynthetic activity but increased disease resistance and drought tolerance, revealing a tradeoff between growth and defense. Consistently, ZmDRR206overexpression reduced the contents of primary metabolites and down-regulated genes involved in photosynthesis, while increasing the contents of major cell wall components, defense phytohormones, and defense metabolites, and up-regulated genes involved in defense and cell-wall biosynthesis in seedlings. ZmDRR206-overexpressing seedlings were resistant to cell-wall stress imposed by isoxaben, and ZmDRR206 physically interacted with ZmCesA10, which is a cellulose synthase unit. Our findings suggest a mechanism by which ZmDRR206 coordinately regulates biosynthesis of cell-wall components for CWI maintenance during maize seedling growth, and might be exploited for breeding strong disease resistance in maize.

Head smut, caused by the fungal pathogen Sporisorium reilianum, poses a grave threat to maize (Zea mays) production worldwide. Here we report cytological and molecular evidence for maize resistance to head smut. During early stages of root infection, S. reilianummycelium was capable of penetrating the root epidermis of both resistant (Ji1037) and susceptible (HZ4) inbred lines. S. reilianumhyphae were observed in the root–stem junction at 6days after inoculation. In an attempt to monitor hyphal spread within the maize plant, a highly specific and sensitive real-time PCR method was established to estimate the hyphal content in infected maize tissues. During the upward growth of endophytic S. reilianum, the extent of hyphal spread was markedly different between Ji1037 and HZ4. Very little or no pathogen was detected in aerial parts of Ji1037, whereas large amounts of pathogen accumulated in aboveground tissues, particularly inflorescences, of HZ4. Thus, maize resistance to S. reilianumwas achieved mainly by inhibition of endophytic hyphal growth rather than by prevention of early-root penetration by the pathogen.

In recent years, crowd counting has increasingly drawn attention due to its widespread applications in the field of computer vision. Most of the existing methods rely on datasets with scarce labeled images to train networks. They are prone to suffer from the over-fitting problem. Further, these existing datasets usually just give manually labeled annotations related to the head center position. This kind of annotation provides limited information. In this paper, we propose to exploit virtual synthetic crowd scenes to improve the performance of the counting network in the real world. Since we can obtain people masks easily in a synthetic dataset, we first learn to distinguish people from the background via a segmentation network using the synthetic data. Then we transfer the learned segmentation priors from synthetic data to real-world data. Finally, we train a density estimation network on real-world data by utilizing the obtained people masks. Our experiments on two crowd counting datasets demonstrate the effectiveness of the proposed method.