Search

The conventional definition of hypergraph has two major issues: (1) there is not a standard definition of directed hypergraph and (2) there is not a formal definition of nested hypergraph. To resolve these issues, we propose a new definition of hypergraph that unifies the concepts of undirected, directed and nested hypergraphs, and that is uniform in using hyperedge as a single construct for representing high-order correlations among things, i.e., nodes and hyperedges. Specifically, we define a hyperedge to be a simple hyperedge, a nesting hyperedge, or a directed hyperedge. With this new definition, a hypergraph is nested if it has nesting hyperedge(s), and is directed if it has directed hyperedge(s). Otherwise, a hypergraph is a simple hypergraph. The uniformity and power of this new definition, with visualization, should facilitate the use of hypergraph for representing (hierarchical) high-order correlations in general and chemical systems in particular. Graph has been widely used as a mathematical structure for machine learning on molecular structures and 3D molecular geometries. However, graph has a major limitation: it can represent only pairwise correlations between nodes. Hypergraph extends graph with high-order correlations among nodes. This extension is significant or essential for machine learning on chemical systems. For molecules, this is significant as it allows the direct, explicit representation of multicenter bonds and molecular substructures. For chemical reactions, this is essential since most chemical reactions involve multiple participants. We propose the use of chemical hypergraph, a multilevel hypergraph with simple, nesting and directed hyperedges, as a single mathematical structure for representing chemical systems. We apply the new definition of hypergraph to chemical hypergraph and, as simplified versions, molecular hypergraph and chemical reaction hypergraph., Comment: arXiv admin note: text overlap with arXiv:2310.03623 by other authors

As part of the recent research effort on quantum natural language processing (QNLP), variational quantum sentence classifiers (VQSCs) have been implemented and supported in lambeq / DisCoPy, based on the DisCoCat model of sentence meaning. We discuss in some detail VQSCs, including category theory, DisCoCat for modeling sentence as string diagram, and DisCoPy for encoding string diagram as parameterized quantum circuit. Many NLP tasks, however, require the handling of text consisting of multiple sentences, which is not supported in lambeq / DisCoPy. A good example is sentiment classification of customer feedback or product review. We discuss three potential approaches to variational quantum text classifiers (VQTCs), in line with VQSCs. The first is a weighted bag-of-sentences approach which treats text as a group of independent sentences with task-specific sentence weighting. The second is a coreference resolution approach which treats text as a consolidation of its member sentences with coreferences among them resolved. Both approaches are based on the DisCoCat model and should be implementable in lambeq / DisCoCat. The third approach, on the other hand, is based on the DisCoCirc model which considers both ordering of sentences and interaction of words in composing text meaning from word and sentence meanings. DisCoCirc makes fundamental modification of DisCoCat since a sentence in DisCoCirc updates meanings of words, whereas all meanings are static in DisCoCat. It is not clear if DisCoCirc can be implemented in lambeq / DisCoCat without breaking DisCoCat.

Large Language Models (LLMs) have been successfully used in many natural-language tasks and applications including text generation and AI chatbots. They also are a promising new technology for concept-oriented deep learning (CODL). However, the prerequisite is that LLMs understand concepts and ensure conceptual consistency. We discuss these in this paper, as well as major uses of LLMs for CODL including concept extraction from text, concept graph extraction from text, and concept learning. Human knowledge consists of both symbolic (conceptual) knowledge and embodied (sensory) knowledge. Text-only LLMs, however, can represent only symbolic (conceptual) knowledge. Multimodal LLMs, on the other hand, are capable of representing the full range (conceptual and sensory) of human knowledge. We discuss conceptual understanding in visual-language LLMs, the most important multimodal LLMs, and major uses of them for CODL including concept extraction from image, concept graph extraction from image, and concept learning. While uses of LLMs for CODL are valuable standalone, they are particularly valuable as part of LLM applications such as AI chatbots.

Quantum kernel methods, i.e., kernel methods with quantum kernels, offer distinct advantages as a hybrid quantum-classical approach to quantum machine learning (QML), including applicability to Noisy Intermediate-Scale Quantum (NISQ) devices and usage for solving all types of machine learning problems. Kernel methods rely on the notion of similarity between points in a higher (possibly infinite) dimensional feature space. For machine learning, the notion of similarity assumes that points close in the feature space should be close in the machine learning task space. In this paper, we discuss the use of variational quantum kernels with task-specific quantum metric learning to generate optimal quantum embeddings (a.k.a. quantum feature encodings) that are specific to machine learning tasks. Such task-specific optimal quantum embeddings, implicitly supporting feature selection, are valuable not only to quantum kernel methods in improving the latter's performance, but they can also be valuable to non-kernel QML methods based on parameterized quantum circuits (PQCs) as pretrained embeddings and for transfer learning. This further demonstrates the quantum utility, and quantum advantage (with classically-intractable quantum embeddings), of quantum kernel methods.

Quantum machine learning (QML) is the use of quantum computing for the computation of machine learning algorithms. With the prevalence and importance of classical data, a hybrid quantum-classical approach to QML is called for. Parameterized Quantum Circuits (PQCs), and particularly Quantum Kernel PQCs, are generally used in the hybrid approach to QML. In this paper we discuss some important aspects of PQCs with quantum kernels including PQCs, quantum kernels, quantum kernels with quantum advantage, and the trainability of quantum kernels. We conclude that quantum kernels with hybrid kernel methods, a.k.a. quantum kernel methods, offer distinct advantages as a hybrid approach to QML. Not only do they apply to Noisy Intermediate-Scale Quantum (NISQ) devices, but they also can be used to solve all types of machine learning problems including regression, classification, clustering, and dimension reduction. Furthermore, beyond quantum utility, quantum advantage can be attained if the quantum kernels, i.e., the quantum feature encodings, are classically intractable.

Deep learning for molecular science has so far mainly focused on 2D molecular graphs. Recently, however, there has been work to extend it to 3D molecular geometry, due to its scientific significance and critical importance in real-world applications. The 3D distance-geometric graph representation (DG-GR) adopts a unified scheme (distance) for representing the geometry of 3D graphs. It is invariant to rotation and translation of the graph, and it reflects pair-wise node interactions and their generally local nature, particularly relevant for 3D molecular geometry. To facilitate the incorporation of 3D molecular geometry in deep learning for molecular science, we adopt the new graph attention network with dynamic attention (GATv2) for use with DG-GR and propose the 3D distance-geometric graph attention network (DG-GAT). GATv2 is a great fit for DG-GR since the attention can vary by node and by distance between nodes. Experimental results of DG-GAT for the ESOL and FreeSolv datasets show major improvement (31% and 38%, respectively) over those of the standard graph convolution network based on 2D molecular graphs. The same is true for the QM9 dataset. Our work demonstrates the utility and value of DG-GAT for deep learning based on 3D molecular geometry., Comment: arXiv admin note: substantial text overlap with arXiv:2006.01785, arXiv:2007.03513

Dual embodied-symbolic concept representations are the foundation for deep learning and symbolic AI integration. We discuss the use of dual embodied-symbolic concept representations for molecular graph representation learning, specifically with exemplar-based contrastive self-supervised learning (SSL). The embodied representations are learned from molecular graphs, and the symbolic representations are learned from the corresponding Chemical knowledge graph (KG). We use the Chemical KG to enhance molecular graphs with symbolic (semantic) knowledge and generate their augmented molecular graphs. We treat a molecular graph and its semantically augmented molecular graph as exemplars of the same semantic class, and use the pairs as positive pairs in exemplar-based contrastive SSL.

Motivated by recent findings from cognitive neural science, we advocate the use of a dual-level model for concept representations: the embodied level consists of concept-oriented feature representations, and the symbolic level consists of concept graphs. Embodied concept representations are modality specific and exist in the form of feature vectors in a feature space. Symbolic concept representations, on the other hand, are amodal and language specific, and exist in the form of word / knowledge-graph embeddings in a concept / knowledge space. The human conceptual system comprises both embodied representations and symbolic representations, which typically interact to drive conceptual processing. As such, we further advocate the use of dual embodied-symbolic concept representations for deep learning. To demonstrate their usage and value, we discuss two important use cases: embodied-symbolic knowledge distillation for few-shot class incremental learning, and embodied-symbolic fused representation for image-text matching. Dual embodied-symbolic concept representations are the foundation for deep learning and symbolic AI integration. We discuss two important examples of such integration: scene graph generation with knowledge graph bridging, and multimodal knowledge graphs.

Humans are capable of learning new concepts from only a few (labeled) exemplars, incrementally and continually. This happens within the context that we can differentiate among the exemplars, and between the exemplars and large amounts of other data (unlabeled and labeled). This suggests, in human learning, supervised learning of concepts based on exemplars takes place within the larger context of contrastive self-supervised learning (CSSL) based on unlabeled and labeled data. We discuss extending CSSL (1) to be based mainly on exemplars and only secondly on data augmentation, and (2) to apply to both unlabeled data (a large amount is available in general) and labeled data (a few exemplars can be obtained with valuable supervised knowledge). A major benefit of the extensions is that exemplar-based CSSL, with supervised finetuning, supports few-shot class incremental learning (CIL). Specifically, we discuss exemplar-based CSSL including: nearest-neighbor CSSL, neighborhood CSSL with supervised pretraining, and exemplar CSSL with supervised finetuning. We further discuss using exemplar-based CSSL to facilitate few-shot learning and, in particular, few-shot CIL.

BackgroundElderly patients with gastrointestinal cancer and mental illness have significant comorbidities that can impact the quality of their care. We investigated the relationship between mental illness and frequent emergency department (ED) use in the last month of life, an indicator for poor end-of-life care quality, among elderly patients with gastrointestinal cancers.MethodsWe used SEER-Medicare data to identify decedents with gastrointestinal cancers who were diagnosed between 2004 and 2013 and were at least 66 years old at time of diagnosis (median age: 80 years, range: 66-117 years). We evaluated the association between having a diagnosis of depression, bipolar disorders, psychotic disorders, anxiety, dementia, and/or substance use disorders and ED use in the last 30 days of life using logistic regression models.ResultsOf 160,367 patients included, 54,661 (34.1%) had a mental illness diagnosis between one year prior to cancer diagnosis and death. Patients with mental illness were more likely to have > 1 ED visit in the last 30 days of life (15.6% vs. 13.3%, p < 0.01). ED use was highest among patients with substance use (17.7%), bipolar (16.5%), and anxiety disorders (16.4%). Patients with mental illness who were male, younger, non-white, residing in lower income areas, and with higher comorbidity were more likely to have multiple end-of-life ED visits. Patients who received outpatient treatment from a mental health professional were less likely to have multiple end-of-life ED visits (adjusted odds ratio 0.82, 95% confidence interval 0.78-0.87).ConclusionsIn elderly patients with gastrointestinal cancers, mental illness is associated with having multiple end-of-life ED visits. Increasing access to mental health services may improve quality of end-of-life care in this vulnerable population.

BackgroundElderly patients with gastrointestinal cancer and mental illness have significant comorbidities that can impact the quality of their care. We investigated the relationship between mental illness and frequent emergency department (ED) use in the last month of life, an indicator for poor end-of-life care quality, among elderly patients with gastrointestinal cancers.MethodsWe used SEER-Medicare data to identify decedents with gastrointestinal cancers who were diagnosed between 2004 and 2013 and were at least 66 years old at time of diagnosis (median age: 80 years, range: 66-117 years). We evaluated the association between having a diagnosis of depression, bipolar disorders, psychotic disorders, anxiety, dementia, and/or substance use disorders and ED use in the last 30 days of life using logistic regression models.ResultsOf 160,367 patients included, 54,661 (34.1%) had a mental illness diagnosis between one year prior to cancer diagnosis and death. Patients with mental illness were more likely to have > 1 ED visit in the last 30 days of life (15.6% vs. 13.3%, p < 0.01). ED use was highest among patients with substance use (17.7%), bipolar (16.5%), and anxiety disorders (16.4%). Patients with mental illness who were male, younger, non-white, residing in lower income areas, and with higher comorbidity were more likely to have multiple end-of-life ED visits. Patients who received outpatient treatment from a mental health professional were less likely to have multiple end-of-life ED visits (adjusted odds ratio 0.82, 95% confidence interval 0.78-0.87).ConclusionsIn elderly patients with gastrointestinal cancers, mental illness is associated with having multiple end-of-life ED visits. Increasing access to mental health services may improve quality of end-of-life care in this vulnerable population.

Concept-oriented deep learning (CODL) is a general approach to meet the future challenges for deep learning: (1) learning with little or no external supervision, (2) coping with test examples that come from a different distribution than the training examples, and (3) integrating deep learning with symbolic AI. In CODL, as in human learning, concept representations are learned based on concept exemplars. Contrastive self-supervised learning (CSSL) provides a promising approach to do so, since it: (1) uses data-driven associations, to get away from semantic labels, (2) supports incremental and continual learning, to get away from (large) fixed datasets, and (3) accommodates emergent objectives, to get away from fixed objectives (tasks). We discuss major aspects of concept representation learning using CSSL. These include dual-level concept representations, CSSL for feature representations, exemplar similarity measures and self-supervised relational reasoning, incremental and continual CSSL, and contrastive self-supervised concept (class) incremental learning. The discussion leverages recent findings from cognitive neural science and CSSL.

© 2020 Elsevier Inc. Obesity is an intrinsic driver of PDAC in mice, leading to a remodeling of beta cells to increase CCK secretion and playing a role early in pancreatic cancer development that can be intercepted by weight loss.

Bayesian neural networks provide a direct and natural way to extend standard deep neural networks to support probabilistic deep learning through the use of probabilistic layers that, traditionally, encode weight (and bias) uncertainty. In particular, hybrid Bayesian neural networks utilize standard deterministic layers together with few probabilistic layers judicially positioned in the networks for uncertainty estimation. A major aspect and benefit of Bayesian inference is that priors, in principle, provide the means to encode prior knowledge for use in inference and prediction. However, it is difficult to specify priors on weights since the weights have no intuitive interpretation. Further, the relationships of priors on weights to the functions computed by networks are difficult to characterize. In contrast, functions are intuitive to interpret and are direct since they map inputs to outputs. Therefore, it is natural to specify priors on functions to encode prior knowledge, and to use them in inference and prediction based on functions. To support this, we propose hybrid Bayesian neural networks with functional probabilistic layers that encode function (and activation) uncertainty. We discuss their foundations in functional Bayesian inference, functional variational inference, sparse Gaussian processes, and sparse variational Gaussian processes. We further perform few proof-of-concept experiments using GPflus, a new library that provides Gaussian process layers and supports their use with deterministic Keras layers to form hybrid neural network and Gaussian process models.

Bayesian neural networks utilize probabilistic layers that capture uncertainty over weights and activations, and are trained using Bayesian inference. Since these probabilistic layers are designed to be drop-in replacement of their deterministic counter parts, Bayesian neural networks provide a direct and natural way to extend conventional deep neural networks to support probabilistic deep learning. However, it is nontrivial to understand, design and train Bayesian neural networks due to their complexities. We discuss the essentials of Bayesian neural networks including duality (deep neural networks, probabilistic models), approximate Bayesian inference, Bayesian priors, Bayesian posteriors, and deep variational learning. We use TensorFlow Probability APIs and code examples for illustration. The main problem with Bayesian neural networks is that the architecture of deep neural networks makes it quite redundant, and costly, to account for uncertainty for a large number of successive layers. Hybrid Bayesian neural networks, which use few probabilistic layers judicially positioned in the networks, provide a practical solution.

Probabilistic deep learning is deep learning that accounts for uncertainty, both model uncertainty and data uncertainty. It is based on the use of probabilistic models and deep neural networks. We distinguish two approaches to probabilistic deep learning: probabilistic neural networks and deep probabilistic models. The former employs deep neural networks that utilize probabilistic layers which can represent and process uncertainty; the latter uses probabilistic models that incorporate deep neural network components which capture complex non-linear stochastic relationships between the random variables. We discuss some major examples of each approach including Bayesian neural networks and mixture density networks (for probabilistic neural networks), and variational autoencoders, deep Gaussian processes and deep mixed effects models (for deep probabilistic models). TensorFlow Probability is a library for probabilistic modeling and inference which can be used for both approaches of probabilistic deep learning. We include its code examples for illustration., Comment: arXiv admin note: text overlap with arXiv:1811.06622

The clinical and financial effects of mental disorders are largely unknown among gastrointestinal (GI) cancer patients. Using the Surveillance, Epidemiology, and End Results (SEER)-Medicare linked database, we identified patients whose first cancer was a primary colorectal, pancreatic, gastric, hepatic/biliary, esophageal, or anal cancer as well as those with coexisting depression, anxiety, psychotic, or bipolar disorder. Survival, chemotherapy use, total healthcare expenditures, and patient out-of-pocket expenditures were estimated and compared based on the presence of a mental disorder. We identified 112,283 patients, 23,726 (21%) of whom had a coexisting mental disorder. Median survival for patients without a mental disorder was 52 months (95% CI 50-53 months) and for patients with a mental disorder was 43 months (95% CI 42-44 months) (p < 0.001). Subgroup analysis identified patients with colorectal, gastric, or anal cancer to have a significant association between survival and presence of a mental disorder. Chemotherapy use was lower among patients with a mental disorder within regional colorectal cancer (43% vs. 41%, p = 0.01) or distant colorectal cancer subgroups (71% vs. 63%, p < 0.0001). The mean total healthcare expenditures were higher for patients with a mental disorder in first year following the cancer diagnosis (increase of $16,823, 95% CI $15,777-$18,173), and mean patient out-of-pocket expenses were also higher (increase of $1,926, 95% CI $1753-$2091). There are a substantial number of GI cancer patients who have a coexisting mental disorder, which is associated with inferior survival, higher healthcare expenditures, and greater personal financial burden.

The clinical and financial effects of mental disorders are largely unknown among gastrointestinal (GI) cancer patients. Using the Surveillance, Epidemiology, and End Results (SEER)-Medicare linked database, we identified patients whose first cancer was a primary colorectal, pancreatic, gastric, hepatic/biliary, esophageal, or anal cancer as well as those with coexisting depression, anxiety, psychotic, or bipolar disorder. Survival, chemotherapy use, total healthcare expenditures, and patient out-of-pocket expenditures were estimated and compared based on the presence of a mental disorder. We identified 112,283 patients, 23,726 (21%) of whom had a coexisting mental disorder. Median survival for patients without a mental disorder was 52 months (95% CI 50-53 months) and for patients with a mental disorder was 43 months (95% CI 42-44 months) (p < 0.001). Subgroup analysis identified patients with colorectal, gastric, or anal cancer to have a significant association between survival and presence of a mental disorder. Chemotherapy use was lower among patients with a mental disorder within regional colorectal cancer (43% vs. 41%, p = 0.01) or distant colorectal cancer subgroups (71% vs. 63%, p < 0.0001). The mean total healthcare expenditures were higher for patients with a mental disorder in first year following the cancer diagnosis (increase of $16,823, 95% CI $15,777-$18,173), and mean patient out-of-pocket expenses were also higher (increase of $1,926, 95% CI $1753-$2091). There are a substantial number of GI cancer patients who have a coexisting mental disorder, which is associated with inferior survival, higher healthcare expenditures, and greater personal financial burden.

The distance-geometric graph representation adopts a unified scheme (distance) for representing the geometry of three-dimensional(3D) graphs. It is invariant to rotation and translation of the graph and it reflects pair-wise node interactions and their generally local nature. To facilitate the incorporation of geometry in deep learning on 3D graphs, we propose a message-passing graph convolutional network based on the distance-geometric graph representation: DG-GCN (distance-geometric graph convolution network). It utilizes continuous-filter convolutional layers, with filter-generating networks, that enable learning of filter weights from distances, thereby incorporating the geometry of 3D graphs in graph convolutions. Our results for the ESOL and FreeSolv datasets show major improvement over those of standard graph convolutions. They also show significant improvement over those of geometric graph convolutions employing edge weight / edge distance power laws. Our work demonstrates the utility and value of DG-GCN for end-to-end deep learning on 3D graphs, particularly molecular graphs., Comment: arXiv admin note: substantial text overlap with arXiv:2006.01785

The geometry of three-dimensional (3D) graphs, consisting of nodes and edges, plays a crucial role in many important applications. An excellent example is molecular graphs, whose geometry influences important properties of a molecule including its reactivity and biological activity. To facilitate the incorporation of geometry in deep learning on 3D graphs, we define three types of geometric graph representations: positional, angle-geometric and distance-geometric. For proof of concept, we use the distance-geometric graph representation for geometric graph convolutions. Further, to utilize standard graph convolution networks, we employ a simple edge weight / edge distance correlation scheme, whose parameters can be fixed using reference values or determined through Bayesian hyperparameter optimization. The results of geometric graph convolutions, for the ESOL and Freesol datasets, show significant improvement over those of standard graph convolutions. Our work demonstrates the feasibility and promise of incorporating geometry, using the distance-geometric graph representation, in deep learning on 3D graphs.

Purpose: Radiation therapy (RT) is prescribed for curative and palliative treatment for around 50% of patients with solid tumors. Radiation-induced toxicities of healthy organs accompany many RTs and represent one of the main limiting factors during dose delivery. The existing RT planning solutions generally discard spatial dose distribution information and lose the ability to recognize radiosensitive regions of healthy organs potentially linked to toxicity manifestation. This study proposes a universal deep learning-based algorithm for recognitions of consistent dose patterns and generation of toxicity risk maps for the abdominal area. Methods: We investigated whether convolutional neural networks (CNNs) can automatically associate abdominal computed tomography (CT) images and RT dose plans with post-RT toxicities without being provided segmentation of abdominal organs. The CNNs were also applied to study RT plans, where doses at specific anatomical regions were reduced/increased, with the aim to pinpoint critical regions sparing of which significantly reduces toxicity risks. The obtained risk maps were computed for individual anatomical regions inside the liver and statistically compared to the existing clinical studies. Results: A database of 122 liver stereotactic body RT (SBRT) executed at Stanford Hospital from July 2004 and November 2015 was assembled. All patients treated for primary liver cancer, mainly hepatocellular carcinoma and cholangiocarcinoma, with complete follow-ups were extracted from the database. The SBRT treatment doses ranged from 26 to 50 Gy delivered in 1–5 fractions for primary liver cancer. The patients were followed up for 1–68 months depending on the survival time. The CNNs were trained to recognize acute and late grade 3+ biliary stricture/obstruction, hepatic failure or decompensation, hepatobiliary infection, liver function test (LFT) elevation or/and portal vein thrombosis, named for convenience hepatobiliary (HB) toxicities. The

Purpose: To develop a framework for automated prediction of hepatobiliary (HB) toxicity after liver stereotactic body radiation therapy (SBRT). Materials and methods: A newly recognized toxicity type, named central or HB liver toxicity, had been reported, manifestation of which strongly correlates with the dose delivered to portal vein (PV) during SBRT. We propose a novel framework for automated HB toxicity prediction by combining deep learning-based auto-segmentation, PV anatomy analysis and the previously reported HB toxicity model. For validation of the framework, an IBR approved representative database of 72 patients treated with SBRT from primary (37) and metastatic (35) liver cancer was assembled. Each case included a pre-treatment CT, manual segmentations of tumor and PV, approved treatment plan, and the record of acute and late post-treatment toxicities. Performance of the developed framework was evaluated by quantitative comparison against manual predictions of HB toxicity, as well as post-treatment toxicity follow-ups. Results: The manual and automated predictions of HB toxicity were in agreement for 94% cases using either VBED1030 ≥ 45 cc or VBED1040 ≥ 37 cc dosimetric predictors. When compared to post-treatment follow-ups for primary liver cancer, the proposed automated framework made 86% and 83% correct predictions in comparison to 83% and 80% correct manual predictions using VBED1030 ≥ 45 cc or VBED1040 ≥ 37 cc, respectively. Conclusion: The proposed framework automates the HB toxicity prediction with the accuracy similar to manual analysis-based HB toxicity prediction. The strategy is quite general and extendable to the automated prediction of toxicities of other organs.

Deep learning models are full of hyperparameters, which are set manually before the learning process can start. To find the best configuration for these hyperparameters in such a high dimensional space, with time-consuming and expensive model training / validation, is not a trivial challenge. Bayesian optimization is a powerful tool for the joint optimization of hyperparameters, efficiently trading off exploration and exploitation of the hyperparameter space. In this paper, we discuss Bayesian hyperparameter optimization, including hyperparameter optimization, Bayesian optimization, and Gaussian processes. We also review BoTorch, GPyTorch and Ax, the new open-source frameworks that we use for Bayesian optimization, Gaussian process inference and adaptive experimentation, respectively. For experimentation, we apply Bayesian hyperparameter optimization, for optimizing group weights, to weighted group pooling, which couples unsupervised tiered graph autoencoders learning and supervised graph prediction learning for molecular graphs. We find that Ax, BoTorch and GPyTorch together provide a simple-to-use but powerful framework for Bayesian hyperparameter optimization, using Ax's high-level API that constructs and runs a full optimization loop and returns the best hyperparameter configuration.

Tiered graph autoencoders provide the architecture and mechanisms for learning tiered latent representations and latent spaces for molecular graphs that explicitly represent and utilize groups (e.g., functional groups). This enables the utilization and exploration of tiered molecular latent spaces, either individually - the node (atom) tier, the group tier, or the graph (molecule) tier - or jointly, as well as navigation across the tiers. In this paper, we discuss the use of tiered graph autoencoders together with graph prediction for molecular graphs. We show features of molecular graphs used, and groups in molecular graphs identified for some sample molecules. We briefly review graph prediction and the QM9 dataset for background information, and discuss the use of tiered graph embeddings for graph prediction, particularly weighted group pooling. We find that functional groups and ring groups effectively capture and represent the chemical essence of molecular graphs (structures). Further, tiered graph autoencoders and graph prediction together provide effective, efficient and interpretable deep learning for molecular graphs, with the former providing unsupervised, transferable learning and the latter providing supervised, task-optimized learning., Comment: arXiv admin note: text overlap with arXiv:1806.08804 by other authors

Tiered latent representations and latent spaces for molecular graphs provide a simple but effective way to explicitly represent and utilize groups (e.g., functional groups), which consist of the atom (node) tier, the group tier and the molecule (graph) tier. They can be learned using the tiered graph autoencoder architecture. In this paper we discuss adapting tiered graph autoencoders for use with PyTorch Geometric, for both the deterministic tiered graph autoencoder model and the probabilistic tiered variational graph autoencoder model. We also discuss molecular structure information sources that can be accessed to extract training data for molecular graphs. To support transfer learning, a critical consideration is that the information must utilize standard unique molecule and constituent atom identifiers. As a result of using tiered graph autoencoders for deep learning, each molecular graph possesses tiered latent representations. At each tier, the latent representation consists of: node features, edge indices, edge features, membership matrix, and node embeddings. This enables the utilization and exploration of tiered molecular latent spaces, either individually (the node tier, the group tier, or the graph tier) or jointly, as well as navigation across the tiers.

Stereotactic body radiation therapy (SBRT) is a relatively novel treatment modality, with little post-treatment prognostic information reported. This study proposes a novel neural network based paradigm for accurate prediction of liver SBRT outcomes. We assembled a database of patients treated with liver SBRT at our institution. Together with a three-dimensional (3-D) dose delivery plans for each SBRT treatment, other variables such as patients' demographics, quantified abdominal anatomy, history of liver comorbidities, other liver-directed therapies, and liver function tests were collected. We developed a multi-path neural network with the convolutional path for 3-D dose plan analysis and fully connected path for other variables analysis, where the network was trained to predict post-SBRT survival and local cancer progression. To enhance the network robustness, it was initially pre-trained on a large database of computed tomography images. Following n-fold cross-validation, the network automatically identified patients that are likely to have longer survival or late cancer recurrence, i.e., patients with the positive predicted outcome (PPO) of SBRT, and vice versa, i.e., negative predicted outcome (NPO). The predicted results agreed with actual SBRT outcomes with 56% of PPO patients and 0% NPO patients with primary liver cancer survived more than two years after SBRT. Similarly, 82% of PPO patients and 0% of NPO patients with metastatic liver cancer survived two-year threshold. The obtained results were superior to the performance of support vector machine and random forest classifiers. Furthermore, the network was able to identify the critical-to-spare liver regions, and the critical clinical features associated with the highest risks of negative SBRT outcomes.

Molecular graphs generally contain subgraphs (known as groups) that are identifiable and significant in composition, functionality, geometry, etc. Flat latent representations (node embeddings or graph embeddings) fail to represent, and support the use of, groups. Fully hierarchical latent representations, on the other hand, are difficult to learn and, even if learned, may be too complex to use or interpret. We propose tiered latent representations and latent spaces for molecular graphs as a simple way to explicitly represent and utilize groups, which consist of the atom (node) tier, the group tier and the molecule (graph) tier. Specifically, we propose an architecture for learning tiered latent representations and latent spaces using graph autoencoders, graph neural networks, differentiable group pooling and the membership matrix. We discuss its various components, major challenges and related work, for both a deterministic and a probabilistic model. We also briefly discuss the usage and exploration of tiered latent spaces. The tiered approach is applicable to other types of structured graphs similar in nature to molecular graphs.

Probabilistic generative deep learning for molecular design involves the discovery and design of new molecules and analysis of their structure, properties and activities by probabilistic generative models using the deep learning approach. It leverages the existing huge databases and publications of experimental results, and quantum-mechanical calculations, to learn and explore molecular structure, properties and activities. We discuss the major components of probabilistic generative deep learning for molecular design, which include molecular structure, molecular representations, deep generative models, molecular latent representations and latent space, molecular structure-property and structure-activity relationships, molecular similarity and molecular design. We highlight significant recent work using or applicable to this new approach.

Concepts are the foundation of human deep learning, understanding, and knowledge integration and transfer. We propose concept-oriented deep learning (CODL) which extends (machine) deep learning with concept representations and conceptual understanding capability. CODL addresses some of the major limitations of deep learning: interpretability, transferability, contextual adaptation, and requirement for lots of labeled training data. We discuss the major aspects of CODL including concept graph, concept representations, concept exemplars, and concept representation learning systems supporting incremental and continual learning., Comment: 11 pages

Latent representations are the essence of deep generative models and determine their usefulness and power. For latent representations to be useful as generative concept representations, their latent space must support latent space interpolation, attribute vectors and concept vectors, among other things. We investigate and discuss latent variable modeling, including latent variable models, latent representations and latent spaces, particularly hierarchical latent representations and latent space vectors and geometry. Our focus is on that used in variational autoencoders and generative adversarial networks., Comment: arXiv admin note: text overlap with arXiv:1706.00400 by other authors

Generative concept representations have three major advantages over discriminative ones: they can represent uncertainty, they support integration of learning and reasoning, and they are good for unsupervised and semi-supervised learning. We discuss probabilistic and generative deep learning, which generative concept representations are based on, and the use of variational autoencoders and generative adversarial networks for learning generative concept representations, particularly for concepts whose data are sequences, structured data or graphs.

The NCCN Guidelines for Hepatobiliary Cancers provide treatment recommendations for cancers of the liver, gallbladder, and bile ducts. The NCCN Hepatobiliary Cancers Panel meets at least annually to review comments from reviewers within their institutions, examine relevant new data from publications and abstracts, and reevaluate and update their recommendations. These NCCN Guidelines Insights summarize the panel's discussion and most recent recommendations regarding locoregional therapy for treatment of patients with hepatocellular carcinoma.

PurposeApproximately one half of the radiotherapy (RT) prescribed in the United States is delivered with palliative intent. The purpose of this study was to investigate the patterns of delivery of palliative RT across the United States.MethodsUsing the Surveillance, Epidemiology, and End Results-Medicare linked database, 51,610 patients were identified with incident stage IV breast, prostate, lung, or colorectal cancer diagnosed between 2000 and 2007 and observed through 2009. Multivariate logistic regression determined predictors of palliative RT.ResultsForty-one percent of the study population received palliative RT, including 53% of patients with lung cancer, followed by those with breast (42%), prostate (40%), and colorectal cancers (12%). Multivariate analysis revealed that older patients (P<.001) and those with higher Charlson comorbidity scores (P<.001) were less likely to receive palliative RT. Black patients with prostate cancer were 20% less likely (P<.001), and black patients with colorectal cancer were 28% less likely (P<.001), than white patients to receive palliative RT. Among those treated with RT, 23% of patients with lung cancer died within 2 weeks of completing treatment, followed by those with colorectal (12%), breast (11%), and prostate cancers (8%). In addition to tumor site, significant predictors (P<.05) of death within 2 weeks of receiving RT included increased age, increased comorbidity, and male sex.ConclusionInequality in the receipt of palliative RT exists among the elderly and patients with comorbid conditions and varies with race. In addition, a significant number of patients die shortly after receiving RT. Understanding these patterns of care, along with further research into the underlying causes, will improve access and quality of palliative RT.

PurposeApproximately one half of the radiotherapy (RT) prescribed in the United States is delivered with palliative intent. The purpose of this study was to investigate the patterns of delivery of palliative RT across the United States.MethodsUsing the Surveillance, Epidemiology, and End Results-Medicare linked database, 51,610 patients were identified with incident stage IV breast, prostate, lung, or colorectal cancer diagnosed between 2000 and 2007 and observed through 2009. Multivariate logistic regression determined predictors of palliative RT.ResultsForty-one percent of the study population received palliative RT, including 53% of patients with lung cancer, followed by those with breast (42%), prostate (40%), and colorectal cancers (12%). Multivariate analysis revealed that older patients (P<.001) and those with higher Charlson comorbidity scores (P<.001) were less likely to receive palliative RT. Black patients with prostate cancer were 20% less likely (P<.001), and black patients with colorectal cancer were 28% less likely (P<.001), than white patients to receive palliative RT. Among those treated with RT, 23% of patients with lung cancer died within 2 weeks of completing treatment, followed by those with colorectal (12%), breast (11%), and prostate cancers (8%). In addition to tumor site, significant predictors (P<.05) of death within 2 weeks of receiving RT included increased age, increased comorbidity, and male sex.ConclusionInequality in the receipt of palliative RT exists among the elderly and patients with comorbid conditions and varies with race. In addition, a significant number of patients die shortly after receiving RT. Understanding these patterns of care, along with further research into the underlying causes, will improve access and quality of palliative RT.

Abhishek A Solanki,1 Daniel T Chang,2 Stanley L Liauw11Department of Radiation and Cellular Oncology, University of Chicago, Chicago, IL, USA; 2Department of Radiation Oncology, Stanford University, Stanford, CA, USAAbstract: Most patients who develop rectal cancer present with locoregionally advanced (T3 or node-positive) disease. The standard management of locoregionally advanced rectal cancer is neoadjuvant concurrent chemoradiotherapy (nCRT), followed by radical resection (low-anterior resection or abdominoperineal resection with total mesorectal excision). Approximately 15% of patients can have a pathologic complete response (pCR) at the time of surgery, indicating that some patients can have no detectable residual disease after nCRT. The actual benefit of surgery in this group of patients is unclear. It is possible that omission of surgery in these patients, termed selective nonoperative management, can limit the toxicities associated with standard, multimodal combined modality therapy without compromising disease control. In this review, we discuss the clinical experiences to date using selective nonoperative management and various attempts at escalation of nCRT to improve the number of patients who have a pCR. We also explore several clinical, laboratory, imaging, histopathologic, and genetic biomarkers that have been tested as tools to predict which patients are most likely to have a pCR after nCRT.Keywords: rectal cancer, chemoradiotherapy, total mesorectal excision, nonoperative management, organ preservation