Search

Your search keyword '"Qian, Jian"' showing total 1,666 results
Start Over Author "Qian, Jian"
1,666 results on '"Qian, Jian"'

54. Stochastic Bandits with Vector Losses: Minimizing $\ell^\infty$-Norm of Relative Losses

Author

Shang, Xuedong, Shao, Han, and Qian, Jian

Subjects
Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

Multi-armed bandits are widely applied in scenarios like recommender systems, for which the goal is to maximize the click rate. However, more factors should be considered, e.g., user stickiness, user growth rate, user experience assessment, etc. In this paper, we model this situation as a problem of $K$-armed bandit with multiple losses. We define relative loss vector of an arm where the $i$-th entry compares the arm and the optimal arm with respect to the $i$-th loss. We study two goals: (a) finding the arm with the minimum $\ell^\infty$-norm of relative losses with a given confidence level (which refers to fixed-confidence best-arm identification); (b) minimizing the $\ell^\infty$-norm of cumulative relative losses (which refers to regret minimization). For goal (a), we derive a problem-dependent sample complexity lower bound and discuss how to achieve matching algorithms. For goal (b), we provide a regret lower bound of $\Omega(T^{2/3})$ and provide a matching algorithm., Comment: 14 pages

Published
2020

55. Towards Minimax Optimal Reinforcement Learning in Factored Markov Decision Processes

Author

Tian, Yi, Qian, Jian, and Sra, Suvrit

Subjects
Computer Science - Machine Learning, Mathematics - Optimization and Control, Statistics - Machine Learning
Abstract

We study minimax optimal reinforcement learning in episodic factored Markov decision processes (FMDPs), which are MDPs with conditionally independent transition components. Assuming the factorization is known, we propose two model-based algorithms. The first one achieves minimax optimal regret guarantees for a rich class of factored structures, while the second one enjoys better computational complexity with a slightly worse regret. A key new ingredient of our algorithms is the design of a bonus term to guide exploration. We complement our algorithms by presenting several structure-dependent lower bounds on regret for FMDPs that reveal the difficulty hiding in the intricacy of the structures., Comment: 54 pages

Published
2020

56. Concentration Inequalities for Multinoulli Random Variables

Author

Qian, Jian, Fruit, Ronan, Pirotta, Matteo, and Lazaric, Alessandro

Subjects
Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

We investigate concentration inequalities for Dirichlet and Multinomial random variables., Comment: Tutorial at ALT'19 on Regret Minimization in Infinite-Horizon Finite Markov Decision Processes

Published
2020

69. Importance Resampling for Off-policy Prediction

Author

Schlegel, Matthew, Chung, Wesley, Graves, Daniel, Qian, Jian, and White, Martha

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Statistics - Machine Learning
Abstract

Importance sampling (IS) is a common reweighting strategy for off-policy prediction in reinforcement learning. While it is consistent and unbiased, it can result in high variance updates to the weights for the value function. In this work, we explore a resampling strategy as an alternative to reweighting. We propose Importance Resampling (IR) for off-policy prediction, which resamples experience from a replay buffer and applies standard on-policy updates. The approach avoids using importance sampling ratios in the update, instead correcting the distribution before the update. We characterize the bias and consistency of IR, particularly compared to Weighted IS (WIS). We demonstrate in several microworlds that IR has improved sample efficiency and lower variance updates, as compared to IS and several variance-reduced IS strategies, including variants of WIS and V-trace which clips IS ratios. We also provide a demonstration showing IR improves over IS for learning a value function from images in a racing car simulator., Comment: Recently published in NeurIPS 2019

Published
2019

73. Exploration Bonus for Regret Minimization in Undiscounted Discrete and Continuous Markov Decision Processes

Author

Qian, Jian, Fruit, Ronan, Pirotta, Matteo, and Lazaric, Alessandro

Subjects
Computer Science - Machine Learning, Statistics - Machine Learning
Abstract

We introduce and analyse two algorithms for exploration-exploitation in discrete and continuous Markov Decision Processes (MDPs) based on exploration bonuses. SCAL$^+$ is a variant of SCAL (Fruit et al., 2018) that performs efficient exploration-exploitation in any unknown weakly-communicating MDP for which an upper bound C on the span of the optimal bias function is known. For an MDP with $S$ states, $A$ actions and $\Gamma \leq S$ possible next states, we prove that SCAL$^+$ achieves the same theoretical guarantees as SCAL (i.e., a high probability regret bound of $\widetilde{O}(C\sqrt{\Gamma SAT})$), with a much smaller computational complexity. Similarly, C-SCAL$^+$ exploits an exploration bonus to achieve sublinear regret in any undiscounted MDP with continuous state space. We show that C-SCAL$^+$ achieves the same regret bound as UCCRL (Ortner and Ryabko, 2012) while being the first implementable algorithm with regret guarantees in this setting. While optimistic algorithms such as UCRL, SCAL or UCCRL maintain a high-confidence set of plausible MDPs around the true unknown MDP, SCAL$^+$ and C-SCAL$^+$ leverage on an exploration bonus to directly plan on the empirically estimated MDP, thus being more computationally efficient.

Published
2018

87. Inhibition of DAI refrains dendritic cells from maturation and prolongs murine islet and skin allograft survival

Author

Pengrui Cheng, Qian Jian, Zongli Fu, Ronghai Deng, and Yi Ma

Subjects
dendritic cells, DNA-dependent activator of IFN regulatory factors, immunosuppression, islet transplant, skin transplant, adenovirus vector, Immunologic diseases. Allergy, RC581-607
Abstract

IntroductionCentral to allograft rejection is the T cell-mediated adaptive immune response initiated by activated dendritic cells (DCs). Previous studies have shown that the DNA-dependent activator of IFN regulatory factors (DAI) is involved in the maturation and activation of DCs. Therefore, we hypothesized that inhibition of DAI could prevent DCs from maturation and prolong murine allograft survival.MethodsDonor mouse bone marrow-derived dendritic cells (BMDCs) were transduced with the recombinant adenovirus vector (AdV-DAI-RNAi-GFP) to inhibit DAI expression (DC-DAI-RNAi), and the immune cell phenotype and function of DC-DAI-RNAi upon lipopolysaccharide (LPS) stimulation were evaluated. Then DC-DAI-RNAi was injected into recipient mice before islet transplantation and skin transplantation. The survival times of islet and skin allograft were recorded and the proportions of T cell subsets in spleen and secretion levels of cytokines in serum were measured.ResultsWe identified that DC-DAI-RNAi inhibited the expression of main co-stimulatory molecules and MHC-II, exhibited strong phagocytic ability, and secreted high levels of immunosuppressive cytokines and low levels of immunostimulating cytokines. Recipient mice treated with DC-DAI-RNAi had longer islet and skin allograft survival times. In the murine islet transplantation model, we observed an increase in Treg cells proportion, a reduction in Th1 and Th17 cells proportions in spleen, and similar trends in their secreted cytokines in serum in the DC-DAI-RNAi group.ConclusionInhibition of DAI by adenovirus transduction inhibits the maturation and activation of DCs, affects the differentiation of T cell subsets as well as their secreted cytokines, and prolongs allograft survival.

Published
2023
Full Text
View/download PDF

91. Efficacy and safety of glibenclamide therapy after intracerebral haemorrhage (GATE-ICH): A multicentre, prospective, randomised, controlled, open-label, blinded-endpoint, phase 2 clinical trial

Author

Zhao, Jingjing, Song, Changgeng, Li, Deshuai, Yang, Xiai, Yu, Liping, Wang, Kangjun, Wu, Jun, Wang, Xiaofeng, Li, Dongsong, Zhang, Bo, Li, Binyong, Guo, Jun, Feng, Weikui, Fu, Feng, Gu, Xinrong, Qian, Jian, Li, Jialong, Yuan, Xiangjun, Liu, Qiuwu, Chen, Jiang, Wang, Xiaocheng, Liu, Yi, Wei, Dong, Wang, Ling, Shang, Lei, Yang, Fang, and Jiang, Wen

Published
2022
Full Text
View/download PDF

95. Dysregulation of innate cell types in the hepatic immune microenvironment of alcoholic liver cirrhosis

Author

Ao Ren, Wenjing He, Jiawei Rao, Dongmei Ye, Pengrui Cheng, Qian Jian, Zongli Fu, Xuzhi Zhang, Ronghai Deng, Yifang Gao, and Yi Ma

Subjects
scRNA-seq, MAIT cells, NKT cells, macrophages, fibrosis, Immunologic diseases. Allergy, RC581-607
Abstract

IntroductionThe risk of alcoholic cirrhosis increases in a dose- and time-dependent manner with alcohol consumption and ethanol metabolism in the liver. Currently, no effective antifibrotic therapies are available. We aimed to obtain a better understanding of the cellular and molecular mechanisms involved in the pathogenesis of liver cirrhosis. MethodsWe performed single-cell RNA-sequencing to analyze immune cells from the liver tissue and peripheral blood form patients with alcoholic cirrhosis and healthy controls to profile the transcriptomes of more than 100,000 single human cells and yield molecular definitions for non-parenchymal cell types. In addition, we performed single-cell RNA-sequencing analysis to reveal the immune microenvironment related to alcoholic liver cirrhosis. Hematoxylin and eosin, Immunofluorescence staining and Flow cytometric analysis were employed to study the difference between tissues and cells with or without alcoholic cirrhosis.ResultsWe identified a fibrosis-associated M1 subpopulation of macrophages that expands in liver fibrosis, differentiates from circulating monocytes, and is pro-fibrogenic. We also define mucosal-associated invariant T (MAIT) cells that expand in alcoholic cirrhosis and are topographically restricted to the fibrotic niche. Multilineage modeling of ligand and receptor interactions between the fibrosis-associated macrophages, MAIT, and NK cells revealed the intra-fibrotic activity of several pro-fibrogenic pathways, including responses to cytokines and antigen processing and presentation, natural killer cell-mediated cytotoxicity, cell adhesion molecules, Th1/Th2/Th17 cell differentiation, IL-17 signaling pathway, and Toll-like receptor signaling pathway.DiscussionOur work dissects unanticipated aspects of the cellular and molecular basis of human organ alcoholic fibrosis at the single-cell level and provides a conceptual framework for the discovery of rational therapeutic targets in liver alcoholic cirrhosis.

Published
2023
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results