Search

Refereed/Peer-reviewed Investigation of value based Reinforcement Learning algorithms exhibited a resurgence into mainstream research in 2015 following demonstration of super-human performance when applied to Atari 2600 games. Since then, significant media attention and hype have accompanied this area, and the field of Artificial Intelligence generally, spread across distinct categories. This review paper is focused exclusively on the progression of value based Reinforcement Learning in the last five years. We aim to distill the incremental improvements to stability and performance in this period, highlighting the minimal changes to the base algorithm over this time. This holds true with all but the one exception of the Recurrent Experience Replay in Distributed Reinforcement Learning algorithm, representing a fundamental shift and increase in agent performance through an advanced memory representation. We suggest a new focus area for value based Reinforcement Learning research.

BACKGROUNDVaccination using severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spike (S) protein antigen has been effective in the prevention of coronavirus disease 2019 (Covid-19). NVX-CoV2373 is an adjuvanted, recombinant S protein nanoparticle vaccine that demonstrated clinical efficacy for prevention of Covid-19 in phase 2b/3 trials in the United Kingdom and South Africa.METHODSThis phase 3, randomized, observer-blinded, placebo-controlled trial evaluated the efficacy and safety of NVX-CoV2373 in adults ≥18 years of age in the United States and Mexico during the first quarter of 2021. Participants were randomized in a 2:1 ratio to receive two doses of NVX-CoV2373 or placebo 21 days apart. The primary end point was vaccine efficacy (VE) against reverse transcriptase-polymerase chain reaction-confirmed Covid-19 in SARS-CoV-2-naïve participants ≥7 days after the second dose administration.RESULTSOf the 29,949 participants randomized between December 27, 2020, and February 18, 2021, 29,582 (median age: 47 years, 12.6% ≥65 years) received ≥1 dose: 19,714 received vaccine and 9868 placebo. In the per-protocol population, there were 77 Covid-19 cases; 14 among vaccine and 63 among placebo recipients (VE: 90.4%, 95% confidence interval [CI] 82.9 to 94.6, PCONCLUSIONSNVX-CoV2373 was well tolerated and demonstrated a high overall VE (>90%) for prevention of Covid-19, with most cases due to variant strains.(Funded by the Office of the Assistant Secretary for Preparedness and Response, Biomedical Advanced Research and Development Authority and the National Institute of Allergy and Infectious Diseases (NIAID), National Institutes of Health; PREVENT-19 ClinicalTrials.gov number, NCT04611802.)

Background & Aims Non-alcoholic steatohepatitis is a leading cause of end-stage liver disease. Hepatic steatosis and lipotoxicity cause chronic necroinflammation and direct hepatocellular injury resulting in cirrhosis, end-stage liver disease and hepatocellular carcinoma. Emricasan is a pan-caspase inhibitor that inhibits excessive apoptosis and inflammation; it has also been shown to decrease portal pressure and improve synthetic function in mice with carbon tetrachloride-induced cirrhosis. Methods This double-blind, placebo-controlled study randomized 217 individuals with decompensated NASH cirrhosis 1:1:1 to emricasan (5 mg or 25 mg) or placebo. Patients were stratified by decompensation status and baseline model for end-stage liver disease-sodium (MELD-Na) score. The primary endpoint comprised all-cause mortality, a new decompensation event (new or recurrent variceal hemorrhage, new ascites requiring diuretics, new unprecipitated hepatic encephalopathy ≥grade 2, hepatorenal syndrome, spontaneous bacterial peritonitis), or an increase in MELD-Na score ≥4 points. Results There was no difference in event rates between either of the emricasan treatment groups and placebo, with hazard ratios of 1.02 (95% CI 0.59–1.77; p = 0.94) and 1.28 (95% CI 0.75–2.21; p = 0.37) for 5 mg and 25 mg of emricasan, respectively. MELD-Na score progression was the most common outcome. There was no significant effect of emricasan treatment on MELD-Na score, international normalized ratio, total serum bilirubin, albumin level or Child-Pugh score. Emricasan was generally safe and well-tolerated. Conclusions Emricasan was safe but ineffective for the treatment of decompensated NASH cirrhosis. However, this study may guide the design and conduct of future clinical trials in decompensated NASH cirrhosis. Lay summary Patients with decompensated cirrhosis related to non-alcoholic steatohepatitis are at high risk of additional decompensation events and death. Post hoc analyses in previous pilot studies suggested that emricasan might improve portal hypertension and liver function. In this larger randomized study, emricasan did not decrease the number of decompensation events or improve liver function in patients with a history of decompensated cirrhosis related to non-alcoholic steatohepatitis. ClinicalTrials.gov Identifier NCT03205345 .

Purpose This paper aims to present an approach to quantify in financial terms the value that companies create and reduce for society, based on the KPMG true value methodology. This methodology was developed to quantify the socio-economic and environmental value created and reduced by businesses in a format that can easily be understood and used by business leaders to affect key business decisions based on quantitative data. The paper looks at the business drivers for the development of the methodology and the implications and initial results for companies adopting it. Design/methodology/approach Following a brief introduction of the methodology, and the factors leading up to its development, this paper will present three recent cases of companies that have applied the methodology, their motivation for using it and what some of the initial results have been. The authors led the development of the KPMG true value methodology and have been involved in the application of the methodology across various sectors and companies. Other consultants involved in the implementation of the methodology within the companies in the case studies (below) were also interviewed for this paper. Findings The three cases above represent very different companies from various sectors. Although the approach and implementation of the KPMG true value methodology was similar across all three companies, the results, application of the results and subsequent benefits to the company in question were divergent. To date, only a handful of corporations have measured and publicly disclosed their societal value creation, but momentum is building, and, in this age of internalization, more and more companies will likely follow suit. Corporations that choose a methodology and apply it in a consistent fashion can only stand to benefit from the insights the experience brings. Research limitations/implications This paper provides insight into the KPMG true value methodology and how it has been applied within several large companies from different sectors. Because of confidentiality issues, the companies have been anonymised, and some specific quantitative data have been omitted. This paper does not look in detail at how indicators are calculated, because of space limitations. Given the fact that the methodology has only relatively recently been introduced, long-term results are not yet available. As the methodology further develops over time, there will be considerable opportunities for academic research around the methodology, for example, looking at how the creation of value for society relates to shareholder value or environmental, social and governance performance over time. Practical implications This paper provides examples of how companies have integrated socio-economic and non-financial metrics with standard financial metrics and some of the implications this can have on corporate decision-making processes. Originality/value The KPMG true value methodology was introduced in 2014 with the publication of the 2014 KPMG report “A New Vision of Value: Connecting corporate and societal value creation” (available on-line). This paper is one of the first publications in an academic journal on this topic. In writing this paper, the authors do not assume that readers have previous knowledge of the methodology, and, as such, have borrowed extensively from “A New Vision of Value” in explaining the methodology. This paper, however, goes on to highlight and reflect on the experiences of some of the first companies from different sectors to use the methodology since its launch more than two years prior.

This research describes a study into the ability of a state of the art reinforcement learning algorithm to learn to perform multiple tasks. We demonstrate that the limitation of learning to performing two tasks can be mitigated with a competitive training method. We show that this approach results in improved generalization of the system when performing unforeseen tasks. The learning agent assessed is an altered version of the DeepMind deep Q–learner network (DQN), which has been demonstrated to outperform human players for a number of Atari 2600 games. The key findings of this paper is that there were significant degradations in performance when learning more than one game, and how this varies depends on both similarity and the comparative complexity of the two games.

Acute myeloid leukaemia (AML) is characterized by the accumulation of transformed immature myeloid blasts. While most AML patients treated with standard therapy have poor outcomes, in the APL disease subtype retinoic acid induces leukaemia maturation and can be curative in combination with arsenic trioxide. Recently approved mutant IDH1/2 inhibitors also induce AML maturation, renewing interest in AML differentiation therapy. To examine differentiation therapy dynamics in vivo we have generated a novel mouse AML model driven by reversible RNAi-mediated knockdown of the myeloid transcription factor PU.1. Restoration of endogenous PU.1 in established AML in vivo triggers synchronous differentiation of leukemic blasts and disease clearance. However, despite near-complete remission, mice reproducibly relapse with immature AML. Notably, in vivo time course studies reveal that one week after PU.1 restoration leukemic blasts differentiate into two mature myeloid lineages with distinct immunophenotype and morphology. AML-derived SSClowLy6G+ cells resembling neutrophils initially predominate but are rapidly eradicated in vivo. In contrast, high resolution flow and imaging indicates that mature AML-derived SSChighF4/80+SigF+ eosinophil-like cells persist at low numbers in specific organs during disease remission and appear to seed relapse. In mice transplanted with AML blasts lacking the essential eosinophil lineage transcription factor GATA1, in vivo PU.1 restoration triggers neutrophil but not eosinophil lineage differentiation and thereby eliminates residual disease. These results demonstrate that AML differentiation therapy can produce long-lived sublineages of mature AML-derived cells from which relapse can originate. Understanding the multilineage potential of AML blasts in individual patients may inform new strategies to improve differentiation therapy outcomes.

There is a growing need for students and practitioners of Traditional Chinese Medicine to gain experience with standardized data collection, patient outcomes measurement, and practice-based research. The purpose of this paper is to describe the development of a process for standardized data collection that could eventually be adopted for clinical, research, and quality assurance purposes.The setting for this study was an acupuncture and Oriental medicine teaching clinic in Bloomington, Minnesota.Four (4) aspects of data collection were assessed and improved, including intake and post-treatment questionnaires, follow-up with patients, integration of data collection into clinic flow, and commitment of resources to the project.The outcomes measures were data collection and missing data rates, burden on patients and clinic staff, and efficiency of data entry.Revision to the data collection process resulted in decreased burden to patients and staff, more detailed and aggressive follow-up protocols, enhanced training for clinic staff, and increased personnel and data-related resources.The systematic collection of descriptive and clinical characteristics can be accomplished in a teaching clinic with thoughtful attention paid to data collection protocols, dedicated resources, and the involvement of all relevant personnel.

Recent advances in machine learning with big data sets has allowed for significant advances in the optimisation of classification and recognition systems. However, for applications such as situational awareness systems, the entirety of the available data dwarfs the amount permissible for a training set with tractable machine learning optimization times. Furthermore, the performance of any optimized system is highly dependent of the training set correctly and completely representing the entire data space of scenarios. In this paper we present a technique to characterize the entire data space to ascertain the key factors for representation and subsequently select a subset that statistically represents the correct mix of scenarios. We demonstrate the effectiveness of these characterization and subset selection techniques by using a genetic algorithm to optimize the performance of a gunfire recognition system.

PurposeSets out to examine the dangers of money laundering as a consequence of natural disasters.Design/methodology/approachLists the potential abuses and scans to which unscrupulous manipulators will resort in order to profit from natural disasters.FindingsFinds that criminal groups have established networks and sophisticated technology to effectively carry out their activities.Originality/valueThis is a detailed and eye‐opening revelation of the various criminal opportunities for money laundering spawned by natural disasters.

Introduces the USA’s Food Stamp Program, which provides large amounts of money to poor people. Shows how this distribution system creates the potential for money laundering, through both the Electronic Benefits Transfer system and food stamp coupons, and the social costs of this misuse of funds. Points out that financial institutions are in a unique position to help the Food and Nutrition Service and law enforcement agencies combat this misuse, by identifying suspicious activities. Highlights six cases relating to food stamp trafficking; these illustrate how money laundering was involved and how financial institutions’ Know Your Customer/Customer Due Diligence policies and Suspicious Activity Reports can be used to minimise it.

This paper describes a study into the impact of local environmental conditions on the detection of point targets using wide field of view infrared sensors on airborne platforms. A survey of the common complexity metrics for measuring IR clutter, and common point target detection algorithms was conducted. A quantitative evaluation was performed using 20 hours of infrared imagery collected over a three month period from helicopter flights in a variety of clutter environments. The research method, samples of the IR data sets, and results of the correlation between environmental conditions, scene complexity metrics and point target detection algorithms are presented. The key findings of this work are that variations in IR detection performance can be attributed to a combination of environmental factors (but no single factor is sufficient to describe performance variations), and that historical clutter metrics are insufficient to describe the performance of modern detection algorithms.

The purpose of this study was to describe patients who seek treatment at an acupuncture and Oriental medicine teaching clinic in the United States, and to systematically measure and describe patients' responses after treatment using a prospective study design.This is a prospective survey of clinic patients at intake and one month following the initial treatment. SETTINGS AND LOCATION: Data were collected in an acupuncture and Oriental medicine teaching clinic located in Bloomington, Minnesota.Of 661 new patients who met eligibility criteria, 485 consented to participate.Patients were administered two self-report questionnaires: one prior to their initial treatment, and a second sent by mail one month later.Data collected at intake included demographics such as age, gender, race, ethnicity, education, and employment, as well as main presenting complaint and chronicity. Patients were also asked at intake whether they had consulted with another health care provider, if they were under continued care, and if they had previous experience with Traditional Chinese Medicine (TCM) treatment. Outcome measures included severity, improvement, and satisfaction. Patients were additionally asked if they continued with TCM care for their presenting condition.Demographics of patients presenting to this teaching clinic were similar to those reported in other outpatient TCM settings. The majority of patients had no previous experience with TCM, and a large percentage was referred by students. Pain was the most common presenting condition, followed closely by wellness care. One month following treatment, most patients reported improvement and satisfaction with care.Standardized data collection and follow-up resulted in a description of patients and outcomes in an acupuncture and Oriental medicine teaching clinic, which can be used for research, educational, quality assurance, and marketing purposes.