Search

This study examines the relationship between house museums and ICTs by looking at well-known examples which demonstrate that the use of ICTs in this context is not generalised. The benefits derived...

Reinforcement learning is a successful explore-and-exploit approach, where a controller tries to learn how to navigate an unknown environment. The principle approach is for an intelligent agent to learn how to maximise expected rewards. But what happens if the objective refers to non-terminating systems? We can obviously not wait until an infinite amount of time has passed, assess the success, and update. But what can we do? This talk will tell.

The expanding role of reinforcement learning (RL) in safety-critical system design has promoted ω -automata as a way to express learning requirements—often non-Markovian—with greater ease of expression and interpretation than scalar reward signals. When ω -automata were first proposed in model-free RL, deterministic Rabin acceptance conditions were used in an attempt to provide a direct translation from ω -automata to finite state “reward” machines defined over the same automaton structure (a memoryless reward translation). While these initial attempts to provide faithful, memoryless reward translations for Rabin acceptance conditions remained unsuccessful, translations were discovered for other acceptance conditions such as suitable, limit-deterministic Büchi acceptance or more generally, good-for-MDP Büchi acceptance conditions. Yet, the question “whether a memoryless translation of Rabin conditions to scalar rewards exists” remained unresolved. This paper presents an impossibility result implying that any attempt to use Rabin automata directly (without extra memory) for model-free RL is bound to fail. To establish this result, we show a link between a class of automata enabling memoryless reward translation to closure properties of its accepting and rejecting infinity sets, and to the insight that both the property and its complement need to allow for positional strategies for such an approach to work. We believe that such impossibility results will provide foundations for the application of RL to safety-critical systems.

Reinforcement learning synthesizes controllers without prior knowledge of the system. At each timestep, a reward is given. The controllers optimize the discounted sum of these rewards. Applying this class of algorithms requires designing a reward scheme, which is typically done manually. The designer must ensure that their intent is accurately captured. This may not be trivial, and is prone to error. An alternative to this manual programming, akin to programming directly in assembly, is to specify the objective in a formal language and have it "compiled" to a reward scheme. Mungojerrie (https://plv.colorado.edu/mungojerrie/) is a tool for testing reward schemes for $\omega$-regular objectives on finite models. The tool contains reinforcement learning algorithms and a probabilistic model checker. Mungojerrie supports models specified in PRISM and $\omega$-automata specified in HOA., Comment: Mungojerrie is available at https://plv.colorado.edu/mungojerrie/

We study reinforcement learning for the optimal control of Branching Markov Decision Processes (BMDPs), a natural extension of (multitype) Branching Markov Chains (BMCs). The state of a (discrete-time) BMCs is a collection of entities of various types that, while spawning other entities, generate a payoff. In comparison with BMCs, where the evolution of a each entity of the same type follows the same probabilistic pattern, BMDPs allow an external controller to pick from a range of options. This permits us to study the best/worst behaviour of the system. We generalise model-free reinforcement learning techniques to compute an optimal control strategy of an unknown BMDP in the limit. We present results of an implementation that demonstrate the practicality of the approach.

We provide the first solution for model-free reinforcement learning of {\omega}-regular objectives for Markov decision processes (MDPs). We present a constructive reduction from the almost-sure satisfaction of {\omega}-regular objectives to an almost- sure reachability problem and extend this technique to learning how to control an unknown model so that the chance of satisfying the objective is maximized. A key feature of our technique is the compilation of {\omega}-regular properties into limit- deterministic Buechi automata instead of the traditional Rabin automata; this choice sidesteps difficulties that have marred previous proposals. Our approach allows us to apply model-free, off-the-shelf reinforcement learning algorithms to compute optimal strategies from the observations of the MDP. We present an experimental evaluation of our technique on benchmark learning problems., Comment: 16 pages, 3 figures

Objective: The aim of this study was to develop a new international classification of acute pancreatitis severity on the basis of a sound conceptual framework, comprehensive review of published evidence, and worldwide consultation. Background: The Atlanta definitions of acute pancreatitis severity are ingrained in the lexicon of pancreatologists but suboptimal because these definitions are based on empiric descriptions of occurrences that are merely associated with severity. Methods: A personal invitation to contribute to the development of a new international classification of acute pancreatitis severity was sent to all surgeons, gastroenterologists, internists, intensive medicine specialists, and radiologists who are currently active in clinical research on acute pancreatitis. The invitation was not limited to members of certain associations or residents of certain countries. A global Web-based survey was conducted and a dedicated international symposium was organised to bring contributors from different disciplines together and discuss the concept and definitions. Result: The new international classification is based on the actual local and systemic determinants of severity, rather than descriptions of events that are correlated with severity. The local determinant relates to whether there is (peri)-pancreatic necrosis or not, and if present, whether it is sterile or infected. The systemic determinant relates to whether there is organ failure or not, and if present, whether it is transient or persistent. The presence of one determinant can modify the effect of another such that the presence of both infected (peri) pancreatic necrosis and persistent organ failure have a greater effect on severity than either determinant alone. The derivation of a classification based on the above principles results in 4 categories of severity - mild, moderate, severe, and critical. Conclusions: This classification is the result of a consultative process amongst pancreatologists from 49 countries spanning North America, South America, Europe, Asia, Oceania, and Africa. It provides a set of concise up-to-date definitions of all the main entities pertinent to classifying the severity of acute pancreatitis in clinical practice and research. This ensures that the determinant-based classification can be used in a uniform manner throughout the world.

Purpose: Some new staging systems in hepatocellular carcinoma (HCC) have been described in the last years. The aim of this study was to compare the survival-predicting capacity of some variables and the prognostic classifications. Methods: Demographic, clinical, analytical variables and tumour characteristics were collected in a study including 115 patients with HCC. Predictors of survival were identified using the Kaplan-Meier test and the Cox model. Comparison between different staging systems was carried out. Results: The 1-, 2- and 3-year estimated survival was 65%, 45% and 30%, respectively. Child-Pugh score and alpha-fetoprotein level greater than 400 UI/l were independent predictors of survival in the Cox model. Although all systems correctly differentiated between patients regarding survival (Kaplan-Meier, log rank < 0.05 for all), the Barcelona Clinic Liver Cancer (BCLC) showed a better discriminatory ability than the other evaluated scores. In addition, the independent homogenizing ability and stratification value of BCLC was better than that of other systems. On the contrary, model for end-stage liver disease (MELD) showed the worst results. Conclusions: Child-Pugh score and alpha-fetoprotein levels were the only independent predictors of survival in patients with HCC. Child-Pugh score showed a better prediction value for survival when compared with MELD. BCLC is more accurate than the other prognostic models evaluated in this investigation.

This study investigated the hepatoprotective effects of N-acetylcysteine and different doses of S-adenosyl-L-methionine after a single intraperitoneal overdose of paracetamol in mice. Plasma concentrations of paracetamol metabolites were also determined. Female mice (Souris OFl strain) 16 weeks old and weighing 30 g were fasted for 18 h prior to intraperitoneal (i.p.) administration of 375 mg/kg (2.5 mmol/kg) of paracetamol. Experimental subgroups included mice administered paracetamol only (control group), those given of N-acetylcysteine 1 g/kg (6.13 mmol/kg) i.p. immediately after paracetamol overdose (T0) and 6 h after dosing (T6) and those administered S-adenosyl-L-methionine at doses of 20 mg/kg (0.05 mmol/kg) and 1 g/kg (2.5 mmol/kg) i.p. at T0 and T6. Twenty-four hours after paracetamol overdose, mortality and liver necrosis were significantly lower (p < 0.01) in mice treated with 2.5 mmol/kg of S-adenosyl-L-methionine and N-acetylcysteine at T0 as compared with the remaining subgroups. Plasma ALT concentrations were significantly lower (p < 0.01) in mice treated with 2.5 mmol/kg of S-adenosyl-L-methionine than in those given N-acetylcysteine. Plasma concentrations of paracetamol metabolites showed an increase in the glucuronide conjugate and a decrease in the mercapturic acid conjugate in N-acetylcysteine-treated mice and an overall decrease in the conjugation pathway without changes in the oxidative pathway in S-adenosyl-L-methionine-treated animals. We conclude that S-adenosyl-L-methionine at doses of 1 g/kg (2.5 mmol/kg) i.p. was equally effective as 1 g/kg (6.13 mmol/kg) N-acetylcysteine for preventing hepatotoxicity after paracetamol overdose in mice. S-adenosyl-L-methionine may be a therapeutic alternative to N-acetylcysteine as an antidote for poisoning with paracetamol.

To ascertain the frequency and to describe the clinical and biochemical features of cirrhotic chylothorax.A descriptive clinical study.A community teaching hospital.Since November 1989 to October 1995, 809 patients with pleural effusions were studied by thoracentesis. Pleural effusions with a concentration of triglycerides higher than 110 mg/dL, a pleural fluid to serum triglyceride ratio higher than 1, and a pleural fluid to serum cholesterol ratio lower than 1 were considered chylothorax.Twenty-four patients had pleural effusions that complied with all three aforementioned biochemical conditions. Five of these 24 patients (20%), were found to have liver cirrhosis as the main cause of chylothorax and in 3 of them, an abdominal source of the effusion could be demonstrated by intraperitoneal injection of a radioisotope (99mTc-sulfur colloid). The cirrhotic chylous effusions had significantly lower (p0.005) protein (median, 1.7; range, 1.4 to 2.7 g/dL), lactate dehydrogenase (LDH) (median, 96; range, 77 to 138 IU/L), and cholesterol (median, 25; range, 22 to 64 mg/dL) levels than chylous effusions resulting from other causes (protein: median, 4.1; range, 1.7 to 6.8 g/dL; LDH: median, 351; range, 140 to 8,600 IU/L; and cholesterol: median, 87; range, 38 to 160 mg/dL). Cirrhotic chylothorax was always a transudate according to Light's criteria.Chylothorax is a rare and apparently underappreciated manifestation of cirrhosis resulting from transdiaphragmatic passage of chylous ascites. Its uniform biochemical characteristics can facilitate its separation from chylous effusions of different etiology, therefore avoiding potentially harmful diagnostic and therapeutic procedures.

The plasma protein binding of salicylate, a representative acidic drug, and of quinidine, a representative basic drug, has been studied in patients with multiple myeloma, a disease characterized by a profound alteration of the plasma protein pattern. Abnormal binding of salicylate was detected only in patients with high values of circulating abnormal protein. Plasma protein binding of quinidine was not altered in any of the patients studied.

The effects of diltiazem on pentagastrin-induced gastric secretion were studied in 11 normal volunteers. Diltiazem 60 mg thrice daily for 10 doses did not produce any clear-cut effect on the increase in gastric acidity induced by either standard or low doses of pentagastrin. Despite the role of calcium on gastric secretion, diltiazem does not show promise as an agent capable of reducing gastric acidity.