Search

A decision maker solicits information from two partially informed experts and then makes a choice under uncertainty. The experts can be either moderately or extremely biased relative to the decision maker, which is their private information. I investigate the incentives of the experts to share their private information with the decision maker and analyze the resulting effects on information transmission. I show that it may be optimal to consult a single expert rather than two experts if the decision maker is sufficiently concerned about taking advice from extremely biased experts. In contrast to what may be expected, this result suggests that getting a second opinion may not always be helpful for decision making.

The management of business information processes needs effective decision-making models. That means to involve different methods, techniques, and principles to improve competitiveness and to achieve the planned business results. In this context, the article deals with the problem of group decision-making under uncertain conditions. To cope with such problems some well-known optimization strategies of Wald, Laplace, Hurwitz, and Savage are modified to take into account the experts’ opinions with different importance when forming the final group decision. Numerical testing is based on a case study for CRM software selection. The results are discussed based on the proposed models under two different cases derived from the case study. The conducted numerical testing of the proposed models demonstrates their applicability to cope simultaneously with multiple experts’ evaluations and uncertainty conditions.

We consider the forecast aggregation problem in repeated settings where the forecasts are of a binary state of nature. In each period multiple experts provide forecasts about the state. The goal of the aggregator is to aggregate those forecasts into a subjective accurate forecast. We assume that the experts are Bayesian and the aggregator is non-Bayesian and ignorant of the information structure (i.e., the distribution over the signals) under which the experts make their forecasts. The aggregator observes the experts’ forecasts only. At the end of each period, the realized state is observed by the aggregator. We focus on the question of whether the aggregator can learn to optimally aggregate the forecasts of the experts, where the optimal aggregation is the Bayesian aggregation that takes into account all the information in the system. We consider the class of partial evidence information structures, where each expert is exposed to a different subset of conditionally independent signals. Our main results are positive: we show that optimal aggregation can be learned in polynomial time in quite a wide range of instances in partial evidence environments. We provide an exact characterization of the instances where optimal learning is possible as well as those where it is impossible.

Background Pre-existing gender-based disparities in academia may have worsened during the COVID-19 pandemic. Being cited as an expert source in newspaper articles about COVID-19 may increase an individual’s research or leadership profile. In addition, visibility in a newspaper article is an important component of representation in academia. Objective To determine whether women were underrepresented as COVID-19 expert sources in print newspapers in the USA. Design We undertook a cross-sectional study of English-language newspaper articles that addressed the COVID-19 pandemic and that were published in the top 10 most widely read newspapers in the USA between April 1 and April 15, 2020. Main Measures We extracted the names of all people cited as expert sources and categorized the gender of each expert source based on pronoun usage within the article or on a business, university, or organization website. The professional role of each expert was assigned based on their description in the article. Key Results Of 2297 expert sources identified, 35.9% (95% confidence interval [CI] 33.9–37.8%; n = 824) were women and 63.7% were men (95% CI 61.8–65.7%; n = 1464). This result was similar when considering unique experts in each newspaper and for all included newspapers; of the 1738 unique experts per newspaper, 34.6% were women (95% CI 32.3–36.8%; n = 601), and of the 1593 unique experts in all newspapers, 36.5% were women (95% CI 34.1–38.9%; n = 581). Of articles with multiple experts referenced (n = 374), 102 cited only men experts (27.3%) and 44 cited only women experts (11.8%). Women were underrepresented as experts as Healthcare Workers and Professionals, Non-STEM Experts, Public Health Leaders, and STEM Scientists. There were no differences in the proportion of women experts between newspapers or between different regions of the USA. Conclusions Altogether, our findings support that men academics outnumber women as COVID-19 experts in newspaper articles. Supplementary Information The online version contains supplementary material available at 10.1007/s11606-020-06579-3.

Tracking algorithm based on correlation filter have been extensively investigated due to their powerful performance in benchmark datasets and competitions. However, the periodic assumption has contributed boundary effects and the complex scenarios will give rise to model drift, which have an extremely negative effect on both tracking precision and success rate. To mitigate these challenges, a novel multi-model and multi-expert correlation filter (MMCF) approach is proposed in this paper. The key innovation of the proposed method is to employ multiple models and experts for tracking. Multiple models can excavate diverse a large quantity of feature information, and the target information between different models can complement each other. Multiple experts provide several possible predictions, while an evaluation mechanism selects the most reliable prediction utilizing their assessment scores. To further improve the performance, an adaption strategy is utilized to update the multiple models, which can reduce the weight of the bad samples to prevent model drift. Experiments performed on three recent benchmark datasets OTB50, OTB2013, OTB100, TC-128 and UAV123@10fps, have demonstrated the superiority of our approach in comparison to the state-of-the-art trackers. Our MMCF tracker operates a speed of about 58 frames per second (FPS) running on a single central processing unit(CPU).

Expert systems (ES) are widely used for engineering and manufacturing applications nowadays. In order to solve the problems in material selection in the aviation field, an aircraft material expert system (AMES) was developed. This expert system is based on the knowledge of multiple experts and aviation data. A decision support system (DSS) was designed for selecting the materials used in aviation fields. The flows of material selection were designed, and the models of AMES were established. A data base system was also designed to implement AMES.

PurposeThe purpose of this study is to develop a methodology which amalgamates quantitative and qualitative approaches to determine the best placement of mobile logistics hubs (MLH) to be established in different parts of Nepal as a part of real-life project, “Augmentation of National and Local-Level Emergency Logistics Preparedness in Nepal” (2017–2020), implemented by the World Food Programme in cooperation with the Government of Nepal.Design/methodology/approachThe study develops a methodology using a combination of a modified version of the maximal covering location problem (MCLP) and focus group discussion. The MCLP model is used to determine the optimal number and spatial location of MLHs, and focus group discussion is used to identify the five first-priority strategic MLH locations using expert knowledge.FindingsThe authors identify the five first-priority locations for establishing MLHs using an amalgamation of quantitative approach (mathematical model) and qualitative approach (focus group discussion). By amalgamating mathematical model with expert knowledge, findings acceptable to a wide range of stakeholders are obtained. The focus group discussion helps to pinpoint the location of MLHs to city-level granularity which is otherwise impossible with data available on hand.Research limitations/implicationsAlthough multiple experts’ judgements were obtained via focus group discussion, subjectivity and possible bias is inevitable. Overall, the quantitative results of the study are purely based on the data available during the study period; therefore, having updated data could possibly improve the quality of the results.Originality/valueThis study is the first of its kind that uses an amalgamation of mathematical model and expert knowledge to determine the strategic locations of MLHs and has been successful to an extent that the selected locations have been vetted by the government of Nepal for establishing MLHs and are undergoing implementation in real life. This study also considers multiple disaster scenarios and employs the concepts of human development, disaster risk and transportation accessibility to reflect Nepal's socioeconomic, geo-climatic and topographical features.

Background This study used existing literature and expert feedback to develop and pilot a novel error-avoidance checklist tool for cricothyrotomy in attending physicians. Prior literature has not focused on expert cricothyrotomy performance. While published checklists teach a specific procedural method, ideal for novice learners, this may hinder expert learners. Objectives We endeavored to create a succinct error-avoidance checklist for cricothyrotomy. We hypothesized that such a checklist would prove feasible and acceptable to attending physicians. Methods This is a multicenter prospective checklist creation, evaluation, and feasibility study. Multiple experts pursued an iterative process to reach consensus on a 7-item error-avoidance checklist. The checklist was trialed for feasibility in pilot sessions at two sites by 45 attending emergency physicians who used the checklist for peer performance assessment and provided feedback. Results During the pilot implementation, 94% of respondents completed the procedure within the allotted 120 s. Greater than 85% of respondents agreed that four of the five procedural errors on the checklist were very or somewhat critical to avoid, including cutting >2 cm from midline, creating a false passage, failing to continuously maintain an object in the trachea, and injuring oneself during the procedure. Only 66% of participants felt severing the cricoid cartilage was critical. Successful breath administration and time under 120 s were critical for 100% and 95% of participants, respectively. The checklist was rated "easy" or "very easy" to use by 93% of participants, and 95% found this checklist reasonable for evaluating attending physicians. Conclusions We present the multicenter development and implementation of a novel error-avoidance checklist tool for use in expert cricothyrotomy performance. Attending emergency medicine (EM) physicians rated our tool easy to use and agreed that most of the proposed errors were critical. Participants overwhelmingly agreed this tool would be reasonable for evaluation of cricothyrotomy performance among attending EM physicians.

In many real life situations, including job and loan applications, gatekeepers must make justified and fair real-time decisions about a person's fitness for a particular opportunity. In this paper, we aim to accomplish approximate group fairness in an online stochastic decision-making process, where the fairness metric we consider is equalized odds. Our work follows from the classical learning-from-experts scheme, assuming a finite set of classifiers (human experts, rules, options, etc) that cannot be modified. We run separate instances of the algorithm for each label class as well as sensitive groups, where the probability of choosing each instance is optimized for both fairness and regret. Our theoretical results show that approximately equalized odds can be achieved without sacrificing much regret. We also demonstrate the performance of the algorithm on real data sets commonly used by the fairness community., Accepted to IJCAI 2021

Using publicly available data to determine the performance of methodological contributions is important as it facilitates reproducibility and allows scrutiny of the published results. In lung nodule classification, for example, many works report results on the publicly available LIDC dataset. In theory, this should allow a direct comparison of the performance of proposed methods and assess the impact of individual contributions. When analyzing seven recent works, however, we find that each employs a different data selection process, leading to largely varying total number of samples and ratios between benign and malignant cases. As each subset will have different characteristics with varying difficulty for classification, a direct comparison between the proposed methods is thus not always possible, nor fair. We study the particular effect of truthing when aggregating labels from multiple experts. We show that specific choices can have severe impact on the data distribution where it may be possible to achieve superior performance on one sample distribution but not on another. While we show that we can further improve on the state-of-the-art on one sample selection, we also find that on a more challenging sample selection, on the same database, the more advanced models underperform with respect to very simple baseline methods, highlighting that the selected data distribution may play an even more important role than the model architecture. This raises concerns about the validity of claimed methodological contributions. We believe the community should be aware of these pitfalls and make recommendations on how these can be avoided in future work., Accepted at PRIME, MICCAI 2021

Safety analysis is essential in the natural gas transmission industry to guarantee effective hazard identification and to prevent the failure of components in advance. The overall aim of this research is to introduce a new hybrid expert system and fuzzy logic to monitor city gate stations as one of the most vital installations of gas distribution networks. The proposed model utilizes fuzzy if-then rules based on multiple experts' opinions to study the compound interrelations between the mechanical and physical elements of city gate stations. The presented expert system accounts for uncertainty associated with the experts’ judgments by fuzzy sets theory. The expert system is implemented in an object-oriented platform and is programmed with C#. The validity of the expert system is confirmed using simulation experiments through a case study of city gate stations.

Natural language understanding (NLU) models in production systems rely heavily on human annotated data, which involves an expensive, time-consuming, and error-prone process. Moreover, the model release process requires offline supervised learning and human in-the-loop. Together, these factors prolong the model update cycle and result in sub-standard model performance in situations where usage behavior is non-stationary. In this paper, we address these issues with a deep reinforcement learning approach that ranks suggestions from multiple experts in an online fashion. Our proposed method removes the reliance on annotated data, and can effectively adapt to recent changes in the data distribution. The efficiency of the new approach is demonstrated through simulation experiments using logged data from voice-based virtual assistants. Our results show that our algorithm, without any reliance on annotation, outperforms offline supervised learning methods.

The global pandemic of COVID-19 has impacted the entire world in ways that were unimaginable months ago. It has changed how professionals work and interact with one another. To ensure the safety of experts and attorneys, also forensic inspection procedures must evolve, while still providing a reliable foundation for an expert testimony, relevant to the task at hand. Traditional field inspections require attendees to be physically present and generally in close proximity. Inspecting artifacts may also require that individual components be serially examined by multiple experts, which increases the risk of transmitting the disease. The authors propose a new approach to forensic inspections aimed at protecting consultants from the biological risk due to COVID-19. This approach takes into account the anticontagion safety protocols established in affected countries around the world.

$k$ -nearest neighbors algorithm ( $k$ -NN) is a simple yet powerful method that can predict an unlabeled object using the $k$ closest samples whose labels are already known. However, in some cases such as medical treatment decision making, multiple experts can provide their individual decisions and there is no single ground truth for the labels of samples. As a result, the accuracy of the predictions can be affected by the uncertainties of historical decisions. In order to learn from uncertain decisions from multiple experts, we propose a new $k$ -NN algorithm. This algorithm firstly measures the uncertainties of labels. Then, the prediction is made based on the uncertain labels of the $k$ closest samples based on the Dempster-Shafer theory. We also introduce two methods to compute the optimal $k$ value. Experiments on a real-world medical treatment dataset prove that our algorithm has better performance compared with other algorithms.

Pharmacoepidemiology is a field of study that seeks to understand the beneficial and harmful effects of drug utilization in populations of people. The discipline is an amalgam of clinical pharmacology, clinical epidemiology, medical informatics, and biostatistics. Also, pharmacoepidemiology can include the misuse/abuse of pharmaceutical drugs or even the study of drugs taken outside of the health-care system, such as illicit drugs. The purpose of this chapter is to provide a broad overview of pharmacoepidemiology from multiple experts currently working in this field.

In real-world applications, inferring the intentions of expert agents (e.g., human operators) can be fundamental to understand how possibly conflicting objectives are managed, helping to interpret the demonstrated behavior. In this paper, we discuss how inverse reinforcement learning (IRL) can be employed to retrieve the reward function implicitly optimized by expert agents acting in real applications. Scaling IRL to real-world cases has proved challenging as typically only a fixed dataset of demonstrations is available and further interactions with the environment are not allowed. For this reason, we resort to a class of truly batch model-free IRL algorithms and we present three application scenarios: (1) the high-level decision-making problem in the highway driving scenario, and (2) inferring the user preferences in a social network (Twitter), and (3) the management of the water release in the Como Lake. For each of these scenarios, we provide formalization, experiments and a discussion to interpret the obtained results.

Formal Concept Analysis (FCA) provides a method called attribute exploration which helps a domain expert discover structural dependencies in knowledge domains that can be represented by a formal context (a cross table of objects and attributes). Triadic Concept Analysis is an extension of FCA that incorporates the notion of conditions. Many extensions and variants of attribute exploration have been studied but only few attempts at incorporating multiple experts have been made. In this paper we present triadic exploration based on Triadic Concept Analysis to explore conditional attribute implications in a triadic domain. We then adapt this approach to formulate attribute exploration with multiple experts that have different views on a domain.

Multiple experts decision-making (MEDM) can be regarded as a situation where a group of experts are invited to provide their opinions by evaluating the given alternatives, and then select the optimal alternative(s). As a useful linguistic expression model, linguistic preference orderings (LPOs) were established in which the order of alternatives and the relationships between two adjacent alternatives are fused well. Considering that prospect theory has the superiority in depicting risk attitudes (risk seeking for losses and risk aversion for gains) during the uncertain decision-making process, this paper develops a consensus model based on prospect theory to deal with MEDM problems with LPOs. Firstly, each LPO provided by expert is transformed into the responding DHLPR with complete consistency. Then, the reference point of expert is determined and the prospect preference matrix is established. Moreover, we can obtain the overall prospect consensus degree for a MEDM problem by calculating the similarity degree between individual and collective prospect preference matrix. Furthermore, a consensus improvement method is developed to complete the consensus reaching process. Finally, we apply the proposed method to deal with a practical MEDM problem involving the construction project investment, and make some comparative analyses with existing methods., National Natural Science Foundation of China (NSFC) 71771155, China Postdoctoral Science Foundation 2020M680151, Sichuan Postdoctoral Science special Foundation, Sichuan University Postdoctoral Interdisciplinary Innovation Startup Foundation, Fundamental Research Funds for the Central Universities YJ202015, European Union (EU) TIN2016-75850-R, Sichuan Province System Science and Enterprise Development Research Center Xq20B03

In the evaluation of technology projects, multiple experts need to be selected to evaluate a group of projects each of which covers different technology fields. However, each expert has his own advantage fields, so it is a significant challenge to find scientifically and automatically a suitable group of experts matching the evaluated projects from a large number of candidates. In this paper, we propose a multi-matching model GES based on genetic algorithm. This model is applied to the established two correlation matrices of “project-field” and “expert-field”, by separately calculating the discrete distribution of projects and experts in the technology field domain. Then GES makes use of the genetic algorithm integrating an evaluation function measuring the matching degree between the projects and the experts to search the optimal candidates. We carried out the throughout experiment based on the realistic electric-power industry data sets, the results show that GES searched effectively and accurately the group of evaluation experts.

The testing and diagnostic systems have been considered to be useful for students because they can point to their learning problems to provide helpful suggestions for improving the knowledge. The previous approach shows that using the knowledge from multiple experts can develop a testing and diagnostic system that provides more accurate suggestions for learners comparing to the system using a single expert. Nevertheless, the low-quality knowledge integration method of the multi-expert approach can provide some inaccurate learning suggestions. This work proposes a practical method for enhancing knowledge integration from multiple experts to provide more effective learning suggestions. An experiment has been conducted on freshmen in university to evaluate the effectiveness of the proposed method. The results show that the proposed approach not only improves the learning achievements of the students but also decreases the number of reconsidering cases.

Digitalization and artificial intelligence are growing in importance as parts of decision-support tools in various application domains. One of the important developments in this vein has been the creation of interactive tools for coaching users of complex decision-support systems to help them successfully and correctly use the said systems. This paper focuses on digital coaching in the context of strategic investment analysis, specifically connected to fuzzy real options analysis (ROA). We present some important and difficult choices connected to ROA and discuss how digital coaching may assist users in better using ROA tools. We illustrate the real-world use of digital coaching in the contexts of cash-flow evaluation with machine learning support and aggregation of cash-flows from multiple experts. The discussion and the cases illustrate well how digital coaching can make a difference, especially for an inexperienced user, in guiding users to use complex tools correctly and in creating better circumstances for credible analyses. The findings presented are new and contribute specifically to the literature on digital coaching and real options analysis.

In composing a Transformer Assessment Index (TAI), weighting factor is a critical consideration. Expert judgment is a common practice in adjusting weighting factor of a TAI. Years of experience from experts is a valuable information to be considered. Several previous researches develop AHP to determine parameter prioritization based on expert judgment. The use of multi-expert can benefit the system developed, but this may lead to another complexity in aggregating the results. As much as five experts with in-depth experience in transformer condition monitoring and diagnostics were taking parts in this study. A row geometric mean method was used to obtain the consensus matrix. This paper proposed an approach to implement the consensus model for multiple experts in power transformer assessment index weighting factor determination using AHP.

Mathematical frameworks are developed to address the representation of inherent uncertainties in the assessment of natural hazard risk for the built environment. Lack of exploration of such theorie...

As one of the key components in many computer vision applications, visual tracking has been widely investigated in recent years. Correlation filter-based trackers have achieved compelling performance in terms of both efficiency and accuracy by speeding up online training and inference procedures in the frequency domain. For a visual object, such trackers often perform tracking by searching the most correlated patch within a limited search region in a new frame. The acknowledged problem is that a small search region prohibits the successful tracking of fast moving targets while a large search region increases the difficulty of model learning. In this paper, we propose to use different candidate search regions to train multiple tracking experts to alleviate this dilemma. Based on these experts, the final tracker learns to make the optimal decision using a decision-theoretic online learning approach. Besides, we also establish multi-scale experts to handle scale variations during tracking. The experimental results on the public datasets demonstrate that the proposed approach achieves favorable performance against several state-of-the-art methods.

Aztreonam-avibactam is a combination antimicrobial agent with activity against carbapenemase-producing Enterobacteriaceae (CPE) with metallo-β-lactamases (MβLs). Although aztreonam-avibactam is not yet approved by the U.S. Food and Drug Administration (FDA), clinicians can administer this combination by using two FDA-approved drugs: aztreonam and ceftazidime-avibactam. This combination of drugs is recommended by multiple experts for treatment of serious infections caused by MβL-producing CPE. At present, in vitro antimicrobial susceptibility testing (AST) of aztreonam-avibactam is not commercially available; thus, most clinicians receive no laboratory-based guidance that can support consideration of aztreonam-avibactam for serious CPE infections. Here, we report our internal validation for aztreonam-avibactam AST by reference broth microdilution (BMD) according to Clinical and Laboratory Standards Institute (CLSI) guidelines. The validation was performed using custom frozen reference BMD panels prepared in-house at the Centers for Disease Control and Prevention (CDC). In addition, we took this opportunity to evaluate a new panel-making method using a digital dispenser, the Hewlett Packard (HP) D300e. Our studies demonstrate that the performance characteristics of digitally dispensed panels were equivalent to those of conventionally prepared frozen reference BMD panels for a number of drugs, including aztreonam-avibactam. We found the HP D300e digital dispenser to be easy to use and to provide the capacity to prepare complex drug panels. Our findings will help other clinical and public health laboratories implement susceptibility testing for aztreonam-avibactam.

This article describes how rank aggregation focuses on synthesizing a single ranked list based on rankings supplied by multiple judges. Such aggregations are widely applied in the areas of information retrieval, web search, and data mining. The problem of rank aggregation has been shown to be NP-hard and this article presents a heuristic approach to create an aggregated ranking score for all items on the lists. The proposed heuristic is scalable and performs. A computational study, as well as a real-life study involving the ranking of 147 engineering colleges in the US is presented to elucidate the performance. The authors' key finding is that the quality of the solution is sensitive to (a) the number of judges available to rank, (b) how the items are assigned to judges, and (c) how consistent/inconsistent the judges are. All these factors are generally considered exogenous in most of the rank aggregation algorithms in extant literature.

When a capital crime is committed the post-mortem interval (PMI) is of particular importance in investigating a suspect's alibi in court. A forensic expert can use different methods to estimate the PMI. This research focuses on who is considered an expert in court and whether the methods used to estimate the PMI are reliable. In this study, the methods used to estimate the PMI and the experts consulted, available in Dutch jurisprudence, in the period 2010-2019 were investigated. Ninety-four judicial cases were included and multiple experts and methods of estimating the PMI were found. As part of this study, the methods that were used to estimate the PMI in court were subjected to the Daubert criteria. Of these methods, only the Henssge nomogram and entomological methods met the Daubert criteria. However, the methods are only useful when applied by the right forensic expert and in the right manner. Unfortunately, this was not always the case.

Objective Estimating carcinogen exposure prevalence is important for preventing occupational cancers. To develop the Korean version of CARcinogen EXposure (CAREX), a carcinogen surveillance system used in many countries, we estimated lead exposure prevalence in the Korean working population. Methods We used three Korean nationwide data sources to obtain objective database-derived prevalences of lead exposure across industries: airborne lead measurement data from the work environment measurement database (WEMD), blood lead measurement data from the special health examination database (SHED), and lead exposure prevalence computed using data from the work environment condition survey (WECS), which is a nationwide occupational exposure survey. We also asked a panel of 52 experts with ≥20 years of experience in industrial hygiene practice for their judgment about lead exposure prevalence across industries after they reviewed the database-derived prevalences computed from the three exposure databases. We developed and compared various estimation methods for combining the experts' judgments. The 2010 census was used as the reference population to estimate the number of lead-exposed workers in 228 industries by multiplying the exposure prevalence by the number of workers in each industry. Results The database-derived prevalences of lead exposure in the 228 industries were calculated using data collected between 2009 and 2011 from the WEMD and SHED and from the 2009 WECS. From the various estimation methods assessed, the median values of experts' responses were selected as our estimates of lead exposure prevalence in each industry. As a result, it was estimated that 129,250 Korean workers were exposed to lead in 2010. Conclusions Based on objective databases, we developed a method for estimating exposure prevalence for the CAREX system by combining experts' judgments. This work may offer an unbiased approach to the development process that accounts for the uncertainty in exposure.