Search

Construct-based models have become a mainstay of management and information systems research. However, these models are likely overfit to the data samples upon which they are estimated, making them risky to use in explanatory, prescriptive, or predictive ways outside a given sample. Empirical researchers currently lack tools to analyze why and how their models may not generalize out of sample. We propose a composite overfit analysis (COA) framework that applies predictive tools to describe the sources and ramifications of overfit in terms of the focal concepts important to empirical researchers: cases, constructs, and causal paths. The COA framework begins by using a leave-one-out crossvalidation procedure to identify cases with unusually high predictive error given their in-sample fit—a difference we describe as predictive deviance. The framework then employs a novel deviance tree method to group deviant cases that have similar predictive deviance and for similar theoretical reasons. We then employ a leave-deviant-group-out method, which sequentially analyzes how each deviant group affects model parameters, thereby identifying potentially unstable paths in the model. We can then infer descriptive reasons for why and how overfit affects a given model and data sample using the grouping criteria of the deviance tree, construct scores of deviant groups, and resulting unstable paths. These insights allow researchers to identify unexpected behavior that could define boundary conditions of their theory or point to new theoretical phenomena. We demonstrate the practical utility of our analytical framework on a technology adoption model in a new context. This paper was accepted by Dongjun Wu, information systems. Funding: This work was partially supported by the Ministry of Science and Technology, Taiwan [Grants 109-2811-H-007-503 and 108-2410-H-007-091-MY3] and by the 2021 Arts and Social Sciences Benefaction Fund of Trinity College Dublin, Ireland. Supplemental Material: The data files and online appendix are available at https://doi.org/10.1287/mnsc.2023.4705. [ABSTRACT FROM AUTHOR]

Forecasting hierarchical or grouped time series using a reconciliation approach involves two steps: computing base forecasts and reconciling the forecasts. Base forecasts can be computed by popular time series forecasting methods such as exponential smoothing (ETS) and Autoregressive Integrated Moving Average (ARIMA) models. The reconciliation step is a linear process that adjusts the base forecasts to ensure they are coherent. However, using ETS or ARIMA for base forecasts can be computationally challenging when there are a large number of series to forecast, as each model must be numerically optimized for each series. We propose a linear model that avoids this computational problem and handles the forecasting and reconciliation in a single step. The proposed method is very flexible in incorporating external data. We illustrate our approach using a dataset on monthly Australian domestic tourism, as well as a simulated dataset. We compare our approach to reconciliation using ETS and ARIMA, and show that our approach is much faster while providing similar levels of forecast accuracy. Supplementary files for this article are available online. [ABSTRACT FROM AUTHOR]

In response to growing recognition of the social impacts of new artificial intelligence (AI)-based technologies, major AI and machine learning (ML) conferences and journals now encourage or require papers to include ethics impact statements and undergo ethics reviews. This move has sparked heated debate concerning the role of ethics in AI research, at times devolving into name calling and threats of "cancellation." We diagnose this conflict as one between "atomist" and "holist" ideologies. Among other things, atomists believe facts are and should be kept separate from values, while holists believe facts and values are and should be inextricable from one another. With the goal of reducing disciplinary polarization, we draw on numerous philosophical and historical sources to describe each ideology's core beliefs and assumptions. Finally, we call on atomists and holists within the ever-expanding data science community to exhibit greater empathy during ethical disagreements and propose four targeted strategies to ensure AI research benefits society., Competing Interests: The authors declare no competing interests., (© 2022 The Author(s).)