Linear Information Models: An Introduction

Relative entropy identities yield basic decompositions of cat- egorical data log-likelihoods. These naturally lead to the development of information models in contrast to the hierarchical log-linear models. A recent study by the authors clarified the principal difference in the data likelihood analysis between the two model types. The proposed scheme of log-likelihood decomposition introduces a prototype of linear information models, with which a basic scheme of model selection can be formulated accordingly. Empirical studies with high-way contingency tables are exem- plified to illustrate the natural selections of information models in contrast to hierarchical log-linear models. Analysis of contingency tables with multi-way classifications has been a fun- damental area of research in the history of statistics. From testing hypothesis of independence in a 2 × 2 table (Pearson, 1904; Yule, 1911; Fisher, 1934; Yates, 1934) to testing interaction across a strata of 2 × 2 tables (Bartlett, 1935), many discussions had emerged in the literature to build up the foundation of statisti- cal inference of categorical data analysis. In this vital field of applied statistics, three closely related topics have gradually developed and are still theoretically incomplete even after half a century of investigation. The first and primary topic is born out of the initial hypothesis testing for independence in a 2 × 2 table. From 1960's utill the 1980's, Fisher's exact test re- peatedly received criticism for being conservative due to discrete nature (Berkson, 1978; Yates, 1984). Although the arguments in favor of the unconditional tests essentially focused on using the unconditional exact tests in the recent decades, the reasons for preferred p values and sensitivity of the unconditional tests have not been assured in theory. In this respect, a recent approach via information theory proved that the power analysis of unconditional tests is not suitable for