Statistical Data Analysis and Entropy

This book reconsiders statistical methods from the point of view of entropy, and introduces entropy-based approaches for data analysis. Further, it interprets basic statistical methods, such as the chi-square statistic, t-statistic, F-statistic and the maximum likelihood estimation in the context of entropy. In terms of categorical data analysis, the book discusses the entropy correlation coefficient (ECC) and the entropy coefficient of determination (ECD) for measuring association and/or predictive powers in association models, and generalized linear models (GLMs). Through association and GLM frameworks, it also describes ECC and ECD in correlation and regression analyses for continuous random variables. In multivariate statistical analysis, canonical correlation analysis, T²-statistic, and discriminant analysis are discussed in terms of entropy. Moreover, the book explores the efficiency of test procedures in statistical tests of hypotheses using entropy. Lastly, it presents an entropy-based path analysis for structural GLMs, which is applied in factor analysis and latent structure models. Entropy is an important concept for dealing with the uncertainty of systems of random variables and can be applied in statistical methodologies. This book motivates readers, especially young researchers, to address the challenge of new approaches to statistical data analysis and behavior-metric studies.