Predicting Auditor Changes With Financial Distress Variables: Discriminant Analysis And Problems With Data Mining Approaches

Abstract

Our study extends previous research that uses financial distress factors in predicting auditor changes by evaluating the effectiveness of the traditional discriminant analysis model, not used in previous auditor change studies, and by highlighting the importance of evaluating the likelihood that data mining approach classification results occurred by chance. Significance of individual predictor variables, as well as of the full set of 13 financial variables, can be tested using discriminant analysis. Kwak et al. (2011) document overall classification accuracy rates ranging from 61 to 63.5 percent for the four data mining models they compared but did not address whether these rates occurred by chance. Using Kwak et al.’s (2011) data set of firms changing auditors in 2007 or 2008 and matching non-auditor change firms, our discriminant analysis test results show overall accuracy rates of less than 56 percent and true positive rates over 85 percent, but these rates are influenced by a disproportionate number of non-auditor change firms being classified as auditor change firms. Individual predictor variables that are important in the discriminant equation based on standardized canonical coefficients include losses (LOSS) and no payment of dividends (DIV) in the year prior to the auditor change, retained earnings as a percent of total assets (RE/TA), and earnings before interest and taxes as a percent of total assets (EBIT/TA). The Kappa statistic and AUC metrics for all 13 data mining algorithms we used indicate that classifications using these algorithms are no better than random classifications.