Avoiding Statistical Mistakes

Abstract

Avoiding statistical mistakes is important for educators at all levels.  Basic concepts will help you to avoid making mistakes using statistics and to look at data with a critical eye.  Statistical data is used at educational institutions for many purposes.  It can be used to support budget requests, changes in educational philosophy, changes to programs, and addition of new programs.  Therefore, it is important to be sure that decisions based on statistical data are valid.  Also, it is important to be certain that the data has been presented in a fair and unbiased way.  Otherwise, the changes and decisions may not reflect the best interests of the educational institution or its constituents.  Many faculty, staff, and administrators at colleges and universities use statistics to support their findings.  Most conclusions are based on the data collected and the analysis.  If either the data collected or the analysis is faulty, the conclusions may be invalid.  Unfortunately, misuses and abuses are common.  This paper attempts to present basic concepts that will help you to avoid making mistakes when using statistics and to look at data with a critical eye.  No extensive knowledge of mathematics or statistics is needed to be able to judge the validity of the data.  The following questions will be answered: How can administrators avoid making invalid decisions when presented with statistical data? And, how can researchers avoid making mistakes that undermine the validity of their data?  First, we will look at sources of bias.  Experiments and the collection of data can be influenced by many different sources of bias.  These sources include the population used for the study, the sample size and method of selection, the funding of the experiment, statistical assumptions, the publication source, and the biases of the researcher.  Examples from within education will be used to gain an understanding of the biases that can occur.  Second, we will look at the methods used to analyze and display the statistical data.  This can include descriptive statistics and more advanced analysis as well.  The characteristics of good graphic presentations of data will be compared to those that are flawed.  Examples of graphs that present data fairly will be shown as well as graphs that distort the data.  The differences between ordinal and ratio data types will be emphasized.  Appropriate uses of ordinal data will be discussed.  Also for ratio data a discussion of the meaning of average and importance of measures of variance will be examined.  We will also look at the crafting of surveys and the choosing of samples.  Finally, we will look at how the results can be interpreted.  Valid data and analysis can still result in incorrect interpretation.  We will examine examples of cases where misinterpretation is common and results in invalid conclusions.  It is important to realize that statistical data and analysis is subject to many forms of error and misinterpretation.  All consumers of statistical data need to look carefully at the data and the analysis.  Numbers do lie!  However, the use of statistics and data is very important in making decisions.  By using statistics correctly, we can significantly impact education for the better.  This goal can be achieved by those who are knowledgeable about statistics and its use.  This paper presents the tools for using statistics correctly and identifying problems with statistical data.