Detailed Comparison Of Closed Testing Procedures For Pairwise Testing Of Means (pp. 143-172)
Authors: Ramsey, Philip H. (Queens College of CUNY)
Abstract: A Monte Carlo simulation was conducted to compare 17, pairwise multiple comparison procedures. The number of means varied from 4 to 6 and the samples sizes varied from 4 to 16. Procedures were evaluated on the basis of Type I error, any-pair power and all-pairs power. All procedures were found to provide adequate control of Type I errors. No procedure was found to be uniformly most powerful. A modification due to Hayter (1986) of Fisher’s LSD was found to provide the greatest any-pair power in most cases. The greatest all-pairs power was usually provided by a partition based version of Peritz’s method. It is noted that HSD is never most powerful unless one is willing to apply it in the presence of a nonsignificant overall F test. The size of the power advantage of various procedures was discussed. Even small differences in power are worth noting if they are consistent and procedures are of comparable difficulty.