We compared Analysis of Variance (F) and the Welch test (W) with their respective permutation versions (PF and PW) in terms of Type I error rate (alpha) and test power (1-beta) by Monte Carlo simulation technique. Simulation results showed that when the variances were homogeneous, the permutation versions of F and W tests displayed more reliable results in terms of protecting Type I error rate at nominal level, regardless of distribution shape and sample size. Violation of homogeneity of variances adversely affected all tests. Regardless of sample size and effect size, the PF test was slightly more powerful compared to the F test as long as the variances were homogeneous and the distributions were skewed (chi(2)(3) and Exp [0.75]). The PF and F tests had similar power levels when the distributions were symmetrical (Beta (5.5)). The W test was more powerful with homogenous variances, while the PW test was slightly superior with heterogonous variances except for unbalanced sample sizes (i.e., 5:10:15).