One-way repeated measures ANOVA

1 Continuous Dependent Variable with normal distribution

1 (Multi-Level) Categorical Independent Variable

	Task Completion time
Subject	Interface 1	Interface 2	Interface 3
1	12.9	16	4
2	5.7	7.5	4.3
3	16	16	5
4	14.3	15.7	5.3
5	2.4	13.2	4.9
…

One-way repeated measures ANOVA compares how a within-subjects experimental group performs in three or more experimental conditions. The ANOVA compares whether the mean of any of the individual experimental conditions differ significantly from the aggregate mean across the experimental conditions.

In ‘Can You See What I Hear? The Design and Evaluation of a Peripheral Sound Display for the Deaf’ [14], a Berkeley research team evaluates the distraction level of two peripheral interfaces when subjects complete a demanding visual task. The intent of the research is to evaluate peripheral information displays for the hearing impaired. The research team develops two methods for helping deaf individuals to sense non-vocal sounds in their environment. Two peripheral displays are implemented as windows on users’ computer monitors. Both displays show sound waves and the direction they are coming from, and depict activities such as a person walking by or a telephone ringing at a neighboring desk. The intent is to help the hearing-impaired individual become aware of sounds produced by these actions, as currently there are no good devices available which allow them to recognize non-vocal sounds outside their line of sight.

In the experiment, subjects perform an attention-demanding primary task on a standard computer monitor. In one experimental condition, peripheral sound wave interface #1 is visible to the subject. In the second condition, peripheral sound wave interface #2 is visible to the subject. In the third condition, the user performs only the primary task, with no peripheral display visible.

The research team chose one-way repeated measures ANOVA to interpret the experimental results, which initially appears to be the correct statistical test. First, the choice seems appropriate because the intent is to measure the variance in distraction across the subjects’ completion of the 3 test exercises. Second, the test appears to be the correct choice because the dependent variable, distraction level, is measured by the subjects’ performance on the primary task, and their score is measured as a continuous variable. Finally, the independent variable, the peripheral display condition, has 3 categories, and repeated measures ANOVA is appropriate for independent variables with 3 or more categories.

The study included only 8 participants, however, and because of this, use of one-way repeated measures ANOVA becomes questionable. One-way repeated measures ANOVA requires the dependent variable follow a normal distribution, and with a sample size of 8, it is virtually impossible to prove a normal distribution. The central limit theorem can be used to prove a normal distribution in sample sizes of 30 or more, and when the sample size is less than 30, researchers can render a normal distribution by plotting the dependent variable and showing it appears to be normally distributed, within reason [6]. With a sample size of 8, however, a visual plot could be too scarce to prove a normal distribution. The research team, therefore, most likely chose to assume a normal distribution. If this is the case, the researchers did not note their assumption or provide a basis for it.

The Friedman analysis of variance by ranks is an alternative to one-way repeated measures ANOVA, because it does not require the dependent variable to follow a normal distribution. Using the Friedman (or any other non-parametric test), however, it is nearly impossible to achieve p < .05 with a sample size smaller than 12, because the tests are more exacting than their parametric counterparts. Because of this, often the only option is to increase the sample size. To improve their research from an experimental design standpoint, the Berkeley team could increase their sample size to at least 12, and apply a Friedman test rather than one-way ANOVA.

Values to report:

· Degrees of freedom

· F value

· p value