Experimental Design

1. Components of Experimental Design

Experimental design refers to how an experiment is carried out. Many experimental designs include a control group and a treatment group to compare effects of treatment (exercise, drug, video watching, etc.). You can have a good design of an experiment or a poor design of an experiment.

Good experimental design will have these three components:

1. Control
2. Randomization
3. Replication

terms to know

Experimental Design

The way in which an experiment is carried out. A good design has key elements of randomization, replication, and control.

Treatment

Something the researchers administer to the subjects or experimental units.

1a. Control

Control means holding everything else besides what you're trying to measure constant. The purpose is to determine whether or not your treatment is effective. In other words, if there is an observable difference between groups, is it due to the treatments or due to a confounding variable? It is important to control all other variables to help limit confounding.

One common way to control an experiment is with a control group. A control group is a set of samples that do not receive the treatment under consideration. For instance, if you were studying a new cancer treatment, a control group might get the standard cancer treatment care, while the treatment group receives the new drug or treatment being evaluated. In this case, the control group allows researchers to measure the effectiveness of the treatment against a group that is otherwise similar. Generally, the participants won't know if they are in the control or treatment group, as this knowledge can affect the results.

watch

IN CONTEXT

Suppose you are a farmer, and you want to try a new fertilizer in your field. One thing you could do is choose 10 fields with similar soil nutrients, sunlight, and water—all variables that could affect the crop growth.

You could then apply the old fertilizer to five fields and the new fertilizer to the other five. By keeping all the other variables—soil nutrients, sunlight, water—consistent, the differences between the fields can be isolated and attributed to the old fertilizer or the new fertilizer.

Does the new fertilizer work? Is it effective? This is the idea behind controlling for all of these other variables.

terms to know

Control

The principle of experimental design that requires that other variables which may confound the experiment be held constant between the treatment groups so that any differences in the groups can be attributed to the different treatments.

Control Group

A group included in an experiment that does not receive the treatment under consideration and against which other experimental results can be compared and validated.

1b. Randomization

The second big idea of experimental design is randomization. The treatments must be assigned to the subject using a random process, otherwise known as “randomization.” The purpose of random assignment is to try and filter out all the other sources of variation that you couldn't anticipate controlling for.

EXAMPLE

Referring to the farmer example, even though you made the fields as similar as possible with respect to water, sunlight, and soil, it's possible that there is a variable that you didn't think to control for. Perhaps some fields had moles under the ground, and that would affect how the crops grow. How would you know to control for moles?

By randomly assigning treatments to the fields, you can hopefully get some fields with moles in the groups of fields with both the new and old fertilizer. Randomization smooths out those effects that other variables might bring into the equation.

hint

Randomizing also helps avoid bias because you can’t be tempted to assign treatments to the experimental units you think might give favorable outcomes.

Randomization in an experiment does not really achieve the same purpose as a random selection in a sample. When you do a simple random sample, the idea is to get a sample that's representative of the population. In an experiment, the purpose of randomly assigning individuals to groups is to filter out unknown sources of variation. The assignment in an experiment, however, is fairly similar to the way you would randomly select in a sample.

term to know

Randomization

The principle of experimental design that requires that the subjects/experimental units be assigned to groups using some random process. This ensures that the two groups are roughly equal prior to assigning treatments.

1c. Replication

Replication is the last key idea in experimental design, which basically states that a bigger sample is better. Repeating the experiment on multiple subjects or experimental units is a better idea than doing a few. Why is that?

A larger size of the experiment means it's more likely that you can find trends that perhaps you wouldn't have found in a smaller experiment. The more you replicate, and the more experimental units you can get into your experiment, the more likely it is that you're going to find the true trends that arise, rather than some freak anomaly.

think about it

What if the farmer could have just found two fields that were similar to each other, instead of 10 fields, and randomly assigned one to get the new fertilizer and one to get the old. Isn't it possible in that case that maybe the field with the old fertilizer does very well just by random chance?

This would make it seem like the new fertilizer is not effective when perhaps it is. Or the opposite could happen, where it seems like the fertilizer is effective when it's not. It would be better to randomly assign five plots, as opposed to just two, as it is more likely that the farmer is going to find trends among those five plots that are more valid.

term to know

Replication

Repeating the experiment on multiple subjects/experimental units. This principle of experimental design states that a larger experiment with more subjects/experimental units will allow us to more clearly see differences between the treatments.