Random Treatment Assignment Makes Treatment and Control Groups Comparable When Sample Size is Large Enough
By Elena Llaudet, co-author of Data Analysis for Social Science: A Friendly and Practical Introduction (Princeton University Press, 2022)
To simplify matters, imagine there were only five types of individuals in the world: orange, blue, pink, green, and purple. If we were to assign individuals to two different groups--the treatment group and the control group--at random (that is, through a random process such as the flip of a coin), the two groups would contain similar proportions of each type of individual as long as the sample size is large enough. Let's examine this further:
When the sample size is very small, the two groups will likely be different because the sample size is simply too small to create two groups with similar compositions. As shown in the graph on the right, for example, when the sample size is of 20 individuals (4 orange, 2 blue, 4 pink, 6 green, and 4 purple), the treatment group could end up with all the pink invididuals and the control group could end up with all the blue individuals, making the two groups clearly not comparable eventhough the groups were created at random.
(Note about the graph: From left to right, the histrograms show the number of individuals from each type that are (a) in the full sample, (b) in the treatment group, and (c) in the control group. While n stands for the total number of individuals in the sample, n_treatment stands for the total number of individuals in the treatment group, and n_control stands for the total number of individuals in the control group.)
Now, let's increase the sample size and observe how the composition of the treatment and control groups start to approximate each other as a result. (As we can see in the animated GIF on the right, as the sample size increases, the histogram of the treatment group and the histogram of the control group will start to resemble each other.) Once the sample size reaches 300, the two groups are quite similar. They are now both composed of roughly 20% orange individuals, 10% blue individuals, 20% pink individuals, 30% green individuals, and 20% purple individuals. This is because by assigning individuals to the two groups at random, about half of the individuals from each type end up in the treatment group, and the other half end up in the control group. This explains why in randomized experiments, in which, by definition, we assign individuals to treatment and control groups at random, the two groups should be comparable, that is, have similar proportions of pre-treatment characteristics, if the sample size is large enough.
(Looking for more animated GIFs and/or interactive graphs to help explain statistical concepts? Check my website.)
