The Subject Matter of Experiments Essay Example for Free

The Subject Matter of Experiments Essay A well-designed experiment tells us that changes in the explanatory variable cause changes in the response variable. More exactly, it tells us that this happened for specific subjects in the specific environment of this specific experiment. No doubt we had grander things in mind. We want to proclaim that our new method of teaching math does better for high school students in general or that our new drug beats a placebo for some broad class of patients. Can we generalize our conclusions from our little group of subjects to a wider population? The first step is to be sure that our findings are statistically significant, that they are too strong to often occur just by chance. That’s important, but it’s a technical detail that the study’s statistician can reassure us about. The serious threat is that the treatments, the subjects, or the environment of our experiment may not be realistic. For example, a psychologist wants to study the effects of failure and frustration on the relationships among members of a work team. She forms a team of students, brings them to the psychology laboratory, and has them play a game that requires teamwork. The game is rigged so that they lose regularly. The psychologist observes the students through a one-way window and notes the changes in their behavior during an evening of game playing. Playing a game in a laboratory for small stakes, knowing that the session will soon be over, is a long way from working for months developing a new product that never works right and is finally abandoned by your company. Does the behavior of the students in the lab tell us much about the behavior of the team whose product failed? Psychologists do their best to devise realistic experiments for studying human behavior, but lack of realism limits the usefulness of experiments in this area. When experiments are not fully realistic, statistical analysis of the experimental data cannot tell us how far the results will generalize. Experimenters generalizing from students in a lab to workers in the real world must argue based on their understanding of how people function, not based just on the data. It is even harder to generalize from rats in a lab to people in the real world. This is one reason why a single experiment is rarely completely convincing, despite the compelling logic of experimental design. The true scope of a new finding must usually be explored by a number of experiments in various settings. A convincing case that an experiment is sufficiently realistic to produce useful information is based not on statistics but on the experimenter’s knowledge of the subject matter of the experiment. The attention to detail required avoiding hidden bias also rests on subject matter knowledge. Good experiments combine statistical principles with understanding of a specific field of study.