Hypothesis Testing & The Scientific Method
Definitions:
Theory: A unified explanation for a range of phenomena.
Hypothesis: Any aspect of the testing situation that changes. Different variables a re relevant for different hypotheses, and different testing methods. For instance, if you are collecting data on the energy level of college students, time of test is an important variable. Time of test is less important if you are collecting grammaticality judgments.
Independent Variable: The variables manipulated in an experimental situation. In psycholinguistics, this is often something like sentence structure or the relationship between two stimulus items. We often talk about different levels, or conditions, of the independent variable.
Dependent Variable: The response that is measured, the data of interest. It may be a subjective intuition, a response time, a behavioral response, etc.
Subject: The person who provides the response of interest.
Chance Variation: The amount of variation in the dependent variable (the response) that can be predicted based purely on chance. For instance, if you asked someone to pronounce the word you show them as rapidly as possible, there would be some variation in their response time even if you kept showing them the same word over and over again. Some of this variation would be due to chance.
Reliability: Statistical evidence for a relationship between the independent variable and the dependent variable. Statistical tests are used primarily to rule out chance variation as an explanation for a particular pattern of responses.
Replication: The same outcome over numerous trials. One way to evaluate research is to attempt to replicate it.
Falsification: When a hypothesis has been proved incorrect. If an hypothesis is not falsifiable, it is not testable. For example, the hypothesis "ESP exists" is not falsifiable but "ESP does not exist" is. "ESP exists" is not falsifiable because even if you never find evidence of ESP, you may just not be performing the appropriate tests. According to most philosophers of science, well-formed hypotheses can be supported or proved incorrect, but never proved correct. Thus, if you perform numerous tests and do not find evidence of ESP, you have supported the hypothesis that "ESP does not exist" but you have not proven it.
Parsimony: Without unnecessary elements; simple. The most elegant theory fully explains the phenomena in the simplest possible way (e.g., with the fewest number of rules).