Use LEFT and RIGHT arrow keys to navigate between flashcards;
Use UP and DOWN arrow keys to flip the card;
H to show hint;
A reads text to speech;
85 Cards in this Set
- Front
- Back
Hypothesis Test
|
- is a statistical method that uses sample data to evaluate a hypothesis about a population
- sample data --> inference about population |
|
Steps to Hypothesis Test (simple)
|
1. State a hypothesis about population (usually concerning the value of a parameter)
2. Use hypothesis to predict that characteristics that the sample should have (the sample should be reflective of population) 3. Select a random sample from population 4. Compare sample data with the prediction that was made from the hypothesis ... sample mean is consistent with prediction we conclude the hypothesis is reasonable vice versa |
|
What are the known and unknown populations?
|
- known = exist before treatment
- unknown = after the treatment is administered; hypothetical population if treatment were administered |
|
What is one basic assumption made about the effect of the treatment condition?
|
- If the treatment condition has any effect, it is simply to add/subtract a constant amount from each individual score
|
|
What is the consequence of this basic assumption?
|
- the adding or subtracting of the constant after the treatment condition does not change the shape of the population or the standard deviation
|
|
What is the goal the hypothesis test?
|
- to determine whether the treatment has any effect on the individuals in teh population
|
|
What does a research study involve?
|
1. selecting a sample
2. treating the sample 3. recording scores |
|
What two "real" components allow is to make inferences about the unknown population/ hypothetical population?
|
- original population
- treated sample |
|
Hypothesis Test
|
1. State the Hypothesis
2. Set the Criteria for a Decision 3. Collect Data and Compute Sample Statistics 4. Make a Decision |
|
State the Hypothesis
|
- state two opposing hypothesis about the population parameters
|
|
Null Hypothesis
|
- most important of the two hypothesis
- treatment has no effect - no change/effect/difference (population mean stays the same) |
|
Alternative (scientific) Hypothesis
|
- treatment had an effect
- unknown population mean does not equal the known population mean |
|
Directional vs Non Directional Test
|
a difference in one direction vs any difference
|
|
Step 2: Set the Criteria for a Decision
|
- Data from sample will be used to evaluate the credibility of the null hypothesis
|
|
What does it mean to formalize the decision making process?
|
- use the null hypothesis to predict the kind of sample mean that ought to be obtained
- sample means consistent with the null and vice versa - provide boundaries for what is "near the mean" what is "far away from the mean" - Dividing the DSM into two sections |
|
The DSM is divided into...
|
(2 sections)
1. Sample means that are likely to be obtained if null is true; that is, sample means that are close to the null hypothesis; body 2. Sample means that are very unlikely to be obtained if null is true; that is, sample means that are very different from the null hypothesis; tails |
|
If the null is true where does the sample mean fall?
|
- inside the high probability section of the distribution "close" to the population mean
|
|
The Alpha Level
|
- level of significance
- small probability used to identify low probability samples |
|
Alpha Levels by the Numbers
|
- = 0.05 (aka 5%) z = ±1.96
- = 0.01 (aka 1%) z = ±2.58 - = 0.001 (aka 0.1%) z = ±3.30 |
|
Critical Region
|
- extremely unlikely values, as defined by the alpha level
- reflect values not consistent with the null - sample mean located in critical region; reject the null |
|
If the alpha level is increased from a = .01 to a = .05, the size of the critical region also increases. True or False?
|
- True, a larger alpha means that the boundaries for the critical region move closer to the center of the distribution
|
|
Step 3: Collect Data and Compute Sample Statistics
|
- find z score that describes exactly where the sample mean is located relative to the hypothesized population mean from null
(this hypothesized population mean is the DSM of n = sample size where null were true) - plot z score of sample mean relative to null mean which is a DSM where n = sample size and the mean remains the same from the known population |
|
Step 4: Make a Decision
|
ONLY 2 CHOICES
- reject the null (treat has an effect // z score in the critical region) - accept the null ( treat has no effect // z score not in the critical region) |
|
THE ONLY TWO CONCLUSIONS
|
- reject the null hypothesis
- fail to reject the null hypothesis |
|
Trial Metaphor
|
- not trying to prove innocent or guilty; your prove guilty or not guilty
~ reject the Null or fail to reject the null |
|
Freestyle Hypothesis Test
|
- we are reorganizing the population into a population of samples in order to make a statement about the population with out testing every individual ; we are then finding efficacy of a treatment by comparing the average our sample to what would be expected of a sample without the treatment; we then reject or fail to reject a null hypothesis
|
|
Generally, a large value for a test statistic indicates...
|
- large discrepancy between sample and null hypothesis
|
|
what is the null hypothesis ...
|
- its reinterpretation of the known population in terms of the treatment sample size
|
|
Large deviation / Critical Hits / far from 0 / in the tails tell the researcher...
|
- its is likely the treat condition had an effect
- the null hypothesis should be rejected |
|
What can get fucked in the research study to lead to an erroneous conclusion?
|
- picking an extreme sample not representative the population can cause the a strong treatment effect
ex if you randomly chose 20 English lit graduates to show the effect of flashcards on vocabulary improvement |
|
Type I Error
|
- occurs when a researcher rejects a null hypothesis that is actually true
- typically means that the researcher is claiming a functional relationship when their is not one - mislead by sample |
|
Type I Error occurs when the researcher unknowingly obtains a ...
|
- extreme sample
|
|
Can a sample mean be obtained that falls in the critical region from a sample that does not receive the treatment condition?
|
yes, in fact statistically it is improbable that it will never occur given that the critical region is apart of the null hypothesis
|
|
Alpha Level
|
- the probability that the test will lead to a Type I error ( a failure to accept the null hypothesis)
- determines the probability of obtaining sample data in the critical region even though the null hypothesis is true |
|
Essentially the alpha level is used to
|
- establish the probability that a sample was effected by the treatment condition by providing a line of demarcation
|
|
The probability that Type I error will occur is equal to ...
|
- the alpha level/ critical region
|
|
Type II Error
|
- failed to reject the null
- hypothesis test failed to detect a real treatment effect - unlike Type I error whose probability is factored into the null hypothesis there is not absolute measure of probability for Type II error - influenced by a variety f factors - researcher can refit and refire - represented by greek letter betta ~ (B) |
|
Two Possible Outcomes of Hypothesis Test
|
1. the sample data provide sufficient evidence to reject the NULL and conclude that treatment has an effect
2. the sample data does not provide enough evidence to reject the NULL; fail to reject the NULL and treatment does not appear to have an effect |
|
By choosing the alpha level the researcher directly determines ...
|
- the probability to Type I error (sample scoring critical hit --> false functional relationship)
- researchers determine their own risk of a false report |
|
Selecting an alpha level has two direct consequences...
|
1. boundaries for critical region
2. probability of Type I Error |
|
State the consequences of raising the alpha level and decreasing the alpha level..
|
- raising the alpha level increases Type I error by expanding the critical region
- by lowering the alpha level you decrease the chances of concluding that the treatment has an effect thus increasing Type II error |
|
what type of error would a small treatment effect produce ...
|
Type II... failure to reject the NULL
|
|
Reporting the Conclusion
|
- report whether the treatment was statistically significant
- z = raw score, p < or > alpha Example The treatment with alcohol had a significant effect on the birth weight of newborn rats. z= 3.00, p < .05 |
|
Result is said to be Significant or Statistically Significant
|
- unlikely to occur when the null hypothesis is true
- result is sufficient to reject the null hypothesis |
|
what does p > .01 entail?
|
it means that the probability of that that treatment condition had an effect is greater than the alpha 1%
- a significant result will always be reported as a probability less than the alpha level |
|
Factors that Influence A Hypothesis Test
|
- size between sample mean and original score (numerator of z score)
- variability of scores (deviations or variance and influences the denominator of the z score) - number of scores in sample (denominator) |
|
Size of Mean Difference
|
- larger the difference the larger the treatment effect, which produces a larger z score and increases the likelihood of finding a significant effect
|
|
The Variability of Scores
|
- high variability makes it hard to see patterns generally
- increasing the variability of the scores produces a larger standard error and a smaller value (closer to zero) for the z score |
|
The Number Scores in the Sample
|
- increasing the number of scores in the sample produces a smaller standard error and a larger value for the z score
|
|
The Fundamental Assumptions Behind Hypothesis Testing
|
1. Random Sampling (equal chance)
2. Independent Observations (with replacement) 3. The Value of the standard deviation is unchanged by treatment (+/- a constant) 4. Normal Sampling Distribution (shape) |
|
Random Sampling
|
- selected randomly to ensure that the sample is representative of the population
|
|
Independent Observations
|
- the probability of occurrence of scores are affected by each other
(ex with replacement) ` |
|
The Value of the standard deviation is unchanged by treatment
|
- in order to find standard error and to make inferences about the treatment population you need to know the deviation of the treatment population
- the assumption is made by researchers that the deviation does not change thus allowing them to find standard error and find a z score for the treatment sample - also the implication that the treatment will act as a constant on the population |
|
Normal Sampling Distribution
|
- in order to identify the critical region the distribution must be normal
|
|
What is the most critical factor in the deciding whether the treatment has an effect/ reject the null?
|
- the size difference between the treated sample and the original population
|
|
What is the major difference between a one tailed test and a two tailed test?
|
- a one tailed test allows you to reject the NULL when the difference between the sample and the pop. is relatively small, provided the difference is in a specific direction
- a two tailed test requires relatively large difference independent of direction |
|
Alpha boundary difference between one tailed and two tailed test at .05
|
- one tail = 1.64 (in one direction
- two tail = 1.96 (+/-) |
|
Benefits and Downfalls of Two Tailed (non directional) Test
|
- more rigorous/ more evidence needed to reject NULL
- stronger demonstration of treatment effect - best when no direction is specified or their are competing predictions - might fail to prove treatment significance (Type II Error) |
|
Benefits and Downfalls of One Tailed (directional) Test
|
- more sensitive (smaller treatment effect can produce statistical significance)
- more precise because they test a directional effect - should be used when their is strong justification for making a directional hypothesis - can lead to Type I Error |
|
Two Major Limitations of Hypothesis Testing (in regards to establishing significance)
|
- focus of hypothesis test is on data rather than the hypothesis
- demonstrating significance does not indicate magnitude of the treatment effect |
|
The First Limitation of the Hypothesis Testing
|
- rejecting the NULL makes a statement about the data not the hypothesis
- Rejecting the NULL with a = .05 does not justify a conclusion that the probability of the NULL being true is less 5% - conclusion reports a conditional statement about the probability about the sample in relationship to the NULL but not a statement about the NULL |
|
The Second Limitation of Hypothesis Testing
|
- Statistical significance does not provide any real information about the absolute size of a treatment effect
- the test is making a relative comparison: the size of the treatment effect is being evaluated relative to standard error |
|
"The test is making a relative comparison: the size of the treatment effect is being evaluated relative to standard error...." what are the implications behind this fact?
|
- if the sample size is large enough, any treatment effect, no matter how small, can be enough for us to reject the NULL
|
|
Absolute size of treatment effect is ...
|
- not evaluated.
|
|
Effect Size
|
- intended to provide a measurement of the absolute magnitude of a treatment effect, independent of the size of the sample being used
|
|
Cohen's D
|
- mean difference/ deviation
- treatment(unknown) population mean - no treatment(known) population mean / deviation * remember deviation is assumed constant |
|
Cohen's D is an expression of effect size in terms of ...
|
- deviations
- deviation standardizes Cohen's D |
|
why is a actual Cohen's D impossible in research?
|
- you need the population mean after treatment
|
|
Actual Calculations of Cohen's D
|
M(treatment) - U(treatment) / deviation
|
|
d= 0.4 indicates ...
|
- treatment effect size was 0.4 standard deviations
|
|
Even if their is substantial significance in treatment effect their can still substantial _____ between non treatment and treatment distributions.
|
- Overlap
|
|
Cohen's D measures...
|
- degree of separation between two distributions in terms of standard deviation
- does not consider sample size |
|
Power
|
- the probability that the test will correctly REJECT the NULL if the treatment has an effect
- find a treatment effect |
|
Power (by the numbers)
|
1 - beta
|
|
Power is...
|
- the probability of correctly rejecting the NULL
|
|
If the power of a test is 70% what is the probability of Type II error
|
- 30%
|
|
Type I Error is to the Critical Region as Type II Error is to ....
|
- Power
|
|
Factors that Effect the Power
|
- Sample Size
- Alpha Level |
|
To calculate the power of a test you must determine...
|
~what portion of the distribution on the right-hand side~
1. Locate the boundary for the critical region 2. Find the probability value in the unit table multiply the standard error by the boundary of the critical region (any point value over this number in the critical region) 3. find what portion of the treated DSM is greater than the critical region boundary point score 4. Find z score of the critical region boundary point score on the treated DSM 5. use the unit table and report the probability corresponding to shaded area |
|
The most critical step in determining the power ...
|
- restating the alpha boundary as a z score of the unknown/treated DSM
|
|
As the effect size increases....
|
- the DSM of the right hand size moves farther to the right
- more samples are beyond the critical region boundary - probability of rejecting the NULL goes up ---> power goes up |
|
Sample size and Power
|
- directly related
- sample size effect standard error which in turn can increase or decrease the point score of the alpha boundary - when the sample size is small the standard error is large the point score of the alpha boundary is large - this will give the alpha boundary a high z score on the treated DSM thus reducing its Power |
|
If you increase the sample size the power will ...
|
- increase
|
|
Alpha Level
|
- lowering the alpha lowers the power
- it moves the alpha boundary from the mean; giving it a higher point score; moving it farther into the treated DSM |
|
Changing a 2 tailed test to a one tail test does what to the power?
|
increases the power by expanding the critical region (moving the alpha boundary to the left and closer to the known mean)
|