Chapter 1. P-Value of a Test of Significance


Statistical Applets

To set up the test, fill in the boxes: What null hypothesis H0 about the mean μ do you want to test? Which alternative hypothesis Ha do you have in mind, and what level of significance α do you require? What value of the standard deviation σ is known to be true? How many observations n will you have (250 or fewer)?

If you already have a sample mean, enter this value and click UPDATE to display the sample mean on the graph and calculate the P-value. Or you can specify the true population mean μ and use the GENERATE SAMPLE button to create a random sample from the population, display the observations and sample mean (note that some of the points in the sample may be too far from μ to appear in the display), and calculate the P-value.

Click the "Quiz Me" button to complete the activity.

This applet illustrates the P-value of a test of significance. Here we're testing a hypothesis about the mean of a normal distribution whose standard deviation we know, but the concepts are essentially the same for any other type of significance test.

The normal curve shows the sampling distribution of the sample mean when your null hypothesis is true. The blue arrow shows what kinds of values of count as evidence against H0 in favor of your alternative Ha. Try changing Ha to see how the arrow changes. Once you have a value of from data, the graph will show you the P-value for this : it is the probability—calculated taking H0 to be true—of getting a value at least that far away from H0 in the direction of the arrow.

Question 1.1

Suppose you're planning to collect a set of data in an experiment where the null hypothesis states that the population mean will be 15. You plan to collect 30 observations, and you expect the population standard deviation to be 6.5.

Use the applet to calculate the P-value for your final test of significance, considering the possibilities that your sample mean comes out to 12, 13, or 14, and considering the two possible alternative hypotheses µ < 15 and µ ≠ 15. Fill the P-values into the table below. The P-value for one cell in the table—where the sample mean is 12 and Ha is µ < 15—is filled in for you.

Ha: µ < 15 Ha: µ ≠ 15
12 0.0057 Q6tJlBQS88vvH3a3
13 iQt1UV70zqqG1Zlc w9QXOByLQcy4Opwz
14 h+b8FBy9QMnvvL9K VTXpwY5Rh9rfXZzO
Try again.
Incorrect. See above for the correct answers.
Great job.

Question 1.2

Considering these examples, you can conclude that:

  • Your experiment is more likely to result in a statistically significant result the jFJecs3ptdJkIhQfxDDNog== the value of your sample mean turns out to be.
  • Your experiment is more likely to result in a statistically significant result if you choose to test the alternative hypothesis that µ is OEJBTFZRqEZozhS4b8wtF51u7c5vTmYNILjVJg== 15.
Incorrect. See above for the correct answers.
Great job.

Question 1.3

Increasing the sample size, which an experimenter often has some control over, always increases the likelihood of a statistically significant result (assuming, of course, that the experimental manipulation actually has an effect!). This is because by taking a larger sample, your sample mean is more likely to be closer to the true population. Practically speaking, in terms of the numbers that go into tests of significance, because a larger sample size means a smaller standard error, and therefore a larger test statistic and a smaller P-value.