Chapter 1. P-Value of a Test of Significance

Introduction

Statistical Applets

To set up the test, fill in the boxes: What null hypothesis H₀ about the mean μ do you want to test? Which alternative hypothesis H_a do you have in mind, and what level of significance α do you require? What value of the standard deviation σ is known to be true? How many observations n will you have (250 or fewer)?

If you already have a sample mean, enter this value and click UPDATE to display the sample mean on the graph and calculate the P-value. Or you can specify the true population mean μ and use the GENERATE SAMPLE button to create a random sample from the population, display the observations and sample mean (note that some of the points in the sample may be too far from μ to appear in the display), and calculate the P-value.

Click the "Quiz Me" button to complete the activity.

This applet illustrates the P-value of a test of significance. Here we're testing a hypothesis about the mean of a normal distribution whose standard deviation we know, but the concepts are essentially the same for any other type of significance test.

The normal curve shows the sampling distribution of the sample mean when your null hypothesis is true. The blue arrow shows what kinds of values of count as evidence against H₀ in favor of your alternative H_a. Try changing H_a to see how the arrow changes. Once you have a value of from data, the graph will show you the P-value for this : it is the probability—calculated taking H₀ to be true—of getting a value at least that far away from H₀ in the direction of the arrow.

Question 1.1

Suppose you're planning to collect a set of data in an experiment where the null hypothesis states that the population mean will be 15. You plan to collect 30 observations, and you expect the population standard deviation to be 6.5.

Use the applet to calculate the P-value for your final test of significance, considering the possibilities that your sample mean comes out to 12, 13, or 14, and considering the two possible alternative hypotheses µ < 15 and µ ≠ 15. Fill the P-values into the table below. The P-value for one cell in the table—where the sample mean is 12 and H_a is µ < 15—is filled in for you.

	H_a:µ < 15	H_a:µ ≠ 15
12	0.0057	Q6tJlBQS88vvH3a3
13	iQt1UV70zqqG1Zlc	w9QXOByLQcy4Opwz
14	h+b8FBy9QMnvvL9K	VTXpwY5Rh9rfXZzO

Table

Try again.

Incorrect. See above for the correct answers.

Great job.

Question 1.2

Considering these examples, you can conclude that:

Your experiment is more likely to result in a statistically significant result the jFJecs3ptdJkIhQfxDDNog== the value of your sample mean turns out to be.
Your experiment is more likely to result in a statistically significant result if you choose to test the alternative hypothesis that µ is OEJBTFZRqEZozhS4b8wtF51u7c5vTmYNILjVJg== 15.

Incorrect. See above for the correct answers.

Great job.

Question 1.3

sP9QGiqjAnnhzCYMxjFpALbgoDbxRi1bbzIcpfPr4HdamCScXMPB3MVM8j7C3lAx8orBoaAifJRpHKv+y/gl6RW60Jp6AQJxMxJGgO8hbn0T/4VBkOs028E9SN9DeMlgyFn9q7ey/Smc2fywcLHPDIeWqhukUqO+7PK2Z2YKsg3Y5iFcb78vJsCdUlXY/EmCB2CJrs+aGPTyknrEqeLQbn5+MmzT9y65KZQveaPzfZTxYyQqWBy7l30S82dakyvQE4g7fCHx0hvRGyC7ZiEVGo474gS6GU9wN98xihLr/cw=

Increasing the sample size, which an experimenter often has some control over, always increases the likelihood of a statistically significant result (assuming, of course, that the experimental manipulation actually has an effect!). This is because by taking a larger sample, your sample mean is more likely to be closer to the true population. Practically speaking, in terms of the numbers that go into tests of significance, because a larger sample size means a smaller standard error, and therefore a larger test statistic and a smaller P-value.