#---------------------------------------------------------
# File: MIT18_05S22_in-class17-script.txt
# Author: Jeremy Orloff
#
# MIT OpenCourseWare: https://ocw.mit.edu
# 18.05 Introduction to Probability and Statistics
# Spring 2022
# For information about citing these materials or our Terms of Use, visit:
# https://ocw.mit.edu/terms.
#
#---------------------------------------------------------
Class 17 Frequentist stats. NHST
Jerry
Slide 1:
Slide 2 Announcements/Agenda (2 minutes)
Slide 3 Frequentist school (2 minutes)
Slide 4-6 Fork in the road (4-6) (4 minutes)
For slides 5 and 6, don't focus on the numbers
The key point is the missing prior in slide 6
Slide 7 CONCEPT QUESTION: (5 minutes)
What would a frequentist accept
We are going to say roughly a million times: Frequentists don't compute probabilities of hypotheses.
Slide 8 What is a stat (2 minutes)
We did this earlier in an earlier class
Many people are bothered by this definition when they first
hear it, but I think it is a good working definition.
It correpsonds to the physics notion of an observable.
Slide 9 DISCUSSION QUESTIONS (4 minutes)
Start the repetition that Frequentists don't compute prob. of hypoth.
Series of 4 questions: is it a stat?
Just
Slide 10 Cards: (6 minutes)
The goal here is to illustrate Fisher's insight into using 5% as a standard significance level.
I will bring a deck of cards with all red cards but the cellophane wrapping intact
In class, I'll say that's it's a new deck and give it to Jen to open.
I'll explain the game:
Jen will let random people draw a card.
If it's black I'll put a dollar in the kitty. \
If it's red she'll put a dollar in the kitty.
The game ends when one of us quits
By the 3rd table there will be some grumbling. By the 4th people will strongly suspect the game is rigged and by the 5th they're sure
At this point Jen will quit and we can discuss likelihood.
Compute likelihood P(5 red cards | deck is fair) = 1/32 < 0.05
This is one of Fisher's insights and in another context the origin of the use of 0.05 as the threshold for significance
Slide 11 NHST ingredients (2 minutes)
Review of the region
Note that p values are not in the list.
Jen
Slide 12 Choosing rejection regions (2 minutes)
They'll find this pretty abstract
Don't explain it here. We'll do that in the context of the following example.
Slide 13 TABLE QUESTION: significance and rejection regions (work 5 minutes, discussion 5 minutes)
Slide 14 Solution and DISCUSSION
Emphasize: rejection region is a range of values of the test stat
Emphasise it's chosen to be more likely under the alternative hypothesis
Slide 15 Concept Question: (5 minutes)
Which region represents significance
Again we can point out that the rejection region is on the side of H_A
Encourage them to really learn to read pictures like this. The pictures have a lot of explanatory power and we will use them frequently.
They should ask -and keep asking- if anything is unclear.
Slide 16,17 z-tests p-values (4 minutes)
Walk through slide 16 slowly. State what these things are, but don't repeat and don't worry about being fully understood. Let them know the next slide and board questions are designed to reinforce this.
The point of 17 is that it's identical except for the choice of rejection region is guided by the one-sided H_A
Move to next slide:
Slide 18 visualization (5 minutes)
Use the board to explain that the p-value means the probability of seeing the data or data more extreme --do this for right-sided H_A since it's easier to work with. Extreme is measured relative to H_A.
Our take is that the p-value is just a numerical gadget to tell you if the test statistic is in the rejection region, i.e. 2.4 is in the rejection region is the same as the p-value (blue striped are) is < alpha (orange area)
Define critical values z_{0.05} (right-sided version of quantiles) etc
Slide 19-21 BOARD question z-test (Work 12, discussion 6)
make sure they draw and label a picture
Our goal is for them to understand the role of all the pieces
Slides 20,21 Solution
DISCUSSION Use the solution slides
Note we are using the more precise z_0.25 = 1.96 for standard normal
We stand a chance of getting here depending on the time we can just finish the class with them working on this.
Slide 22 BOARD question NHST for 2 coins
DISCUSSION