Welcome to STA 240!
Probability for Statistical Inference
Teaching Team
| Mug | Name | Role | Office Hours |
|---|---|---|---|
 |
Hu, Yuang | TA | Mon 7:30 PM - 9:30 PM |
 |
Liu, Aurora | Head TA | WeTh 4:30 pm - 5:30 pm |
 |
Ma, Liane | TA | Sun 10:00 am - 12:00 pm |
 |
Zito, John | Instructor | Tue 3:00 pm - 6:00 pm |
Statistical Science Majors Union
Welcome First-Years Event!
- 9/4 at 7 PM
- East Duke 108
GBM #1 & Research Panel
- 9/18 7 PM
- Old Chem 116
listserv
DukeGroups
Example: the birthday problem
Are all birthday’s in fact equally likely?
We will assume this when we do the calculation. But actually…
Example: the Monty Hall problem
Let’s play: https://montyhall.io/
Very counterintuitive
Most people start out thinking that the two doors are equally likely to contain the prize, so switching doesn’t matter. In fact, you have a 2/3 chance of winning if you switch.
You people
Example: which is “truly” random?
- 27 of you said Left is pure random;
- 8 of you said Right is pure random.
Example: which is “truly” random?
- Left: uniform random scatter;
- Right: points manipulated to “repel” each other (no clumps).
This is why Spotify shuffle stinks
- It used to be a pure random shuffle;
- Then people complained that it didn’t feel random to them:
- streaks of the same artist in a row;
- clumps of the same genre, etc;
- Now they do lord-only-knows-what.
Example: the wisdom of crowds
At a 1906 country fair in Plymouth, 800 people participated in a contest to estimate the weight of an ox. Francis Galton observed that the median guess, 1207 lbs, was accurate within 1% of the true weight of 1198 lbs.
Lesson
The aggregation of many imperfect estimates/guesses is often better than a needle-in-haystack search for the “best” individual guess.
Isn’t that obvious?
It took humans a long time to realize this. The first recorded uses of an “average” were during Isaac Newton’s lifetime (see Stigler’s Seven Pillars of Statistical Wisdom).
Guess the weight
Guess the price
Guess the age
The modern-day wisdom of crowds?
(source: CNN)
Sally Clark (1964 - 2007)
- English solicitor;
- 1996: first son died in bed at 11 weeks;
- 1998: second son died in bed at 8 weeks;
- Clark arrested on suspicion of murder;
- Defense claimed it was SIDS (sudden infant death syndrome).
Sir Roy Meadow (1933 - )
- Pediatrician specializing in “Munchausen syndrome by proxy;”
- Gave erroneous “expert” testimony that there is a “1 in 73,000,000” chance of two SIDS deaths in the same family;
- first, that’s not true;
- second, so what?
- 1999: Clark convicted.
A sad story
- Stats community loudly panned Meadow’s testimony;
- 2003: conviction overturned on appeal after revelation that exculpatory evidence was suppressed;
- 2007: Clark drinks herself to death.
Reason 1: It’s necessary.
Human intuition and “common sense” about probability and statistics are often just flat out wrong, in silly and dangerous ways. Mere mortals require the scaffolding of mathematics to discipline our thinking.
New title: The Unreasonable Effectiveness of Probability in…Everything?
Example: the mathematics of insurance
- When an insurance company sells you a policy, they are making a bet that you won’t need it;
- If all policyholders make a claim, they’re ruined;
- Given so much uncertainty, who should they sell to, and at what price, to ensure they don’t go broke?
- Ask an actuary! Licensed professionals with expertise in probability, statistics, and risk management.
Example: the mathematics of insurance
- Nurse practitioner
- IT Manager
- Physician Assistant
- Financial Manager
- Software Developer
- Information Security Analyst
- Health Services Manager
- Data Scientist
- Speech-Language Pathologist
- Actuary
- Future prospects (30%)
- Wage potential (25%)
- Employment (20%)
- Job safety/stability (15%)
- Work-life balance (10%)
Probably isn’t boring these days
Plenty of complex, challenging, high-impact issues:
Example: whence evolution?
Hella simplified:
- Mutation: random errors during DNA replication and repair;
- Genetic drift: chance events shift allele frequencies in the population;
- Natural selection: randomness generates options; selection chooses among them, producing evolution.
Example: quantum mechanics
key words: Schrödinger’s cat, uncertainty principle, etc
- Probability is fundamentally braided into the fabric of reality;
- Particles at the (sub)atomic level do not have a definite position and momentum until measured;
- There are only probabilities associated with the possible outcomes;
- These ideas are hard, but some of the math will be accessible to us.
Reason 2: It’s useful.
No area of science or technology can be properly understood without knowing something about probability. Period.
Two old hags
Bertrand Russell, Mysticism and Logic and Other Essays (1917)
“Mathematics, rightly viewed, possesses not only truth, but supreme beauty cold and austere, like that of sculpture, without appeal to any part of our weaker nature, without the gorgeous trappings of painting or music, yet sublimely pure, and capable of a stern perfection such as only the greatest art can show.”
Albert Einstein, “The Late Emmy Noether” (1935)
“Pure mathematics is, in its way, the poetry of logical ideas.”
This is a theorem we will prove:
Reason 3: It’s beautiful.
Ostensibly random behavior can nevertheless be quite patternful, in ways that we can actually penetrate with elegant mathematics.
An archetypal probability problem
Given a fair coin, how many flips will it take on average until you observe the first head?
An archetypal statistics problem
Given 28 flips from a mysterious coin, can you tell if it is fair?
Probability and statistics
| Probability | Forward problem Deductive Reasons from rules to consequences |
| Statistics | Inverse problem Inductive Observe consequences, and infer rules |
Forward problem versus inverse problem
Forward: read the rulebook, and then play a game of chess;
Inverse: watch a chess match, and based on the players’ behavior, try to guess the rules.