In the first "Are You A Betting Man (or Woman)?" thread:

https://www.s2ki.com/forums/showthread.php?...&threadid=44492

I mentioned that I would post what was said on the radio. Here it is:

I was listening to KPCC, an NPR station in Pasadena, CA, when one of the hosts, Kitty, was talking about the standard exam that all California high school students must pass to graduate. She figured that, if high school students should be able to pass this test, then the employees at KPCC should be able to pass it as well. So she asked this question of the other host, Steve:

Kitty: Mary tosses a coin five times and it comes up heads all five times. What is the theoretical probability of getting a tail on the sixth toss?

Steve: 50%.

Kitty: Congratulations! You passed high school!

As soon as I got to work I sent an e-mail to KPCC to explain to them why Steve's answer was wrong, and why that was a poor question for a high school graduation exam. I'll get to that in a bit.

You'll note that the question I posed in this thread was a bit different than Kitty's. This was done to make it simpler for a poll, and to ensure that there is a correct answer.

Four people (S2R, Psicho54, Lips2000, tokyo_james) noted that the tosses are independent (though S2R said "mutually exclusive", which is completely different, I think he meant "independent"); the implication is that the probability of getting a tail does not change from toss to toss, irrespective of the results of previous trials. This is correct.

Four people (WestSideBilly, mingster, The Raptor, krhorrocks) considered the possibility that the coin might not be fair; i.e., that the probability of getting a tail might not be 50%. This is also correct. However, nobody pursued that reasoning to its conclusion.

Two people (WestSideBilly and The Raptor) said that the streak of five heads in a row in meaningless. That is incorrect. One person (Lips2000) said that the trend had to mean something, but his conclusion was incorrect.

The streak of five heads in a row is statistically meaningful. It is strong evidence that the coin is biased toward heads. Think of it this way: if the probability of a head is 100% (e.g., a double-headed coin), you'll always get five heads in a row, if the probability of a head is 50% (a fair coin), you'll get five heads in a row only 3% of the time (1/32).

Here's a proper analysis:

The coin may be fair. In that case, betting on heads is as good as (but no better than) betting on tails.

The coin may not be fair. In that case there is strong evidence to suggest that it is biased toward heads, not tails. Betting on heads is better than betting on tails.

In the first case, you tie in the long run by betting on heads. In the second case, you win in the long run by betting on heads.

Conclusion: Bet on heads, definitely.

What bothered me about having a question like this on a high school graduation exam with the answer they gave is that it encourages students to hold on to their prejudices (coins are fair) even in the face of strong evidence that their prejudices are wrong. (For those of you who disagree with this statement, let me say that I'm certain that the choice of five heads in a row by the question's author was arbitrary. He could have chosen three heads in a row, or four, or six, or seven, and still have given the same answer. If Mary got 10 heads in a row would you start to think that the coin is not fair? How about 100? 1,000? 1,000,000?) He also threw that word "theoretical" into the question, without any explanation as to which theory he had in mind. (I suspect that he didn't have any specific theory in mind.)

The second "Are You A Betting Man (or Woman)?" thread:

https://www.s2ki.com/forums/showthread.php?...&threadid=44539

was an attempt to pose the same theoretical question in a completely different context.

Two people (ltweintz, ICEMAN666) suggested that the scenarios aren't comparable because of psychological considerations. As I didn't intend for psychology to enter the analysis, I later posted a reply trying to eliminate it. Without psychological factors the two are equivalent.

What was intriguing was how readily people were willing to acknowledge the existence of bias (toward Red) in the second scenario, but were not willing to acknowledge the existence of bias (toward heads) in the first. (Note the stark differences in the poll results!) The second thread was my attempt to gently nudge people to consider that the coin in the first thread might not be fair.

So what's my point?

1. The California high school graduation exam is (at least in part) poorly written.
2. When you see evidence that your prejudices may be wrong, be willing to change them.
3. You can't easily do #2 unless you make sure you know what your prejudices are. (How many never realized that they were assuming a fair coin?)

 

i think i'll send my kids to prep schools instead...

 

Thanks for the explanation. Two points:

First, as the question was posed on the radio, Steve's answer was correct. Any number other than 100% or 50% would be a complete guess, and even a biased coin is not going to come up heads (in this case) 100% of the time. Unless both sides are "heads" that is, in which case something is amiss The way you posed the question is slightly different, as you asked "which would you pick?".

Second, as for my dismissing the streak of 5, it was based on years of statistics in which you have to assume a coin is fair in order to gain any meaningful insight into what's occuring. So, yes, I have a prejudice in this sense. I will note that back in my role playing days, I have seen a (fair) 6 sided die come up "6" five times in a row. The probability of this is .01% - not very likely but it happens. So going on a streak of 5 coin tosses (a 3.1% probability) is a leap of faith. A streak of 10 or 100 heads would probably yield a slightly different result in the poll, as the odds that the coin is legit start waning.

If I was forced to bet on a flip, you're correct, heads is the only way to go. However, statistically it should not matter, so the probability is always 50%.

 

[QUOTE]Originally posted by WestSideBilly
[B]Thanks for the explanation.

 

Alright, damnit, I'm going back to high school. I hope the desks are more comfortable.

 

[QUOTE]Originally posted by magician
[B]... (Lips2000) said that the trend had to mean something, but his conclusion was incorrect.

The streak of five heads in a row is statistically meaningful.

 

Thanks for the complete explanation - I accept your conclusions (on the heads/tails issue), now that I know how you went about it. No treading was done, as I have a lowly Bachelor's in mechanical engineering; one formal statistics class, plus statistical analysis in a few other classes (not based on luck - based on samples in the hundreds/thosands, science and metallurgy); and a minor fascination with numbers and manipulating them. Your background has mine beat

Getting back to your point(s), do you really expect a high school student to have that sort of familiarity with statistics (or math in general) to answer 86% (6/7) liklihood for a heads? All education is a building block for further education - your Master's education clearly presenting you with more mathematical education than my Bachelor's, as evidence of this. I could nitpick nearly every physics question in a high school exam if I wanted to, and along with a few friends (or s2ki members for that matter) probably find a flaw in every question on a high school exam. So it may be poorly written in the eyes of a group of college graduates, but in truth is a fair exam for the level of education its takers should possess.

By the way - do you know of a good book (or web site) on Bayes' statistical logic? Sounds interesting.

 

[QUOTE]Originally posted by WestSideBilly
[B]Thanks for the complete explanation - I accept your conclusions (on the heads/tails issue), now that I know how you went about it.

 

While I understand your point, I don't think it applies in this case. In my interpretation, asking the "theoretical probability" of a coin flipping problem requires a few assumptions, one of which is that the coin is "fair".

 

As I said, using the word "theoretical" in the question is, at best, a poor idea (at worst, some might say it's malicious), because there is no reference to the specific theory the question's author intended.

Is man theoretically descended from apes? Yes, if you choose the theory of evolution. No, if you choose the theory of creation.

(Please don't post an argument that one of these theories is intrinsically preferable to the other. I chose this example because it provides an easy-to-pose question which everone here will understand. There are perhaps better opposing theories to contrast, but the questions would be too obscure to be useful here.)

The definition of a "fair" coin is that the probability of getting a tail is 50%. If you concede that this assumption is required (your word, not mine), then the question posed on the exam amounts to nothing more than "What is your assumption?" In that case, if the student answers honestly, his answer is correct no matter what it is. Once again, we're back to my point that the question is a poor one.

Let's try another example:

A company manufactures bows and arrows. To the best of its manufacturing ability, the bows are identical. They have a testing machine which holds the bow, draws back the string, loads the arrow, shoots the arrow, and measures the flight distance. Each bow is tested five times. When bow #1 is tested, all five of its flight distances are longer than each of the five flight distances for bow #2. If both bows were tested a sixth time, what would be the theoretical probability that bow #2's flight distance would be longer than bow #1's?

Is this also a situation where you would assume that the probability is 50%? I doubt it. (What I mean is, you shouldn't.) There is strong evidence that bow #1 shoots farther. And the situation is identical to the coin tossing. No subjectivity, no psychology, no human intervention to bias the results. (Wait a minute here! In the coin tossing there was human intervention, because Mary tossed the coin. Maybe the bias exists in Mary's thumb. (OK, we'll ignore that possibility for sake of discussion.)) Two completely mechanical systems designed to produce identical results, yet both show identical evidence of bias.

In coin tossing, is an assumption of fairness better than an assumption of bias? Probably. Why? The corollary to Bayes' Theorem which I mentioned earlier. "Aha!" you say. "Now we have a theorem which can be used as the basis for the theory presumed by the author's use of 'theoretical' which justifies our assumption!" Nope. Because that conclusion applies only in the absence of data, and once you start getting data the theory is quite explicit about how you change your estimate (read "assumption") of the probability.

Please start to recognize the need to question the assumption of fairness.

Man, I'm getting long-winded here! My brain is starting to hurt. I need a break; I'm going to the gym, to ponder the theory of how moving heavy objects farther away from the floor results in my t-shirt getting soaking wet.