Watching Your Set

Jenny is agoraphobic: she fears open spaces and, as such, spends all her time inside. Her house, luckily, is custom-built for her predicament, so she suffers little; she even has a "sun room" whereby she can remain indoors and yet work on her tan.

One thing Jenny does a lot of is watch television. She's recently discovered the sports channels, in particular golf. She's a big Tiger Woods fan.

Jenny, watching golf on t.v., sees how good all these players are and decides, "Wow - as a species, humans are really good at golf!" She also eventually decides that "par" means "loser", though she rarely sees anyone actually play so badly as to qualify as "par".

Of course, we as normal observers know that Jenny's wrong - not all humans are good at golf. In fact, most of us are quite bad at it or never bother trying to play.

The issue here is called a selection bias. Jenny only knows about golf what she sees on television. As those of us who aren't so restricted know, television generally only shows the best players at tournaments. Therefore, what Jenny sees is limited to the best. Furthermore, there's a second selection bias in that most people don't actually play golf. So, not only is Jenny limited to the set of "best", she's limited to the set of "best of people who actually play". If we let her see the average person playing, she'd understand what "par" is all about (though she'd probably still like Tiger Woods).

Selection bias is a significant problem when we're doing any kind of statistical analysis: if a selection set is biased, it doesn't represent the real spread of possible inputs and, thus, any analysis we do on that set is basically useless.

Another example of television-based selection bias is the well-known problem of "people on t.v. are more beautiful than everyone else", which (theoretically) causes all sorts of unhealthy self-image problems in people when they try to compare themselves to these "perfect" specimens. Television tends to only show attractive individuals, so the "set" against which people compare themselves is biased, and thus the assumptions derived (I'm uglier than average) aren't valid.

Selection bias pops up in all sorts of places, though, not just on television. A simple example is that we, as individuals, generally only remember events that stand out - extremes of either pleasure or discomfort. We don't tend to remember "average" experiences. So, when looking back over our interactions with (for example) waiters, we'll likely focus on the times something got screwed up and use that as our sample size - ignoring all the times when everything went perfectly fine. Thus, we tend to judge the overall experience as highly negative, whereas (taken as a whole) it was probably average.

In the debate over global climate change, we see a lot of selection bias. Many denialists pick specific data points or narrow date ranges and say, "Look, compared to this other date, the temp is lower!" One extremely popular tactic is to point out that temperatures have decreased since 2001 - which happened to be one of the hottest years on record. Obviously, the sample size is biased: when we talk about climate change, we're referring to periods spanning years and temperatures across larger locations, generally in comparison to likewise large spans of data. Comparing point X to point Y means nothing.

A less obvious example is usually referred to as the anthropic principle. This one's a bit more complex, but I'll try to summarize.

Many ID or New-Early or Young-Earth proponents, and especially anti-evolution activists, try to use probability and statistics to justify the idea that there had to have been an intelligent designer creating Earth. "Look," they say, "at evolution. Evolution says that small changes add up over time. But to get from an amoeba to a human, you have to have a really specific set of changes take place in a certain order. Even scientists state that such a chain of specific events is highly improbable. Therefore, there must have been a designer, because it's statistically impossible for us to have evolved naturally!" That's not necessarily the exact argument, but many of the arguments boil down to the notion of the high improbability of evolution resulting in humans.

Which is correct - it *is* highly improbable that the chain of events leading to the evolution of people would occur. Before anti-evolutionists jump up and down, we have to point out one small fact, however: if they hadn't occurred that way, we wouldn't be here to comment.

The anthropic principle states a lot of things, but one aspect of it is this kind of inherent selection bias: as we look back and examine the history of events that led to the state of the universe today, we have to take into account that they *had* to occur that way or we wouldn't have this perspective. Right now we say, "it's amazing that event x1 occurred when it was so improbable!", but if x2 had occurred instead of x1, we'd be saying, "it's amazing that event x2 occurred when it was so improbable!" In a very important way, we are experiencing a selection bias of "success" or "actual result".

Another example: if the dinosaurs hadn't died out, mammals likely wouldn't have taken over, which means humans probably would never have evolved. So, while whatever caused the mass extinction of the dinosaurs was likely a very improbably event, if it *hadn't* happened, we wouldn't be around to point out the fact.

Furthermore, on a universal scale, if the big bang hadn't happened just so, if gravity and electromagnetism and the weak and strong forces weren't balanced just so, if the matter/energy ratio of the universe weren't just so, we likely would never have come into existence to be able to say "gee, it's amazing all these things worked out perfectly for us to be here."

We have an inherent selection bias when looking at the universe, because if the universe didn't meet all the myriad requirements for us to exist, we wouldn't be around to notice.

That doesn't mean it had to happen or was made to happen; simply that we are the end result of all that happened. In some parallel universe, perhaps there is a race of saurians looking into space through their telescopes and thinking, "Whew! It's amazingly lucky that that comet missed us! It must have been the claw of God!"