Chris Gorgolewski: Multiple Comparisons

I'm impatient by nature. When I'm on the interner, however, I'm like Twitchy on coffee  (now imagine what happens when I'm browsing the web after having a cup of the dark brew). I don't stay on one website longer than five minutes, I struggle to finish watching a youtube video without checking something in the background, I catch myself mindlessly going to websites and asking myself "Why did I opened this tab?". It's bad. It's a constant crave for new information, new stimuli, something surprising, something funny. But it didn't use to be this way. I used to be able to finish reading a book in one sitting (even though it did not happen often). Now I struggle with one academic paper without a break. Something changed in me about how hard it is for me to focus on one task for an extended period of time. The obvious thing to blame was the Internet. And I know I'm not the only one. Why services like rescuetime.com and keepmeout.com are...

A friend of mine once had a crush on a girl. The girl wasn't really interested, but valued him as a friend and was too polite to tell him he does not stand a chance (or maybe she just enjoyed the extra attention - who knows!). So the chase went on reaching pretty pathetic levels. At the same time another girl was basically throwing herself at him. Yet he wasn't paying attention to her and preferred to chase an illusion. It's not a single case. Even if you have not experienced it yourself (on which I congratulate you - I'll try to touch on individual differences later on), you must've heard similar stories from your friends. Actually if you look for most popular dating advice you will learn that the trick is to let go and maintain the magical balance of not caring and being interested (or as John Green would say in his witty way: "dumpees should fight the clingy urge" ). Speaking more general my friend was assessing a potential relationship mostly basing ...

Even though this blog is not going to be only about multiple comparisons (I could not think of another name), I decided to write about an old problem in slightly new way. Multiple Comparisons Whenever we are testing many hypotheses and are trying to figure out which of them are true we stumble upon so called Multiple Comparisons problem. This is especially evident in fields where we do tens of thousands tests (such as neuroimaging or genetics). So what is the big deal? Imagine that you divide the brain into a buch of regions (voxels) and for each of them you will perform some statistical test (checking for example if this part of the brain is involved in perception of kittens). Some of the regions will yield high statistical values (suggesting relation to kittens) and some will not. Lets try to show this with a simple simullation. Let's assume for now that we will test 100 voxels and only 10 of them will be related to kittens. We will model both populations of voxels u...