Chris Gorgolewski: Multiple Comparisons

It is a puzzling mental exercise to reflect upon the processes behind your own thoughts. Philosophers have been struggling with the problem of consciousness for ages and recently entered into dialog with neuroscientists to look for its neural correlates. The road to understanding how the human brain works is very long and we have only just started this journey. Despite efforts of many great minds, our knowledge of mechanisms underlying human activities such as  love, compassion, problem solving, speech, or even vision is still far from complete. Undeniably we have made some progress: we have learned how to map different brain areas to minimize damage during brain surgeries, treat depression and Parkinson’s disease by directly stimulating a particular part of the brain and develop drugs that alter chemicals in the brain and allow to manage some mental disorders. Despite the limitations of our knowledge one thing is certain: whoever we are, whatever we do, and whatever we fe...

"Plurality is not to be posited without necessity" says one of the major rules of scientific thinking - The Occam's Razor . In other words when we have two explanations of the same evidence the one that is less complicated (or require fewer assumptions) is more likely to be true. Of course it is rarely the case that the models we are considering are explaining the data to the exact same extent, but modern statistical techniques have been developed deal with this (see BIC , AIC and Bayes factor ). In short, those methods combine evidence for each model (such as goodness of fit or likelihood) with complexity of the said model (for example the number of free parameters) in a way that penalizes overly complex solutions.  All of this should be nothing new to anyone dealing with models, data, and theories. There is, however, one additional aspect concerning the complexity of explanations - a social one. We intrinsically like simple stories. It's not only because it is...

Consider a typical fMRI study:  Twenty participants scanned for an hour = 10000 USD. Research Assistant to run participants = 20000 USD. Postdoc to invent the study and write it up = 40000 USD. 70000 USD later science is richer by an eight page paper, peer reviewed and published in an academic journal. The authors might look at the data again some time later, maybe join it with some other of their dataset to improve power. Maybe. Or maybe they will not have time. We may never learn if there was anything more in the data (all 360 million datapoints of it) than what those eight pages described. Most scientists agree that sharing data makes sense and leads to better, more reproducible, transparent, and objective science. Funding agencies (the guys who turn your taxes into academic papers) understand how expensive data collection is and want to squeeze as much as possible out of existing data. But the perspective of an individual scientist is different. Sharing data does no...

I'm impatient by nature. When I'm on the interner, however, I'm like Twitchy on coffee  (now imagine what happens when I'm browsing the web after having a cup of the dark brew). I don't stay on one website longer than five minutes, I struggle to finish watching a youtube video without checking something in the background, I catch myself mindlessly going to websites and asking myself "Why did I opened this tab?". It's bad. It's a constant crave for new information, new stimuli, something surprising, something funny. But it didn't use to be this way. I used to be able to finish reading a book in one sitting (even though it did not happen often). Now I struggle with one academic paper without a break. Something changed in me about how hard it is for me to focus on one task for an extended period of time. The obvious thing to blame was the Internet. And I know I'm not the only one. Why services like rescuetime.com and keepmeout.com are...

A friend of mine once had a crush on a girl. The girl wasn't really interested, but valued him as a friend and was too polite to tell him he does not stand a chance (or maybe she just enjoyed the extra attention - who knows!). So the chase went on reaching pretty pathetic levels. At the same time another girl was basically throwing herself at him. Yet he wasn't paying attention to her and preferred to chase an illusion. It's not a single case. Even if you have not experienced it yourself (on which I congratulate you - I'll try to touch on individual differences later on), you must've heard similar stories from your friends. Actually if you look for most popular dating advice you will learn that the trick is to let go and maintain the magical balance of not caring and being interested (or as John Green would say in his witty way: "dumpees should fight the clingy urge" ). Speaking more general my friend was assessing a potential relationship mostly basing ...

Even though this blog is not going to be only about multiple comparisons (I could not think of another name), I decided to write about an old problem in slightly new way. Multiple Comparisons Whenever we are testing many hypotheses and are trying to figure out which of them are true we stumble upon so called Multiple Comparisons problem. This is especially evident in fields where we do tens of thousands tests (such as neuroimaging or genetics). So what is the big deal? Imagine that you divide the brain into a buch of regions (voxels) and for each of them you will perform some statistical test (checking for example if this part of the brain is involved in perception of kittens). Some of the regions will yield high statistical values (suggesting relation to kittens) and some will not. Lets try to show this with a simple simullation. Let's assume for now that we will test 100 voxels and only 10 of them will be related to kittens. We will model both populations of voxels u...