🐟 How a dead salmon taught scientists a lesson in biostatistics
The salmon was placed in an MRI machine and shown photographs of people in emotional situations. When the data was analyzed, statistically significant changes in the signal were detected in the fish’s brain area.
One detail: at the time of the study, the salmon was dead.
🔬 The story began in the spring of 2005. Neurobiologist Craig Bennett bought a whole Atlantic salmon at a supermarket and brought it to the research center at Dartmouth College. The scientists were testing MRI protocols using unusual objects for scanning. The fish was followed by a pumpkin and a chicken carcass.
The salmon was given a task involving emotion recognition: it was shown photographs of people in social situations, and, as the authors described, it was asked to determine what the depicted characters were feeling.
The article even noted that additional measures to prevent the subject from moving turned out to be practically unnecessary: the salmon maintained exceptional stillness.🧠 When the data were processed without corrections for multiple comparisons, “activation” regions were identified. For individual image elements, a seemingly strict threshold was used — p < 0.001.
❓How is this possible?
When analyzing fMRI data, the image is divided into many small volumetric elements — voxels. In each of them, you can separately check whether the signal differs during the presentation of photos and during periods of rest. This results in thousands of statistical tests, and along with them, many opportunities for random findings.
📊 Let’s consider a simpler example: a researcher conducts 100 tests, even though there are no real differences anywhere. With a significance level of 0.05, we can expect, on average, about five false‑positive results — solely due to chance.
Therefore, the significance threshold for an individual test does not yet describe the risk of an error in the entire analysis.
When the authors applied multiple comparison control methods — FDR and FWER — there were no longer any statistically significant areas left for the salmon.
The results were presented at a conference in 2009. In 2012, the work received the Ig Nobel Prize in Neuroscience.
This story is also useful for research: the more factors, outcomes, and subgroups we analyze, the more opportunities we have to detect a random “pattern”.
❓ What are multiple comparison control methods — FDR and FWER? We’ll look at them in the next post.
The salmon was placed in an MRI machine and shown photographs of people in emotional situations. When the data was analyzed, statistically significant changes in the signal were detected in the fish’s brain area.
One detail: at the time of the study, the salmon was dead.
🔬 The story began in the spring of 2005. Neurobiologist Craig Bennett bought a whole Atlantic salmon at a supermarket and brought it to the research center at Dartmouth College. The scientists were testing MRI protocols using unusual objects for scanning. The fish was followed by a pumpkin and a chicken carcass.
The salmon was given a task involving emotion recognition: it was shown photographs of people in social situations, and, as the authors described, it was asked to determine what the depicted characters were feeling.
The article even noted that additional measures to prevent the subject from moving turned out to be practically unnecessary: the salmon maintained exceptional stillness.🧠 When the data were processed without corrections for multiple comparisons, “activation” regions were identified. For individual image elements, a seemingly strict threshold was used — p < 0.001.
❓How is this possible?
When analyzing fMRI data, the image is divided into many small volumetric elements — voxels. In each of them, you can separately check whether the signal differs during the presentation of photos and during periods of rest. This results in thousands of statistical tests, and along with them, many opportunities for random findings.
📊 Let’s consider a simpler example: a researcher conducts 100 tests, even though there are no real differences anywhere. With a significance level of 0.05, we can expect, on average, about five false‑positive results — solely due to chance.
Therefore, the significance threshold for an individual test does not yet describe the risk of an error in the entire analysis.
When the authors applied multiple comparison control methods — FDR and FWER — there were no longer any statistically significant areas left for the salmon.
The results were presented at a conference in 2009. In 2012, the work received the Ig Nobel Prize in Neuroscience.
This story is also useful for research: the more factors, outcomes, and subgroups we analyze, the more opportunities we have to detect a random “pattern”.
❓ What are multiple comparison control methods — FDR and FWER? We’ll look at them in the next post.
Просмотры: 6
