tag:blogger.com,1999:blog-21605329.post6687511637188330577..comments2018-02-20T17:21:30.312-08:00Comments on The Neurocritic: I'm a People Person!The Neurocritichttp://www.blogger.com/profile/08010555869208208621noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-21605329.post-12326425008272667692009-06-01T01:13:16.959-07:002009-06-01T01:13:16.959-07:00Anon: I'm not suspicious of all scatterplots, but ...Anon: I'm not suspicious of all scatterplots, but Vul et al correctly point out that a scatterplot of this nature is essentially meaningless if the voxels across which the plot is calculated (which is often not stated precisely) are chosen in certain ways.<br /><br />It's not true that "If you correct for multiple comparisons you are fine". If you correct for multiple comparisons, you only have a 5% chance of getting a result out of pure noise.<br /><br />But, and this is Vul's point, you can create an amazingly impressive result (correlation coefficent of .8 or more, and a very pretty scatterplot) out of a very weak one, by doing a non-independent analysis.Neuroskeptichttps://www.blogger.com/profile/06647064768789308157noreply@blogger.comtag:blogger.com,1999:blog-21605329.post-80700469856806757762009-05-26T16:52:44.352-07:002009-05-26T16:52:44.352-07:00It is interesting that the Vul & al. paper has...It is interesting that the Vul & al. paper has simply generated suspicion for scatterplots of this kind without generating real understanding of the problem (in part, because neither of them are statisticians and because they hyped the problem for their Blair Witch Project style pre-release campaign). <br />If you correct for multiple comparisons you are fine, for correlations and for simple effects (you'll get your 5% of false positives, but that is life). The question is how to correct for multiple comparisons. If you use Bonferroni, you are OK by definition, it's a worst case scenario. A less extreme scenario models the spatial correlation in the data. Monte Carlo methods work well in this case. Also, if your ROIs include voxels that are all above the correlation threshold, then presenting a scatterplot of the average within the ROI is usually not problematic. <br />All statistics are surrogates for replication. Our field desperately needs a Journal of Null Effects.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-21605329.post-74986655737941244272009-05-25T16:04:46.467-07:002009-05-25T16:04:46.467-07:00Anonymous - Ha, ha! It turns out that dog experts ...Anonymous - Ha, ha! It turns out that dog experts have been studied in the context of the debate on human face processing abilities as innate vs. acquired expertise (<A HREF="http://www.ncbi.nlm.nih.gov/pubmed/16616910" REL="nofollow">Robbins & McKone, 2007</A>).<br /><br />Neuroskeptic - Lebreton et al. didn't use ROIs, they did a whole brain analysis to create probabilistic maps of gray matter, white matter and CSF for each subject. Then they used non-parametric statistics (permutation testing) to test for significant correlations:<br /><br />"<I>Associations between RD and adult brain structure, adjusted for total gray matter volume, novelty seeking and harm avoidance were tested by multiple linear regression onto the values of GMD at each voxel using permutation-based methods implemented in Cambridge Brain Analysis (CamBA) software...</I>"<br /><br />According to the Vul and Kanwisher chapter (<A HREF="http://www.edvul.com/pdf/vk.nonindependence.pdf" REL="nofollow">PDF</A>):<br /><br />"<I>Permutation tests are one particularly effective method when analyses are particularly complicated. Researchers can randomly permute the condition labels for their data and undertake the same analysis. If this is done enough times, it is possible to empirically estimate the probability of the outcome observed with the true data labels. Unlike simpler (and faster) white-noise simulations, this permutation analysis includes the non-gaussian structure of the BOLD signal noise, and is thus more accurate.</I>"The Neurocritichttps://www.blogger.com/profile/08010555869208208621noreply@blogger.comtag:blogger.com,1999:blog-21605329.post-73445107157156379792009-05-25T08:28:19.384-07:002009-05-25T08:28:19.384-07:00One wonders if the findings also hold for Dog Pers...One wonders if the findings also hold for Dog Persons.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-21605329.post-73087995710664131722009-05-25T00:33:34.938-07:002009-05-25T00:33:34.938-07:00Are those scatterplots products of the Vul et al e...Are those scatterplots products of the Vul et al error? I guess not if they're based on anatomical ROIs...Neuroskeptichttps://www.blogger.com/profile/06647064768789308157noreply@blogger.com