The mechanics of calculating p values with mne.stats.spatio_temporal_cluster_test

đź’¬ Support & Discussions
1

Hi everyone,

I recently migrated my spatiotemporal clustering tests of MEG data to MNE from Eelbrain since the latter does not work on my university’s HPC cluster. To make sure that my code was working properly, I reran some old analyses previously done in Eelbrain with the expectation of replicating the results, but this was only partially the case. While I did observe the exact same clusters (in terms of size and duration), the p values differed drastically between the two analyses. For example, the p value of one cluster changed from p < 0.05 in Eelbrain to being 0.16 in MNE.

Since the clusters are the same across the two analyses, it must have to do with how the clustering statistic is computed, and I’d like to understand the mechanics of what each package does. As such, I was wondering if you could walk me through the steps in computing the clustering statistic in MNE? Any insights on the difference between the two packages are naturally appreciated as well, but I’m also reaching out to Christian Brodbeck to ask for how the clustering statistic is computed in Eelbrain.

I use the MNE function mne.stats.spatio_temporal_cluster_test and the Eelbrain function Eelbrain.testnd.ANOVA.

Thank you in advance!

Simone

2

Hello Simone,

thank you for your question, I didn’t compare the code line by line but one potential reason might be the number of permutations. The MNE default number of permutations is 1024, in eelbrain, it is 10.000. Could you check if you get the same results if you set the same number of permutations in both tests to be the same?

Best,

Carina

3

Hi Carina,

Thanks for your quick response! I’m using 10,000 as the number of permutations for both tests already so that shouldn’t be the reason.

Best,

Simone

1 Like
4

Just as an update: I haven’t had time yet to dig deeper into the code, could you update if you hear something from the eelbrain community?

Best,

Carina