How to set up a cluster permutation test between two independent sample groups?

Support & Discussions
1

Hello all,

I want to compare two independent groups (Group X1 has N=12 participants and Group X2 has N=13 participants)
The data I want to compare is decoding accuracy over time. Therefore, for each group I have an array of N participants X Time-points.

shape(X1)
Out[258]: (12, 201)

shape(X2)
Out[259]: (13, 201)

I want to run a cluster permutation test to compare the independent groups, but I am unsure how to set-up the treshold and statsfun parameters for the ne.stats.permutation_cluster_test. At the moment, I have the following call:

X=[X1, X2]
F, clusters, cluster_pv, H0 = mne.stats.permutation_cluster_test(X, 
                                                                 threshold=None, 
                                                                 n_permutations=1024, 
                                                                 tail=0, 
                                                                 stat_fun=None, 
                                                                 adjacency=None, 
                                                                 n_jobs=1, 
                                                                 seed=None, 
                                                                 verbose=None)

My specific questions are:

  • Does leaving the threshold = None selects a threshold for p<.05 for independent sample comparisons
  • Does the stat_fun need to specify that the test independent samples?

Thanks!

2

From my understanding you need the evoked objects from each participant rather than the grand average object, as you need both the difference between groups at a given time point but also the variation within each group - just like you would when performing a standard t-test or similar. So you will need to supply the function with a 3d array for each group, each containing the evoked objects that went into the grand average.
If I remember right you can use np.dstack(evoked_data_list) to stack you data

3

Thanks for the question @anapesq! One note is that I would recommend seeding your functional call with any number you like for reproducibility. To answer one question, yes the default is an F statistic corresponding to p < 0.05 mne.stats.permutation_cluster_test — MNE 1.0.0 documentation. The independent samples are assumed because otherwise you would use mne.stats.permutation_cluster_1samp_test — MNE 1.0.0 documentation and subtract one group from another. I hope that helps!

4

@Erik, good thought, people usually do use permutation cluster testing on 2D+ data but you can actually only use the p dimension mne.stats.permutation_cluster_test — MNE 1.0.0 documentation as in here to only permute in time. The examples (e.g. Non-parametric between conditions cluster statistic on single trial power — MNE 1.0.0 documentation) tend to do 2D or even 3D clusters but there is one that only does time Permutation F-test on sensor data with 1D cluster level — MNE 1.0.0 documentation. Here we are looking at epochs as the independent condition (you might not have the same number in each condition so you can’t do the one sample cluster permutation) and time as the dimension to find clusters in. This is similar to @anapesq’s example but she is using subjects instead of epochs to find a group-level difference (whereas epochs would find a condition-level difference like auditory vs visual).

5

Thank you for this input and links to documentation!