Walkthrough for paired permutation cluster test needed

Support & Discussions
1

This issue has been brought up different times as people on this forum try to do permutation tests. My issue in particular is that I find the API doc and tutorial to be extremely confusing, to the extent that I have no idea what data format is required to run this test in the right way…

Let’s say I have a dataset with 35 participants, 4 conditions and 1800 timepoints with single channel (pupil_right in my case), how should I structure my data so that it can be accepted into permutation_cluster_test()? Do I have 1) a list of 35 arrays, each array has 4 columns and 1800 rows, or 2) a list of 4 arrays, 35 columns and 1800 rows, or 3) have each of my arrays transposed?

This tutorial Mass-univariate twoway repeated measures ANOVA on single trial power — MNE 1.10.1 documentation gave an example that puts number of replications/subjects as the first dimension, akin to 1), but the API doc mne.stats.permutation_cluster_test — MNE 1.10.1 documentation tells me I should take after 2)? Which one should I follow to get the right result?

1 Like
2

This is exactly why we had a Google Summer of Code project to overhaul the statistics API. We agree with you that it is confusing in its current state. We are working on it in this PR: new cluster test API by CarinaFo · Pull Request #12663 · mne-tools/mne-python · GitHub

2 Likes
3

Hi @Cathaway

I also agree that the documentation can be confusing, I’ve been tripped up by it myself more than once!

I think that in your case with permutation_cluster_test, option #2 is the correct approach: you should pass a list containing 4 Numpy arrays to the X parameter. Each array in the list represents 1 of your 4 conditions, and should be of shape (n_participants, n_times) == (35, 1800) . For example:

X = [condition_A_data, condition_B_data, condition_C_data, condition_D_data]

Here, n_times comes last because the last dimension is the one along which adjacency is defined (time, in your case).

Can you point us to the specific line of code you’re referring to? I may be misunderstanding your description.. To me, “option #1” sounds like passing a list of n_observations different arrays, but that’s not the same thing as having n_observations as the 0-axis of a single array.

Looking at this section of the tutorial, it seems consistent with the API: they pass a list of 4 arrays, each shaped (n_observations, n_frequencies, n_times) to permutation_cluster_test.

That being said, I’m always open to being wrong, and to documentation improvements! Thanks for opening up a ticket on GitHub!

1 Like