I’ve got a question about the repeated measures ANOVA cluster test with regards to what kind of repeated measures it supports. My study design is a cross-over study with 4 different conditions, all tested on the same 10 subjects, each with multiple repeated trials (68 after bad epoch rejection and equalizing counts among them).
I understand the basic data structure needed for the test as passing a list of
[condition1array, condition2array, … ], where conditionNarray has shape (samples(subjects), time, space)
However, I’m a little confused about the samples(subjects) dimension. Is it proper to use all the samples from all the subjects (i.e. conditionNarray has shape (10*68, time, space))? Part of me thinks that I could do that and then use an f-threshold of
mne.stats.f_threshold_mway_rm(n_subjects = 10)
But part of me is also a little confused with how the permutations would work, since, as far as I can tell, the permutations do not keep track of which data are originally from which subjects. Does this test support repeated measures across multiple subjects, or just repeated measures across one subject at a time?
I really appreciate any insight you can provide. Thank you!
You would generally average across the 68 trials, so you would have for each subject a total of 4 Evoked objects (one for each condition). Then, condition1array would be created from the 10 Evoked objects belonging to the first condition, condition2array from the 10 Evoked objects belonging to the second condition, and so on.
Dear @Kosnoff ,
I was following this tutorial by @Denis et al. and was wondering along the same line. My goal was to perform a permutation test rmANOVA to check for the effect of a repeated Factor (condition with 3 levels) in X subjects measured in S sampling time points. I seem to have adopted the tutorial code successfully and the output looks plausible. However, I would like to understand how the permutation was conducted. I tried looking into the code of the cluster_permutation_test() function, but I have difficulty understanding it. I would like to understand whether and how this function in the case of rmANOVAs restricts the permutations within subjects.
Do you by any chance know the answer to your question by now? Thanks in advance
I haven’t fully used the tutorial and do not know what the intended purpose of it was. It claims repeated measures which to me sounds like a within-subjects design but I am not sure if it was really meant for that purpose?
In regards to the permutations, if I understand it correctly, the permutation scheme concatenates all conditions, and then reassign a random subset of that data back to one of the groups, so the size and shape of data within each group remains the same but with shuffled content. If I am not mistaken, that means that the data is permuted across all conditions and there is no accounting for repeated measures.
This is what the _do permutations_ function is doing internally, applied to the data in the turorial:
X_full = np.concatenate(epochs_power, axis = 0)
n_samples_per_condition = [x.shape[0] for x in epochs_power]
@aabielefeldt Thanks for digging into the source code. The more I look into and consider this function, the more I share your understanding. For example, comparing the permutation_cluster_test function to scipy’s permutation_test, I would have expected an input specifying pairing_type, there’s not one (and, as you pointed out, the code accordingly seems to just randomly shuffle across conditions using an independent samples structure).
I understand this works with the paired t-test because we are supposed to subtract the pairings to pair them prior to permuting, but we don’t do that for the F-tests. Instead, the only thing (to me) that seems to be connecting the repeated measures is the stat function.
For the tutorial on single-trial analysis, I sort of follow the logic; when you condition on a single subject, you can treat the single-trial samples as independent. However, if this was the intention, I think an independent samples F-test for the stat function would be more suitable? In the tutorial I originally linked, the opening paragraph specifically says it’s supposed to be for a group of subjects though.
Re-reading the tutorial, I wonder if the baseline correction is meant to somehow match/normalize the data in preparation for permutations–similar to how a permutation paired t-test has us subtract condition A from condition B? If that’s the case, though, maybe it could be clarified.
If the baseline correction does not serve that purpose, I think the implementation somewhat boils down to, fundamentally, the merits/validity of treating the samples as independent for the permutations generating the rm_F-stat histogram. My understanding is that this probably violates the exchangeability assumptions of permutation tests, but my knowledge of stats is not strong enough to make definitive statements about whether or not this violation is meaningful in the long run.
