Is it possible to plug a 2x3 Mixed ANOVA into the spatio_temporal_cluster_test?

Hi everyone!

  • MNE version: MNE 1.9.0
  • macOS 15.6

I am doing fully exploratory analysis and have been trying to run a permutation cluster test with a 2 (group: patients x HC) x 3 (task: task1 x task2 x task3) ANOVA, instead of a simple bivariate comparison.

I have run the one-way F-test for each group separately. There is an effect for two tasks but not for one. This requires a test of interaction. I initially tried to do this based on differences per group (e.g. task1-task2), but this does not give me meaningful results and does not find any significant clusters at all, no matter what the threshold.

So I figured, I better run a real ANOVA instead. I used the following code, but I could not progress due to errors that I could not solve.


# -------------------------------
# Define groups (HC = 1, SSD = 0)
# -------------------------------
groups = np.array(
    [1]*hc_group_data_for_spatiotemporal_test["ec"].shape[0] +
    [0]*ssd_group_data_for_spatiotemporal_test["ec"].shape[0]
)
n_subjects = len(groups)
print("Groups array:", groups)

# -------------------------------
# Prepare X_list for all 3 conditions (ec, eo, hct)
# -------------------------------
X_list = []
for task in ["ec", "eo", "hct"]:
    X_hc  = hc_group_data_for_spatiotemporal_test[task]  # shape: n_HC × n_channels × n_times
    X_ssd = ssd_group_data_for_spatiotemporal_test[task] # shape: n_SSD × n_channels × n_times
    X_all = np.concatenate([X_hc, X_ssd], axis=0)       # n_subjects_total × n_channels × n_times
    # Transpose for MNE: (n_subjects, n_times, n_channels)
    X_all = np.transpose(X_all, (0, 2, 1))
    X_list.append(X_all)

print("Prepared X_list shapes (subjects, times, channels) for each condition:")
for cond_idx, arr in enumerate(X_list):
    print(f"{['ec','eo','hct'][cond_idx]}: {arr.shape}")


# -------------------------------
# Build combined adjacency (spatial + temporal)
# -------------------------------
# Use one evoked from HC as template
template_evoked = list(hc_data['ec'].values())[0][0]  # first subject, first evoked
channels_of_interest = list(range(0, 31))             # adjust as needed

# Spatial adjacency
adjacency, ch_names = find_ch_adjacency(template_evoked.info, ch_type='eeg')
ch_idx = channels_of_interest
adjacency = adjacency[ch_idx, :][:, ch_idx]

# Temporal adjacency: 1-neighbor adjacency along time
n_times = X_list[0].shape[1]
temporal_adj = np.eye(n_times, k=1) + np.eye(n_times, k=-1)

# Combine adjacency
combined_adj = combine_adjacency(adjacency, temporal_adj)
print("Combined adjacency shape:", combined_adj.shape)def mixed_anova_stat_fun_multi_channel(X, groups):
    # X is a tuple of arrays (one per condition)
    n_conditions = len(X)
    n_subjects = X[0].shape[0]

    # Infer number of times and channels from total features
    n_features = X[0].shape[1]
    n_times = 75
    n_channels = 31

    # Reshape each condition back to (subjects, times, channels)
    X_reshaped = [x.reshape(n_subjects, n_times, n_channels) for x in X]
    
    F_vals_condition = np.zeros((n_times, n_channels))
    F_vals_interaction = np.zeros((n_times, n_channels))

    for t in range(n_times):
        for ch in range(n_channels):
            data = np.column_stack([cond[:, t, ch] for cond in X_reshaped])
            # Create a dataframe for mixed ANOVA
            subjects = np.arange(n_subjects)
            df = pd.DataFrame({
                'y': data.flatten(),
                'condition': np.repeat(['ec', 'eo', 'hct'], n_subjects),
                'subject': np.tile(subjects, n_conditions),
                'group': np.tile(groups, n_conditions)
            })

            aov = pg.mixed_anova(dv='y', within='condition', between='group',
                     subject='subject', data=df)
            F_vals_condition[t, ch] = aov.loc[aov['Source'] == 'condition', 'F'].values[0]
            F_vals_interaction[t, ch] = aov.loc[aov['Source'] == 'Interaction', 'F'].values[0]

    # Stack along a new axis: 0 = condition effect, 1 = interaction
    F_vals = np.stack([F_vals_condition, F_vals_interaction], axis=0)  # shape (2, n_times, n_channels)
    return F_vals



# run cluster perm test
F_obs_condition, F_obs_interaction, clusters, cluster_pv, H0 = spatio_temporal_cluster_test(
    X_list,
    adjacency=combined_adj,
    n_permutations=1000,
    threshold=None,
    tail=0,
    stat_fun=lambda *X: mixed_anova_stat_fun_multi_channel(list(X), groups),
    verbose=True
)

# Error I get:
ValueError                                Traceback (most recent call last)
Cell In[532], line 4
      1 # -------------------------------
      2 # Run spatio-temporal cluster test
      3 # -------------------------------
----> 4 F_obs_condition, F_obs_interaction, clusters, cluster_pv, H0 = spatio_temporal_cluster_test(
      5     X_list,
      6     adjacency=combined_adj,
      7     n_permutations=1000,
      8     threshold=None,
      9     tail=0,
     10     stat_fun=lambda *X: mixed_anova_stat_fun_multi_channel(list(X), groups),
     11     verbose=True
     12 )
     15 print("Cluster test finished!")
     16 print("F_obs_condition shape:", F_obs_condition.shape)

File <decorator-gen-306>:10, in spatio_temporal_cluster_test(X, threshold, n_permutations, tail, stat_fun, adjacency, n_jobs, seed, max_step, spatial_exclude, step_down_p, t_power, out_type, check_disjoint, buffer_size, verbose)

File ~/miniconda3/envs/mne/lib/python3.12/site-packages/mne/stats/cluster_level.py:1554, in spatio_temporal_cluster_test(X, threshold, n_permutations, tail, stat_fun, adjacency, n_jobs, seed, max_step, spatial_exclude, step_down_p, t_power, out_type, check_disjoint, buffer_size, verbose)
   1552 else:
   1553     exclude = None
-> 1554 return permutation_cluster_test(
   1555     X,
...
---> 13 X_reshaped = [x.reshape(n_subjects, n_times, n_channels) for x in X]
     15 F_vals_condition = np.zeros((n_times, n_channels))
     16 F_vals_interaction = np.zeros((n_times, n_channels))

ValueError: cannot reshape array of size 103000 into shape (103,75,31)

Maybe you have a solution or another idea as to how I should approach this analysis?

I would really appreciate it!

Cheers,

Deniz

1 Like

Hi Deniz,

Thank you for the code example. I am pretty sure the 2x3 ANOVA should work with the cluster permutation function implemented.
It’s pretty difficult to debug your shape mismatch without some sample data. So just some thoughts I had going through your code:

  1. n_times and n_channels are hard-coded but should be inferred from the evoked data shapes to avoid shape mismatches

  2. I am not sure why you are reshaping within the anova_stat_fun

    # Reshape each condition back to (subjects, times, channels)

    X_reshaped = [x.reshape(n_subjects, n_times, n_channels) for x in X]

    after you reshaped the data while creating the condition arrays:

    # Transpose for MNE: (n_subjects, n_times, n_channels)
    X_all = np.transpose(X_all, (0, 2, 1))

Let me know if this helped; otherwise, I suggest providing minimal data (toy data or real data) so we can produce a minimal working code.

Cheers,

Carina

1 Like

Hi Carina,

Thank you very much, I appreciate the support!

  1. That was just a debugging trace on my end. It wasn’t a cause of error, though; fixed now!
  2. I added this reshape statement because, without it, I get the following error:
 def mixed_anova_stat_fun_multi_channel(X, groups):
    # X is a tuple of arrays (one per condition)
    n_conditions = len(X)
    n_subjects = X[0].shape[0]

    # Infer number of times and channels from total features
    n_features = X[0].shape[1]
    n_times = hc_group_data_for_spatiotemporal_test["ec"].shape[2]
    n_channels = hc_group_data_for_spatiotemporal_test["ec"].shape[1]

    # Reshape each condition back to (subjects, times, channels)
    #X_reshaped = [x.reshape(n_subjects, n_times, n_channels) for x in X]
    
    F_vals_condition = np.zeros((n_times, n_channels))
    F_vals_interaction = np.zeros((n_times, n_channels))

    for t in range(n_times):
        for ch in range(n_channels):
            data = np.column_stack([cond[:, t, ch] for cond in X])
            # Create a dataframe for mixed ANOVA
            subjects = np.arange(n_subjects)
            df = pd.DataFrame({
                'y': data.flatten(),
                'condition': np.repeat(['ec', 'eo', 'hct'], n_subjects),
                'subject': np.tile(subjects, n_conditions),
                'group': np.tile(groups, n_conditions)
            })

            aov = pg.mixed_anova(dv='y', within='condition', between='group',
                     subject='subject', data=df)
            F_vals_condition[t, ch] = aov.loc[aov['Source'] == 'condition', 'F'].values[0]
            F_vals_interaction[t, ch] = aov.loc[aov['Source'] == 'Interaction', 'F'].values[0]

    # Stack along a new axis: 0 = condition effect, 1 = interaction
    F_vals = np.stack([F_vals_condition, F_vals_interaction], axis=0)  # shape (2, n_times, n_channels)
    return F_vals


# See error below:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[549], line 4
      1 # -------------------------------
      2 # 5️⃣ Run spatio-temporal cluster test
      3 # -------------------------------
----> 4 F_obs_condition, F_obs_interaction, clusters, cluster_pv, H0 = spatio_temporal_cluster_test(
      5     X_list,
      6     adjacency=combined_adj,
      7     n_permutations=1000,
      8     threshold=None,
      9     tail=0,
     10     stat_fun=lambda *X: mixed_anova_stat_fun_multi_channel(list(X), groups),
     11     verbose=True
     12 )
     15 print("Cluster test finished!")
     16 print("F_obs_condition shape:", F_obs_condition.shape)

File <decorator-gen-306>:10, in spatio_temporal_cluster_test(X, threshold, n_permutations, tail, stat_fun, adjacency, n_jobs, seed, max_step, spatial_exclude, step_down_p, t_power, out_type, check_disjoint, buffer_size, verbose)

File ~/miniconda3/envs/mne/lib/python3.12/site-packages/mne/stats/cluster_level.py:1554, in spatio_temporal_cluster_test(X, threshold, n_permutations, tail, stat_fun, adjacency, n_jobs, seed, max_step, spatial_exclude, step_down_p, t_power, out_type, check_disjoint, buffer_size, verbose)
   1552 else:
   1553     exclude = None
-> 1554 return permutation_cluster_test(
   1555     X,
...
---> 20         data = np.column_stack([cond[:, t, ch] for cond in X])
     21         # Create a dataframe for mixed ANOVA
     22         subjects = np.arange(n_subjects)

IndexError: too many indices for array: array is 2-dimensional, but 3 were indexed


When I add it, this happens:

def mixed_anova_stat_fun_multi_channel(X, groups):
    # X is a tuple of arrays (one per condition)
    n_conditions = len(X)
    n_subjects = X[0].shape[0]

    # n_features = total features (flattened times × channels)
    n_features = X[0].shape[1]
    n_times = hc_group_data_for_spatiotemporal_test["ec"].shape[2]
    n_channels = hc_group_data_for_spatiotemporal_test["ec"].shape[1]

    # reshape each 2D array back to 3D (subjects, times, channels)
    X_reshaped = [x.reshape(n_subjects, n_times, n_channels) for x in X]

    F_vals_condition = np.zeros((n_times, n_channels))
    F_vals_interaction = np.zeros((n_times, n_channels))

    for t in range(n_times):
        for ch in range(n_channels):
            data = np.column_stack([cond[:, t, ch] for cond in X_reshaped])
            subjects = np.arange(n_subjects)
            df = pd.DataFrame({
                'y': data.flatten(),
                'condition': np.repeat(['ec', 'eo', 'hct'], n_subjects),
                'subject': np.tile(subjects, n_conditions),
                'group': np.tile(groups, n_conditions)
            })

            aov = pg.mixed_anova(dv='y', within='condition', between='group',
                                 subject='subject', data=df)
            F_vals_condition[t, ch] = aov.loc[aov['Source'] == 'condition', 'F'].values[0]
            F_vals_interaction[t, ch] = aov.loc[aov['Source'] == 'Interaction', 'F'].values[0]

    # Stack along a new axis: 0 = condition effect, 1 = interaction
    F_vals = np.stack([F_vals_condition, F_vals_interaction], axis=0)  # shape (2, n_times, n_channels)
    return F_vals



# See error below:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[554], line 4
      1 # -------------------------------
      2 # 5️⃣ Run spatio-temporal cluster test
      3 # -------------------------------
----> 4 F_obs_condition, F_obs_interaction, clusters, cluster_pv, H0 = spatio_temporal_cluster_test(
      5     X_list,
      6     adjacency=combined_adj,
      7     n_permutations=1000,
      8     threshold=None,
      9     tail=0,
     10     stat_fun=lambda *X: mixed_anova_stat_fun_multi_channel(list(X), groups),
     11     verbose=True
     12 )
     15 print("Cluster test finished!")
     16 print("F_obs_condition shape:", F_obs_condition.shape)

File <decorator-gen-306>:10, in spatio_temporal_cluster_test(X, threshold, n_permutations, tail, stat_fun, adjacency, n_jobs, seed, max_step, spatial_exclude, step_down_p, t_power, out_type, check_disjoint, buffer_size, verbose)

File ~/miniconda3/envs/mne/lib/python3.12/site-packages/mne/stats/cluster_level.py:1554, in spatio_temporal_cluster_test(X, threshold, n_permutations, tail, stat_fun, adjacency, n_jobs, seed, max_step, spatial_exclude, step_down_p, t_power, out_type, check_disjoint, buffer_size, verbose)
   1552 else:
   1553     exclude = None
-> 1554 return permutation_cluster_test(
   1555     X,
...
---> 12 X_reshaped = [x.reshape(n_subjects, n_times, n_channels) for x in X]
     14 F_vals_condition = np.zeros((n_times, n_channels))
     15 F_vals_interaction = np.zeros((n_times, n_channels))

ValueError: cannot reshape array of size 248775 into shape (110,75,31)

I am happy to provide some fully anonymized data (it is just some matrices anyway :slight_smile: ).

My X_list is saved here:

This code should recreate it:

data = np.load(file_name)
X_list_loaded = [data['ec'], data['eo'], data['hct']]
groups_loaded = data['groups']

I look forward to your reply and thank you very much in advance.

All the best,

Deniz