IndexError: Epoch index 148 is out of bounds when drop preselected epochs

If you have a question or issue with MNE-Python, please include the following info:

  • MNE-Python version: 0.24.dev0
  • operating system: ubuntu

The objective is to use list of an index to drop epochs.

However, the compiler return an error
IndexError: Epoch index 148 is out of bounds

The code is

import mne
from autoreject import Ransac, get_rejection_threshold  # noqa
epochs = mne.read_epochs ( 'raw-epo.fif', preload=True )
epoch_len = len ( epochs )
epochs.drop_channels ( ch_names=[epochs.ch_names [indices] for indices in
                                      list ( mne.pick_types ( epochs.info, meg=False, eeg=False, eog=True,
                                                              ecg=True ) )] )
epochs.drop_bad ( reject=get_rejection_threshold ( epochs, cv=10, decim=11 ), flat=None )


epo_selection = epochs.selection.tolist ()
chunk_size=10
epcohs_chunks=list(epo_selection [i:i+chunk_size] for i in range(0, len(epo_selection ), chunk_size))
epochs_drop_idx = [list ( set ( epo_selection ) - set ( a ) ) for a in epcohs_chunks]
selected_epoch=epochs_drop_idx[0]
epochs.drop ( selected_epoch, reason=['User reason'] * len ( selected_epoch ) )

May I know whether this is bug or incorrect setting from my side?

The files use in this report is accessible via:
raw-epo link

your raw data file is not accessible (access denied)

Thanks for the response @drammock . I have tweak the access permission, kindly try again. Thanks

I can’t quite figure out what you’re trying to do. Here’s an edited version of your script:

import mne
from autoreject import Ransac, get_rejection_threshold  # noqa
epochs = mne.read_epochs('raw-epo.fif', preload=True)

epochs.pick('eeg')  # this is equivalent to the drop_channels() call you were doing
reject = get_rejection_threshold(epochs, cv=10, decim=11)
epochs.drop_bad(reject=reject, flat=None)

# make groups of good epoch indices of size 10
epo_selection = epochs.selection.tolist()
chunk_size = 10
epochs_chunks = list(epo_selection[i:i+chunk_size]
                     for i in range(0, len(epo_selection), chunk_size))

# I can't figure out what you're trying to do with these last 3 lines
epochs_drop_idx = [list(set(epo_selection) - set(a)) for a in epochs_chunks]
selected_epoch = epochs_drop_idx[0]
epochs.drop(selected_epoch, reason=['User reason'] * len(selected_epoch))

Can you clarify what the goal is? you’ve already dropped some epochs based on autoreject threshold, why are you trying to drop more epochs, and how are you choosing which additional ones to drop?

Thanks for the time entertaining this request.

Out of curiosity, I would like to visualize how each IComponent (present individually) affect the signal of the remaining good epoch.

Since, Im going to save offline the zeroed-all-except for one IC, this mean, only few epochs are printed at a particular time. Since large n_epochs make it difficult to view the signal clearly. For this, Im thinking to manually reject the remaining good epoch as per the line epochs.drop and save the epochs.plot output batch by batch (i.e., epochs_chunks)

P.S., I am aware, it is possible to view interactively all the good epochs using the epochs.plot().

So, you want to run ICA, and you want to reject all but one IC and apply it to the epochs so you can see its contribution across all epochs, and then manually reject some epochs (based on that plot). Is that right?

It is possible to zoom in on an epochs plot by presssing the home or end keys, to view fewer or more epochs at one time. I think this, combined with scrolling via shift+right arrow makes it easier to achieve what you want by plotting all the epochs at once and marking bad ones interactively, rather than splitting into chunks, saving the plot images, and reviewing them later. But maybe I’m still misunderstanding your goal.

We are on the same page on this.

[quote="balandongiv, post:5, topic:3

Since large n_epochs make it difficult to view the signal clearly.
[/quote]

This statement refer to when viewing the plot offline.

When viewing interactively a large channels and epochs number, my system usually take ages to refresh when moving left-right (epochs) and up-down(channels). This is one main motivation why I prefer to save it offline. This is also make life easier for future review.

OK, I understand the problem now. You want to do something like this:

for ix, drop in enumerate(epochs_drop_idx):
    fig = epochs.copy().drop(drop).plot(show=False)
    fig.savefig(f'{ix}.png')

But get the IndexError: Epoch index 148 is out of bounds error. This happens because (quoting from the Epochs.drop() docstring):

The indices refer to the current set of undropped epochs rather than the complete set of dropped and undropped epochs. They are therefore not necessarily consistent with any external indices (e.g., behavioral logs). To drop epochs based on external criteria, do not use the preload=True flag when constructing an Epochs object, and call this method before calling the mne.Epochs.drop_bad or mne.Epochs.load_data methods.

In your case this isn’t possible because you’ve already preloaded epochs in order to do the earlier dropping. So I think what you need to do is use numpy to get the correct indices. Something like:

for ix, drop in enumerate(epochs_drop_idx):
    boolean_drop = np.in1d(epo_selection, drop)
    fig = epochs.copy().drop(boolean_drop).plot(show=False)
    fig.savefig(f'{ix}.png')

Great, thanks for the detail explanation . It really help :smiling_face_with_three_hearts: