Epoch rejection according to deviation from mean

  • MNE-Python version: 0.22.0
  • operating system: Ubuntu 20.04.2 LTS

Dear MNE community,

Is there a way of rejecting Epochs that are above a certain threshold, e.g 5 or 6 standard deviations above from the mean?

In the Makoto’s preprocessing pipeline page this method is recommended and described:

https://sccn.ucsd.edu/wiki/Makoto's_preprocessing_pipeline

Reject epochs for cleaning (11/13/2019 Updated)
 
Aim for around 5-10% rejection rate (Delorme et al., 2007). 
Apply amplitude threshold of -500 to 500 uV (don't capture eye blinks; 
you may want to exclude channels near AFp1 and AFp2 for this purpose),
 reject the selected epochs, and **apply improbability test** with 
**6SD** for **single channels** and **2SD** for **all channels**, and 
reject the selected again.

I’ve searched in the MNE website, but all I’ve found were instructions for setting a rejection criterion based on the amplitude (100 - 150 uV). Is there a standard practice recommended for MNE users? Can I just skip this ‘improbability test’?

I’ll be thankful for any feedback,

Kind regards,

Bruno

Hello and welcome to the forum!

No, this is currently not easily possible – you’d have to do this manually after creating the Epochs.

However, it should be quite easy to add support for this, by allowing the reject and flat dictionaries to accept callables (functions) as values, e.g. something like:

epochs = mne.Epochs(..., reject=dict(eeg=lambda x: True if np.median(x) > 1e-5 else False))

@larsoner @agramfort WDYT?

1 Like

Thanks for the fast reply, @richard. I’ll try this solution :slight_smile:

Just to avoid any misunderstanding: the code snippet I provided will not work with the current version of MNE-Python. It’s just something I could envision to support in the future.

For now, you could iterate over your Epochs and manually call a function. Let me know if you need help in doing that.

1 Like

Oh, ok… I was really trying to make that code snippet work.

I don’t know if I actually need to reject Epochs using standard deviation. My original problem is that I was using a rejection criterion of 100 micro Volts, just as in ’ Rejecting Epochs based on channel amplitude’ in the MNE website:

https://mne.tools/stable/auto_tutorials/preprocessing/plot_20_rejecting_bad_data.html?highlight=rejection

But even there they say that it might be too stringent.

I was deleting Epochs according to this threshold (100 microVolts) after visually inspecting the data and rejecting really bad data spans and also after performing ICA, so I thought 100 micro V. would be an acceptable threshold. However, most of my subjects had more than 80% of epochs rejected, or even 100%… So I thought it could be reasonable to delete epochs according to SD instead of setting a fix amplitude threshold… But there might be other solutions.

Would that be fine if I just increase the threshold to 200 micro Volts, or even more? That would avoid having to set a threshold based on standard deviations.

The Makoto pipeline uses 500 micro Volts, but I guess they perform it before ICA.

The “right” value to set the rejection threshold is highly data-dependent (how clean/noisy the original recordings are) and also will depend on what else you did first (filtering, ICA, etc). So unfortunately there is no one-size-fits-all advice for where to set the rejection threshold. You may be interested to experiment with the https://autoreject.github.io/ package.

Personally I would be hesitant to use a SD-based rejection, especially if applying it on a per-subject basis (imagine one subject where most/all data are very noisy, and another with a nice clean recording; a per-subject SD-based threshold will reject some from each subject and retain some from each subject, whereas maybe what you want to do is keep all epochs from the clean recording and reject all epochs from the noisy one).

1 Like

Yes, I agree with that!

1 Like

@richard @drammock
Is there any workaround for this?
Thanks in advance

A workaround for what…?

@richard I want to reject epochs based on standard deviation from the mean.

The article mentioned in the reference says

Any stimulus repeat for which the waveform deviated by six times the standard deviation (SD) from the mean at any time bin was considered as a bad repeat for that electrode.

Reference:
https://en.bio-protocol.org/en/bpdetail?id=4379&type=0