# Epoch rejection according to deviation from mean

**URL:** <https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980>\
**Category:** Support & Discussions\
**Tags:** eeg\
**Created:** [April 13, 2021, 1:42pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980 "2021-04-13T13:42:55Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![bmmansur](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/bmmansur/32/142_2.png) [@bmmansur](https://mne.discourse.group/u/bmmansur)\
**Post date:** [April 13, 2021, 1:42pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/1 "2021-04-13T13:42:55Z")

</div>

- MNE-Python version: 0.22.0
- operating system: Ubuntu 20.04.2 LTS

Dear MNE community,

Is there a way of rejecting Epochs that are above a certain threshold, e.g **5 or 6 standard deviations above from the mean**?

In the Makoto’s preprocessing pipeline page this method is recommended and described:

[https://sccn.ucsd.edu/wiki/Makoto's\_preprocessing\_pipeline](https://sccn.ucsd.edu/wiki/Makoto%27s_preprocessing_pipeline)

```python
Reject epochs for cleaning (11/13/2019 Updated)
 
Aim for around 5-10% rejection rate (Delorme et al., 2007). 
Apply amplitude threshold of -500 to 500 uV (don't capture eye blinks; 
you may want to exclude channels near AFp1 and AFp2 for this purpose),
 reject the selected epochs, and **apply improbability test** with 
**6SD** for **single channels** and **2SD** for **all channels** , and 
reject the selected again.

```

I’ve searched in the MNE website, but all I’ve found were instructions for setting a rejection criterion based on the amplitude (100 - 150 uV). Is there a standard practice recommended for **MNE users**? Can I just skip this ‘improbability test’?

I’ll be thankful for any feedback,

Kind regards,

Bruno

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [April 13, 2021, 2:13pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/2 "2021-04-13T14:13:29Z")

</div>

Hello and welcome to the forum!

> [@bmmansur](#):
>
> Is there a way of rejecting Epochs that are above a certain threshold, e.g **5 or 6 standard deviations above from the mean**?

No, this is currently not easily possible – you’d have to do this manually after creating the `Epochs`.

However, it should be quite easy to add support for this, by allowing the `reject` and `flat` dictionaries to accept callables (functions) as values, e.g. something like:

```python
epochs = mne.Epochs(..., reject=dict(eeg=lambda x: True if np.median(x) > 1e-5 else False))

```

@larsoner @agramfort WDYT?

---

<div class="post-metadata">

**Author:** ![bmmansur](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/bmmansur/32/142_2.png) [@bmmansur](https://mne.discourse.group/u/bmmansur)\
**Post date:** [April 13, 2021, 2:42pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/3 "2021-04-13T14:42:54Z")

</div>

Thanks for the fast reply, @richard. I’ll try this solution 🙂

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [April 13, 2021, 3:02pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/4 "2021-04-13T15:02:51Z")

</div>

Just to avoid any misunderstanding: the code snippet I provided will **not** work with the current version of MNE-Python. It’s just something I could envision to support in the future.

For now, you could iterate over your `Epochs` and manually call a function. Let me know if you need help in doing that.

---

<div class="post-metadata">

**Author:** ![bmmansur](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/bmmansur/32/142_2.png) [@bmmansur](https://mne.discourse.group/u/bmmansur)\
**Post date:** [April 13, 2021, 3:54pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/5 "2021-04-13T15:54:14Z")

</div>

Oh, ok… I was really trying to make that code snippet work.

I don’t know if I actually need to reject Epochs using standard deviation. My original problem is that I was using a rejection criterion of **100** micro Volts, just as in ’ Rejecting Epochs based on channel amplitude’ in the MNE website:

[https://mne.tools/stable/auto\_tutorials/preprocessing/plot\_20\_rejecting\_bad\_data.html?highlight=rejection](https://mne.tools/stable/auto_tutorials/preprocessing/plot_20_rejecting_bad_data.html?highlight=rejection)

But even there they say that it might be **too stringent**.

I was deleting Epochs according to this threshold (100 microVolts) **after visually inspecting the data and rejecting really bad data spans and also after performing ICA** , so I thought 100 micro V. would be an acceptable threshold. However, most of my subjects had more than **80%** of epochs rejected, or even 100%… So I thought it could be reasonable to delete epochs according to SD instead of setting a fix amplitude threshold… But there might be other solutions.

Would that be fine if I just increase the threshold to **200** micro Volts, or even more? That would avoid having to set a threshold based on standard deviations.

The Makoto pipeline uses **500** micro Volts, but I guess they perform it **before ICA**.

---

<div class="post-metadata">

**Author:** ![drammock](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/drammock/32/4_2.png) [@drammock](https://mne.discourse.group/u/drammock)\
**Post date:** [April 13, 2021, 6:28pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/6 "2021-04-13T18:28:32Z")

</div>

The “right” value to set the rejection threshold is highly data-dependent (how clean/noisy the original recordings are) and also will depend on what else you did first (filtering, ICA, etc). So unfortunately there is no one-size-fits-all advice for where to set the rejection threshold. You may be interested to experiment with the [https://autoreject.github.io/](https://autoreject.github.io/) package.

Personally I would be hesitant to use a SD-based rejection, especially if applying it on a per-subject basis (imagine one subject where most/all data are very noisy, and another with a nice clean recording; a per-subject SD-based threshold will reject some from each subject and retain some from each subject, whereas maybe what you want to do is keep _all_ epochs from the clean recording and reject _all_ epochs from the noisy one).

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [April 13, 2021, 6:34pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/7 "2021-04-13T18:34:02Z")

</div>

> [@drammock](#):
>
> Personally I would be hesitant to use a SD-based rejection, especially if applying it on a per-subject basis

Yes, I agree with that!

---

<div class="post-metadata">

**Author:** ![MukilSaravanan](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@MukilSaravanan](https://mne.discourse.group/u/MukilSaravanan)\
**Post date:** [May 24, 2023, 11:10am UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/8 "2023-05-24T11:10:46Z")

</div>

@richard @drammock  
Is there any workaround for this?  
Thanks in advance

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [May 24, 2023, 12:29pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/9 "2023-05-24T12:29:34Z")

</div>

A workaround for what…?

---

<div class="post-metadata">

**Author:** ![MukilSaravanan](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@MukilSaravanan](https://mne.discourse.group/u/MukilSaravanan)\
**Post date:** [May 24, 2023, 1:07pm UTC](https://mne.discourse.group/t/epoch-rejection-according-to-deviation-from-mean/2980/10 "2023-05-24T13:07:11Z")

</div>

@richard I want to reject epochs based on standard deviation from the mean.

The article mentioned in the reference says

> Any stimulus repeat for which the waveform deviated by six times the standard deviation (SD) from the mean at any time bin was considered as a bad repeat for that electrode.

Reference:  
[https://en.bio-protocol.org/en/bpdetail?id=4379&type=0](https://en.bio-protocol.org/en/bpdetail?id=4379&type=0)