# Dropped epochs when preload=True **URL:** \ **Category:** Support & Discussions\ **Tags:** preprocessing, eeg, evoked\ **Created:** [January 16, 2021, 1:56pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503 "2021-01-16T13:56:30Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![robpetrosino](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/robpetrosino/32/51_2.png) [@robpetrosino](https://mne.discourse.group/u/robpetrosino)\ **Post date:** [January 16, 2021, 1:56pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/1 "2021-01-16T13:56:30Z")
I have a question regarding the `mne.Epochs()` method. I have noticed that epochs are dropped only when `preload=True`. ```python epochs = mne.Epochs(raw_highpass, events, event_id, tmin, tmax, metadata=regressorsPd, baseline=(-.1, 0), preload=False, reject=None, reject_by_annotation=False) Adding metadata with 6 columns Replacing existing metadata with 6 columns 67 matching events found Applying baseline correction (mode: mean) 0 projection items activated epochsPreload = mne.Epochs(raw_highpass, events, event_id, tmin, tmax, metadata=regressorsPd, baseline=(-.1, 0), preload=True, reject=None, reject_by_annotation=False) Adding metadata with 6 columns Replacing existing metadata with 6 columns 67 matching events found Applying baseline correction (mode: mean) 0 projection items activated Loading data for 67 events and 451 original time points ... 45 bad epochs dropped ``` The 45 bad epochs are dropped because ‘too short’ as indicated by the log: ```python epochsPreload.drop_log Out[86]: ((), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',), ('TOO_SHORT',)) ``` I don’t understand two things. 1. What is the reason of this? The helper says that epochs are ‘too short’ if they do not “contain enough data names of channels that exceeded the amplitude threshold”. I don’t understand what this means, especially since I have set no amplitude threshold for epoching. 2. Why are ‘too short’ epochs being dropped only when they are preloaded? ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [January 16, 2021, 2:08pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/2 "2021-01-16T14:08:01Z")
> [@robpetrosino](#): > > I don’t understand two things. > > 1. What is the reason of this? The helper says that epochs are ‘too short’ if they do not contain enough data names of channels that exceeded the amplitude threshold. However, I have set no amplitude threshold. This clearly makes no sense, yes. Did you by any chance crop the data without dropping the cropped-out events? > [@](#): > > 1. Why are ‘too short’ epochs being dropped only when they are preloaded? Because only when you preload, epochs will be dropped on instantiation . To do it later on, you need to call `Epochs.drop_bad()`. ---
**Author:** ![robpetrosino](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/robpetrosino/32/51_2.png) [@robpetrosino](https://mne.discourse.group/u/robpetrosino)\ **Post date:** [January 16, 2021, 2:18pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/3 "2021-01-16T14:18:51Z")
> [@richard](#): > > This clearly makes no sense, yes. Did you by any chance crop the data without dropping the cropped-out events? Not sure what you mean here. I did retrieve only one event type from the annotations, if that is what you mean, but I don’t think I did any further cropping… ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [January 16, 2021, 2:22pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/4 "2021-01-16T14:22:05Z")
> [@robpetrosino](#): > > Not sure what you mean here. I did retrieve only one event type from the annotations, if that is what you mean, but I don’t think I did any further cropping… I wonder if something went wrong there (going from Annotations to events), in the sense that the event onsets are shifted to later times – time points beyond the last time point in your raw data. If you look at the `events` array, the first column contains the onset samples. Could you check for the last few rows which samples these are, and if they fall outside the sample range of your raw data? ---
**Author:** ![robpetrosino](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/robpetrosino/32/51_2.png) [@robpetrosino](https://mne.discourse.group/u/robpetrosino)\ **Post date:** [January 16, 2021, 2:32pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/5 "2021-01-16T14:32:45Z")
I don’t think so, if I understand you are suggesting: ```python raw_events[-10:] Out[108]: array([[1600812, 0, 30], [1605047, 0, 30], [1611996, 0, 30], [1617988, 0, 30], [1631646, 0, 30], [1680936, 0, 30], [1694680, 0, 30], [1699852, 0, 30], [1720664, 0, 30], [1746342, 0, 30]]) raw_highpass Out[104]: ``` The last event began at `1746342` and the sample range of the data is `0,1803542`. Is this what you mean? I also did some downsampling before epoching, but I also made sure that both data and events were downsampled together (as we discussed in a separate discussion)… ---
**Author:** ![robpetrosino](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/robpetrosino/32/51_2.png) [@robpetrosino](https://mne.discourse.group/u/robpetrosino)\ **Post date:** [January 16, 2021, 2:42pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/6 "2021-01-16T14:42:40Z")
UPDATE: I have tried to do epoching of the raw data without downsampling, and no epoch was dropped. So, I think the issue is linked to downsampling. I’ll look into this more later tonight. ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [January 16, 2021, 4:05pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/7 "2021-01-16T16:05:04Z")
Could you share the Annotations of the downsampled data? ---
**Author:** ![robpetrosino](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/robpetrosino/32/51_2.png) [@robpetrosino](https://mne.discourse.group/u/robpetrosino)\ **Post date:** [January 18, 2021, 11:38am UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/8 "2021-01-18T11:38:51Z")
Hi @richard, I think I have sent you the annotations file via direct message. ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [January 18, 2021, 11:54am UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/9 "2021-01-18T11:54:07Z")
Hello, yes, thank you! I looked at the Annotations and they seem fine to me… Do you think you could share the original file(s) you used to run into this issue, so I could try to reproduce myself? ---
**Author:** ![robpetrosino](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/robpetrosino/32/51_2.png) [@robpetrosino](https://mne.discourse.group/u/robpetrosino)\ **Post date:** [January 18, 2021, 1:29pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/10 "2021-01-18T13:29:15Z")
I’ll be sending you the file via direct message – thanks! ---
**Author:** ![robpetrosino](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/robpetrosino/32/51_2.png) [@robpetrosino](https://mne.discourse.group/u/robpetrosino)\ **Post date:** [January 19, 2021, 1:59pm UTC](https://mne.discourse.group/t/dropped-epochs-when-preload-true/2503/11 "2021-01-19T13:59:43Z")
I finally figured out what was going on. It was a silly mistake of mine – in the `mne.Epochs` call, I called the `events` array created _before_ downsampling, rather than the `down_events` array created after it. So, for future reference – if you ever run across a similar issue, make sure you have select the correct event array when epoching.