# Epochs.save drops some epochs **URL:** \ **Category:** Support & Discussions\ **Tags:** preprocessing, meg, epochs\ **Created:** [July 29, 2021, 2:22pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470 "2021-07-29T14:22:02Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Fosca](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/fosca/32/724_2.png) [@Fosca](https://mne.discourse.group/u/Fosca)\ **Post date:** [July 29, 2021, 2:22pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/1 "2021-07-29T14:22:02Z")
- MNE-Python version: 0.19.0 - operating system: Linux Dear MEG developers, My colleague and I are encountering a problematic situation while saving the epochs of a participant. While initially all the epochs are good, at the moment of saving, the epochs.save function removes 2 epochs. We checked that : - epochs.drop\_log() contained only empty lists - epochs.drop\_bads() doesn’t drop any epoch We set annotations to None by running: epochs.set\_annotations(None). When saving, no message indicates that bad epochs were dropped but yet, when we load the epochs, the two last epochs are dropped. Something that seems to be at the origin of this problem is that we use « STI008 » as the stim channel and that there is a trigger in the channel « STI101 » that happens before the 2 last triggers of STI008. These 2 last ones are the ones that are later removed automatically by the saving function. We think this is a bug. Here is a snippet of code that should allow you to reproduce the problem (mne version ‘0.19.0’). How can we send you the problematic data ? It is on the NeuroSpin server. Thank you very much. Best, Fosca & Samuel ```python import mne data_folder = "/neurospin/meg/meg_tmp/ABSeq_Samuel_Fosca2019/data/MEG/" data_path = data_folder + "/data_bug_mne/bug_data_raw.fif" tmin = -0.050 tmax = 0.600 raw = mne.io.read_raw_fif(data_path,preload=True) events = mne.find_events(raw, stim_channel="STI008", consecutive=True, min_duration=0.002, shortest_event=1) epochs = mne.Epochs(raw, events, None, tmin, tmax, proj=True, baseline=None, preload=False, decim=1, reject=None) data_path_save = data_folder + "/data_bug_mne/test-epo.fif" epochs.save(data_path_save) ``` ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [July 29, 2021, 2:34pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/2 "2021-07-29T14:34:28Z")
Hello @Fosca and welcome to the forum! > [@Fosca](#): > > (mne version ‘0.19.0’). This version of MNE-Python is ancient. Please update to 0.23 and see if the problem persists. Thanks! ---
**Author:** ![Fosca](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/fosca/32/724_2.png) [@Fosca](https://mne.discourse.group/u/Fosca)\ **Post date:** [July 29, 2021, 3:15pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/3 "2021-07-29T15:15:23Z")
Dear @richard, Samuel and I just checked that the problem persists with MNE 0.23. Thanks in advance for your help. Fosca ---
**Author:** ![drammock](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/drammock/32/4_2.png) [@drammock](https://mne.discourse.group/u/drammock)\ **Post date:** [July 29, 2021, 3:16pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/4 "2021-07-29T15:16:41Z")
Thanks for including a code sample that yields the error. can you provide a download link for the problematic file? ---
**Author:** ![Fosca](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/fosca/32/724_2.png) [@Fosca](https://mne.discourse.group/u/Fosca)\ **Post date:** [August 2, 2021, 2:26pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/5 "2021-08-02T14:26:03Z")
Thanks a lot @drammock and sorry for the late reply. Here is the link to the data. Let me know if you have any problem loading it. > **[data\_bug\_mne.zip](https://wetransfer.com/downloads/57eff3a7641510b7d2d0b545a8e7a30820210802141352/8ebe2c00647408763ea78f3a87cf88db20210802141421/1c8e1a)** > > 1 file sent via WeTransfer, the simplest way to send your files around the world ---
**Author:** ![drammock](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/drammock/32/4_2.png) [@drammock](https://mne.discourse.group/u/drammock)\ **Post date:** [August 2, 2021, 11:24pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/6 "2021-08-02T23:24:37Z")
This is unrelated to saving and re-loading the epochs. Let’s look at your last 3 events, and how far away they are from the end of the file: ```python (raw.last_samp - events[-3:, 0]) / raw.info['sfreq'] array([0.796, 0.544, 0.293]) ``` since your `tmax` value is 0.6 seconds, the last two events are too close to the end of the raw object to be successfully extracted into epochs. If you did `epochs.load_data()` (or passed `preload=True` when you created the epochs) then you would have seen this: ```python Loading data for 736 events and 651 original time points ... 2 bad epochs dropped ``` and then if you did `epochs.plot_drop_log()` or viewed `epochs.drop_log` in the console, you would see that two epochs (the last two, in fact) were dropped for reason `TOO_SHORT`. The crucial step that you missed is probably that you didn’t load the epochs into memory before checking the drop log. Until loaded into memory, the epoch-by-epoch rejection does not happen (whether based on peak-to-peak signal amplitude, duration, annotations, whatever). ---
**Author:** ![Fosca](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/fosca/32/724_2.png) [@Fosca](https://mne.discourse.group/u/Fosca)\ **Post date:** [August 4, 2021, 2:26pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/7 "2021-08-04T14:26:45Z")
Dear @drammock, So we stopped the recording too early and cut part of the data we wanted to analyze. We indeed checked that, when loading the epochs data, the drop\_log now shows “Too short” for the last 2 epochs. Thanks a lot for your very helpful answer and for taking the time to explain everything in details. All the best, Fosca ---
**Author:** ![CarinaFo](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/carinafo/32/746_2.png) [@CarinaFo](https://mne.discourse.group/u/CarinaFo)\ **Post date:** [November 5, 2021, 7:13pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/8 "2021-11-05T19:13:07Z")
Hello MNE community, I have a related question to the topic of dropped epochs because of too late recording or too early stopping the recording. If I add metadata to the mne.epochs function and then epochs get dropped because the epoch length can’t be extracted does the number of trials (metadata) automatically get updated? I assume it does but just want to make sure that I am not misssing something here 🙂 Thank you for your help! Carina ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [November 5, 2021, 7:23pm UTC](https://mne.discourse.group/t/epochs-save-drops-some-epochs/3470/9 "2021-11-05T19:23:33Z")
Hello @CarinaFo, the metadata will be updated automatically such that it matches the retained epochs. Best wishes, Richard