Error when saving epoched data

Support & Discussions
1

Hi,

I’m using mne version 1.7.0 on Windows 11 and want to automatically reject bad EEG segments by using epochs.drop_bad() with a custom function:

def reject_min_max(x):
    max_cond = np.max(x, axis=1) > 70
    min_cond = np.min(x, axis=1) < -70
    
    return ((max_cond.any() or min_cond.any()), ['max_amp', 'min_amp'])


epochs_min_max.drop_bad(reject=dict(eeg=reject_min_max))

The function seems to work and bad epochs are being dropped. However, when trying to save the results using

filename_dat = 'data/preprocessed_data/EEG/' + id + '_run' + str(run) + '-epo.fif'
epochs_min_max.save(filename_dat)

I get the following error:

TypeError                                 Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_10796\638352696.py in <module>
      1 # save
      2 filename_dat = 'data/preprocessed_data/EEG/' + id + '_run' + str(run) + '-epo.fif'
----> 3 epochs_min_max.save(filename_dat)

<decorator-gen-247> in save(self, fname, split_size, fmt, overwrite, split_naming, verbose)

c:\Users\aalab\anaconda3\envs\mne_env\lib\site-packages\mne\epochs.py in save(self, fname, split_size, fmt, overwrite, split_naming, verbose)
   2268         reject_params = _pack_reject_params(self)
   2269         if reject_params:
-> 2270             over_size += len(json.dumps(reject_params)) + 16
   2271         # 8. selection
   2272         total_size += self.selection.size * 4

c:\Users\aalab\anaconda3\envs\mne_env\lib\json\__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    229         cls is None and indent is None and separators is None and
    230         default is None and not sort_keys and not kw):
--> 231         return _default_encoder.encode(obj)
    232     if cls is None:
    233         cls = JSONEncoder

c:\Users\aalab\anaconda3\envs\mne_env\lib\json\encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

c:\Users\aalab\anaconda3\envs\mne_env\lib\json\encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

c:\Users\aalab\anaconda3\envs\mne_env\lib\json\encoder.py in default(self, o)
    177 
    178         """
--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '
    180                         f'is not JSON serializable')
    181 

TypeError: Object of type function is not JSON serializable

Saving the data before applying drop_bad works perfectly fine. When applying drop_bad() with a threshold to reject epochs based on the peak-to-peak amplitude, it’s also possible to save the data. So I assume there is something wrong with the function I’m applying.

Thank you in advance for your help!

2

What happens if you preload the epochs before and/or after dropping (i.e. do epochs.load_data())?

3

I tried both, preloading before and after dropping - the error is still the same as above.

4

You’re setting the reject parameter to a dictionary that points to a function. Not the value that the function would return, but the function itself – this is bound to fail. I’m surprised we don’t catch this early on – it should ideally raise an exception right at this line already, not only when saving.

I think what you actually want is to iterate over epochs, check if the current epoch should be rejected, and if yes add its index to a list of epochs to be dropped. Then, call epochs.drop(...).

Richard

5

Thanks for your reply, iterating over epochs and then saving works.

Though this tutorial (“Rejecting Epochs using callables (functions)”) and the documentation explain how drop_bad() can be used with a function and I tried to do it in a similar way.

6

@leo you are right, passing a callable like you tried should be possible. Indeed, rejection works, but there is an error upon saving, so I suspect this has to be a bug. I suggest that you create an issue at Sign in to GitHub · GitHub.