Unable to pickle load python object with mne.info field after upgrading to 1.1.1

Support & Discussions
1
  • MNE version: 1.1.1
  • operating system: Windows

I recently upgraded my MNE version from 0.24 → 1.1.1, but since then I am unable to load pickled files.
These pickled files contain python 3 objects with attributes, including the mne info field type.

However, I am unable to open these pickled files and I believe the error lies in the compability of different MNE versions:

File ~\anaconda3\lib\site-packages\mne\io\meas_info.py:867 in setstate
self._unlocked = state[‘_unlocked’]

KeyError: ‘_unlocked’

Any suggestions here? Preferrably I would not downgrade my mne version solely for this problem, because I need to maintain my produced code for the company next few years and would like to keep it up-to-date.

2

pickle is not a good file format to persist data between software versions.

there is no way we can fix this. You need to load it again with 0.24 and save your data
to fif to make it work

Alex

PS : note that this is not an MNE problem it’s the pickle format for all python packages

2 Likes
3

I’d like to second that: never, ever use pickle for long-term storage. Stuff will break.

4

You are right, and I do regret this decision now. I made this decision two years ago when I wanted to have one file with all the information on the data processing.

If there is no simple workaround, then I will take your advice.

1 Like
5

If you do as @agramfort suggests, you don’t need to stay downgraded. Create a new virtual environment, install MNE-Python 0.24 to it, load the pickles, save as .fif instead, then switch back to the MNE-Python 1.1 environment.

1 Like
6

Sorry to come back again, but I realized something new. One thing I a wondering now is whether I caused this behavior setting attributes. Somehow these attributes may have survived before the update in the serialization/deserialization process, which is why I never ran into this problem before.

I added new fields to the info struct this way:

     info = cepochs.info
     info.loss = loss
     info.threshold= thresh
     info.bad_channels = bad_indices
     info.bad_epochs = bad_epochs   
     info.dur = dur

Of course none of these new attributes are defined to be to be part of the info class.

If I print the state of the class object right before the error, I get the attribute paramters returned:

{‘loss’: 50, ‘threshold’: 0.0001, ‘bad_channels’: , ‘bad_epochs’: (array(, dtype=int64),), ‘dur’: 177.0, ‘rej_muscle’: 4.737999999999994, ‘rej_eog’: 4.2, ‘method’: ‘removal’}

Perhaps it are these attributes that disallow me to unlock all field names… If so, it possible to ignore them manually? And is there a safe way to add more field names to the info class object?

7

To my knowledge: Not really, no.

Please also note that Info inherits from dict.

8

Okay, thanks for your feedback!