Index Error while reading data

💬 Support & Discussions
1
  • MNE version: 1.1
  • operating system: Windows 11

Hello everyone, I’ve been having trouble reading some of my data. For 3 particular subjects in my dataset I am occasionally getting this error.

I am using .mff data from NetStation, which is produced by egi. The function read_raw_egi usually works perfectly and I can read the data.

Here is the code:

import os
import mne
import mne_bids

root_path = os.path.join("C:/Users/trevo/OneDrive/Desktop/Lab_Files/VWM_LAB/EGI_RAW/07/WM3/isi50.mff")

raw = mne.io.read_raw_egi(root_path)
raw.load_data()

And It produces this error:

IndexError Traceback (most recent call last)
Input In [21], in <cell line: 1>()
----> 1 raw = mne.io.read_raw_egi(root_path)
2 raw.load_data()

File :12, in read_raw_egi(input_fname, eog, misc, include, exclude, preload, channel_naming, verbose)

File ~\anaconda3\envs\mne\lib\site-packages\mne\io\egi\egi.py:156, in read_raw_egi(input_fname, eog, misc, include, exclude, preload, channel_naming, verbose)
154 input_fname = str(input_fname)
155 if input_fname.rstrip(‘/\’).endswith(‘.mff’): # allows .mff or .mff/
→ 156 return _read_raw_egi_mff(input_fname, eog, misc, include,
157 exclude, preload, channel_naming, verbose)
158 return RawEGI(input_fname, eog, misc, include, exclude, preload,
159 channel_naming, verbose)

File :12, in _read_raw_egi_mff(input_fname, eog, misc, include, exclude, preload, channel_naming, verbose)

File ~\anaconda3\envs\mne\lib\site-packages\mne\io\egi\egimff.py:389, in _read_raw_egi_mff(input_fname, eog, misc, include, exclude, preload, channel_naming, verbose)
328 @verbose
329 def _read_raw_egi_mff(input_fname, eog=None, misc=None,
330 include=None, exclude=None, preload=False,
331 channel_naming=‘E%d’, verbose=None):
332 “”“Read EGI mff binary as raw object.
333
334 … note:: This function attempts to create a synthetic trigger channel.
(…)
387 … versionadded:: 0.15.0
388 “””
→ 389 return RawMff(input_fname, eog, misc, include, exclude,
390 preload, channel_naming, verbose)

File :12, in init(self, input_fname, eog, misc, include, exclude, preload, channel_naming, verbose)

File ~\anaconda3\envs\mne\lib\site-packages\mne\io\egi\egimff.py:412, in RawMff.init(self, input_fname, eog, misc, include, exclude, preload, channel_naming, verbose)
408 misc = np.where(np.array(
409 egi_info[‘chan_type’]) != ‘eeg’)[0].tolist()
411 logger.info(’ Reading events …‘)
→ 412 egi_events, egi_info = _read_events(input_fname, egi_info)
413 cals = _get_eeg_calibration_info(input_fname, egi_info)
414 logger.info(’ Assembling measurement info …')

File ~\anaconda3\envs\mne\lib\site-packages\mne\io\egi\events.py:32, in _read_events(input_fname, info)
30 for n, event in enumerate(event_codes):
31 for i in mff_events[event]:
—> 32 events[n][i] = n + 1
33 return events, info

IndexError: index -585184 is out of bounds for axis 0 with size 373418

From what I’ve seen this post, it has something to do with reading in events. Though the solution implemented has not been working for my code.

I’ve been able to read this raw data with other software (NetStation Review) so I don’t believe that the data is corrupted, though I could be wrong.

I’ll include an example of a different scan which is successfully read by read_raw_egi, if that helps:

import os
import mne
import mne_bids

root_path = os.path.join("C:/Users/trevo/OneDrive/Desktop/Lab_Files/VWM_LAB/EGI_RAW/07/WM3/isi0.mff")

raw = mne.io.read_raw_egi(root_path)
raw.load_data()

Reading EGI MFF Header from C:\Users\trevo\OneDrive\Desktop\Lab_Files\VWM_LAB\EGI_RAW\07\WM3\isi0.mff…
Reading events …
Assembling measurement info …
Synthesizing trigger channel “STI 014” …
Excluding events {SESS} …
Reading 0 … 362911 = 0.000 … 362.911 secs…

Measurement date August 19, 2020 16:27:06 GMT
Experimenter Unknown
Participant Unknown
Digitized points 132 points
Good channels 129 EEG, 8 Stimulus
Bad channels None
EOG channels Not available
ECG channels Not available
Sampling frequency 1000.00 Hz
Highpass 0.00 Hz
Lowpass 500.00 Hz
Filenames signal1.bin
Duration 00:06:02 (HH:MM:SS)
2

to debug this we will need access to one of the files that fails to load. Are you allowed to share publicly? If not, maybe you can post a (restricted) google drive link and grant us access individually as we request it?

cc @scott-huberty in case you’ve seen this before and know what the fix should be.

1 Like
3

do you also have the problem when using this package https://pypi.org/project/mffpy/ ?

Alex

1 Like
4

Hello Dan,

You can find a restricted google drive link here to an .mff file that has the issue, it isn’t the exact file from my post and has been renamed to raw.mff.

I’ve added your email to the list of approved emails. I’ll be happy to add anyone else who would like to assist. Thank you, I appreciate the help!

Trevor

1 Like
5

I tried loading the file you shared and am getting the same error (with mne v1.1), but haven’t had the time to look more into it.

A couple questions -

Is there any chance that, between the time of acquisition and the time you tried reading the file, that the MFF file was uploaded to or from a server (gdrive/onedrive etc), without being zipped? I noticed that your paths includes OneDrive and I am not sure whether these files are then automatically uploaded and downloaded between a OneDrive server. Only reason I ask is that since .MFF files are essentially directories, a lot of problems are caused by files in the .mff directory being corrupted during data transfer. If so, can you try grabbing the file from the acquisition machine again? This might not be the issue, but It would be nice to double check.

As was suggested before can you try using the mffPy package to read in the file giving you trouble? This package is maintained by Don Tucker’s lab (founder of EGI), and it would be useful to know if your file breaks their code as well.

Thanks!

1 Like
6

Hi @trevidiann - just checking in. Have you made any progress in troubleshooting this problem? Is there any thing you are stuck on?

Scott

1 Like
8

Has anyone solved this problem? I’m running into the same issue and reading the .mff file using mffpy works just fine.

9

Hi @axel - not that i’m aware of, no.

I think there is a desire to use mffpy under the hood when reading MFF files into MNE (see here and here ) - which I think would solve this issue (since it seems that mffpy works well for you)

But I’m not sure how much work it will entail. I’d be willing to help on that but I likely wouldn’t have the time for it until mid-may.

10

Thanks for your response! I realized that read_raw_egi requires the events to be mutually exclusive, so I did some testing and found that I can read the file in when there are no overlapping triggers. However, when events are very close in time (but not overlapping), read_raw_egi throws an error. This is mainly an issue when I call for EGI and the testing computer to resync, so I can work around that.

I am concerned this will be an issue in the future though if participant responses are too close in time to event onsets. Not yet sure if anything else triggers the error.

11

Actually, scratch some of what I said earlier. I seem to have issues with other files when all trigger events are farther apart in time.
It seems like it’d be great if mffpy was reading files under the hood. Thanks for offering to help! Would really appreciate it.