Clean Line noise (Zapline method) function for MNE using MEEGkit toolbox

💬 Support & Discussions
, ,
1

I wrote a function to clean line noise using zapline method from MEEGkit toolbox to use in MNE with RawArray data:

def zapline_clean(raw, fline):
    data = raw.get_data() # Convert mne data to numpy darray
    sfreq = raw.info['sfreq'] # Extract the sampling freq
   
    #Apply MEEGkit toolbox function
    out, _ = dss.dss_line(data, fline, sfreq, nkeep=1) # fline (Line noise freq) = 50 Hz for Europe

    cleaned_raw = mne.io.RawArray(out, raw.info) # Convert output to mne RawArray again

    return cleaned_raw

To implement this function I use:
cleaned_raw = zapline_clean(raw, 50)

Number 50 is the fline value (50hz line noise in europe)

However, I have a problem when I use this function. This appears in the console: “Reducing nfft to 32” and then the Python interpreter crashes. Has anyone tried doing something similar?

2

Hi @rbn13

can you provide the full stack trace? Looking at your function, I assume the problem occurs somewhere in the MEEGKIT code, so it might be a question for the developers of that toolbox.

Scott

1 Like
3

Thank you again for answering my question. Finally, the error was that the data matrix obtained from an MNE RawArray has a transposed shape, inversely to how it should be implemented in the MEEGtoolkit function. To solve it, it was enough to transpose the data using two alternatives depending on the library we use: data.T (with numpy) or data.transpose (with pandas).

I hope this can be helpful to someone who has encountered the same problem.

2 Likes
4

Hi! I am getting a memory error when running this function:

MemoryError: Unable to allocate 3.19 TiB for an array with shape (661708, 661708) and data type float64

This is a 5 min EEG recording with 2000 Hz sampling rate. Downsampling at 500 Hz didn’t solve the problem.

Could you give me any suggestions please?

5

It looks like you didn’t transpose the data as mentioned in the solution.

6

It’s not that. I have tried to transpose the matrix adn got a value error:

ValueError: len(data) (661708) does not match len(info[“ch_names”]) (63)

So the data was shaped properly in the first place.

7

Please show your code then.

8 
raw_bl = mne.io.read_raw_brainvision(eeg_bl, eog=(), preload=True)
events = mne.events_from_annotations(raw_bl)

data = raw_bl.get_data() # Convert mne data to numpy darray
sfreq = raw_bl.info['sfreq'] # Extract the sampling freq
**out, _ = dss.dss_line(data, 50, sfreq, nremove=4)**
cleaned_raw = mne.io.RawArray(out, raw_bl.info)

I’ve highlighted the line that produces the error.
Here’s the full error message:

MemoryError Traceback (most recent call last) Cell In[23], line 5 2 sfreq = raw_bl.info[‘sfreq’] # Extract the sampling freq 4 #Apply MEEGkit toolbox function ----> 5 out, _ = dss.dss_line(data, 50, sfreq, nremove=4) # fline (Line noise freq) = 50 Hz for Europe 7 cleaned_raw = mne.io.RawArray(out, raw_bl.info) # Convert output to mne RawArray again File d:.…\meegkit\dss.py:230, in dss_line(X, fline, sfreq, nremove, nfft, nkeep, blocksize, show) [228](file:///D:/…\meegkit\dss.py:228) # Compute blockwise covariances of raw and biased data [229](file:///D:/…\meegkit\dss.py:229) n_harm = np.floor((sfreq / 2) / fline).astype(int) → [230](file:///D:/…\meegkit\dss.py:230) c0 = np.zeros((nkeep, nkeep)) [231](file:///D:/…\meegkit\dss.py:231) c1 = np.zeros((nkeep, nkeep)) [232](file:///D:/…\meegkit\dss.py:232) for X_block in sliding_window_view(X_noise_pca, (blocksize, nkeep), [233](file:///D:/…\meegkit\dss.py:233) axis=(0, 1))[::blocksize, 0]: [234](file:///D:/…\meegkit\dss.py:234) # if n_trials>1, reshape to (n_samples, nkeep, n_trials) MemoryError: Unable to allocate 3.19 TiB for an array with shape (661708, 661708) and data type float64

1 Like
9

You did not transpose the data. The line should be:

out, _ = dss.dss_line(data.T, 50, sfreq, nremove=4)
10

I know that. Because the transposed data gives the ValueError I described above. There is no problem with the shape of the data, but rather with its size.

11

Then you should report your problem to the meegkit authors.

12

I’m also confused about the missing transpose :thinking: Something is fishy here

I would assume that you must transpose, and the other error you get then is a problem with MEEGKit

If the dimensions are wrong, memory usage will blow out of proportion as you normally have many more time points than channels.

1 Like
13

meegkit.dss — MEEGkit 0.1.7 documentation also mentions that the required input data shape is the transposed of what MNE produces

1 Like
14

Yes, so the result of dss.dss_line() must be transposed again before using it with mne.io.RawArray.

3 Likes