I have a iEEG data (ECoG) to preprocess, and I used MNE to get some results, but wanted to ask a few questions as this is my first time preprocessing electrophysiology data (I used to do MRI)
It seems that the application of notch filters is not enough to remove the power line interference. As the picture below shows that by applying notch filters, 1. the higher frequencies (200Hz or more) actually get removed too aggressively, 2. the interference at 60Hz is still persistent. (By the way, you can ignore the black and red lines, as they are just EKG signals) Is this normal? It seems that people can modify the notch filter width, should I do this?
What is the optimal/minimal preprocessing that MUST be done to the data? As my goal is to create a deep learning model using large amounts of data, I feel that applying as little preprocessing as possible and keeping as much data as possible is the best approach (even at the cost of not removing some noise). Are there “critical” noise that is 99% noise and can be removed? For example, 1. should I do high pass filtering instead of band-pass filtering (to keep the high frequency information)? 2. should I set the high-pass filter to be at 0.1 instead of 0.5 to preserve the data at 0.1~0.5Hz? Any input would be highly appreciated!
(notice that the picture below shows both the “original data” and "high pass filter at 0.5 Hz with notch filters at multiples of 60)
The notch output looks fine to me. It does what it should be.
Now it seems you have an aliasing issue in data. The line noise wraps around the 1000Hz boundary. It is possible no low pass filter was down in a resampling step in your pipeline?
@agramfort
Thank you for your response! Oh I thought that there shouldn’t be any spikes at the 60Hz thing at all when I apply a notch filter, but I guess this is good enough?
Also, I’m sorry I put the wrong code. h_freq = None was done!
I have a few questions if you don’t mind :
Does the physical values of the signal matter? Or is it OK to normalize them on a channel-by-channel basis? : it seems that unlike EEG, iEEG data varies widely in the mean and standard deviation between the channels. I guess the mean value doesn’t have much meaning since we will rereference it, but I am not the sure about the standard deviation. What is your opinion on this? Does the actual physical (uV) values of the channels matter?
I heard that if I want to resample EEG with a sampling rate of 256Hz to 250Hz, I need to low pass filter it with the Nyquist frequency before resampling to prevent aliasing. If I use both 250Hz and 256Hz data, do you think that I should (a) : resample the 256Hz data to 250Hz even if it means 250Hz data would be low pass filtered whereas 250Hz isn’t (a) don’t resample anything, as 250Hz and 256Hz is only like 2.4% different, (i.e. not that large) so that I can keep the high frequency components of all frequencies
ah wait I looked too fast. The first peak as 60Hz… This is very surprising to me as line noise is typically very narrow band. Here it seems to span over more than 10Hz. Something happened during acquisition or with some preprocessing you did.
regarding your questions it depends on what you want to do. What effect are you looking for?
@agramfort Hi, thank you so much for your response!
I looked at some different sessions of data and found that the line noise is much more narrower (as you said it should). The data is an ECoG of a subject collected over multiple days, so I guess that parts of the data have some weird characteristics? (I believe no preprocessing at all was done to this file).
Below I am attaching three PSD plots of the different session, in the following order : 1. original data 2. 0.5 highpass filter and notch filters at mulitples of 60 (as before), and 3. same as 2 but with notch filter width of 3.
First picture : the line noise at 60Hz in this data is much narrower
Second picture : the notch filter seems to work well (especially at 60Hz), but still some residuals seem to remain at 60Hz
Third picture : increasing the notch width to 3 seems to work better at removing the 60Hz noise?
Here are the questions regarding notches
Do you think that this is now good?
Is it normal to adjust the default notch width in mne to something else? The lab that gave me this data said they use a manual way to get the widths (matlab code below). What are your thoughts on creating an equivalent of their code on MNE (I tried but was unsuccessful)? Do you think that using the default MNE method would differ a lot from using the filtering method below?
Fs = 2000; #! sampling rate
E1_detrend = eegfilt(E1_Hippo, Fs, 0.5, 0); E1_detrend = eegfilt(E1_detrend, Fs, 0, 200); E1_detrend = NotchFilter_ryun(E1_detrend, Fs, 60); E1_detrend = NotchFilter_ryun(E1_detrend, Fs, 120); E1_detrend = NotchFilter_ryun(E1_detrend, Fs, 180);
#where NotchFilter ryun function is
wo = fq/(srate/2); bw = wo/Qfactor;
[b,a] = iirnotch(wo,bw);
filtData = filtfilt(b,a,Data);
#they say that for fq, multiples of 60 are used, and the default value Qfactor is 35.
I am trying to create an EEG + iEEG foundation model that can handle various aspects (ex : ERP’s temporal and STFT’s spectral components) of the data by self-supervision.
Therefore, in order to give the model as much information as possible, my belief is that I should do the bare minimum of preprocessing and let the model decide for itself in a data-driven manner. (For example, I don’t believe that using ICA to denoise would be optimal since using ICA could be viewed as injecting human “bias” in the data).
In this context, how do you think I should handle the issue of “preserving the physical value of the signal” and “low pass filtering to do resampling from 256 to 250Hz”?
Sorry for asking too much, but in this regard, there is also in issue of rereferencing. The CAR method that they use seem like it would introduce unwanted biases during preprocessing as in iEEG subjects have different number and placement of electrodes. Is there another referencing method that one can use or you recommend?
You could also try using zapline or its iterative variant to remove the line noise, but I am not sure whether / how much this pre-processing is “preserving the original signal”.
