I am currently pre-processing eeg data. Before running the ICA, I would like to remove some of the epochs with sudden jumps or very high values in voltage. I saw that in Fieldtrip, this is implemented with the function ft_rejectartifact and FT_ARTIFACT_JUMP. However, I was not able to find something equivalent in mne. Do you have any suggestion?
I have already tried by using autoreject. However, I struggled to properly understand how the function works. It is not clear to me whether having a k higher than rho means that some bad epochs will remain in the data, because only the worst electrodes are interpolated. Why is this so? Ideally one would not keep bad segments of data but rather drop them, even if they could be partially rescued for the worst electrodes…Am I understanding this correctly or I am completely mistaken?
Also, when applying autoreject, I realized that by default the function allows for a maximum number of bad electrodes equal to 32, which for me is really high (I used a cap with 64 electrodes). I therefore reduced the k to 5, but I also used a rho of 5/64 - Ideally, I don’t want to keep any non interpolated epoch if this is bad. However, as a result, I get a very high number of rejected epochs (30%), even when the data looks in principle pretty good. Could you help me to better understand this and tell me if there is another way to check for high voltage values/jumps in the signal?
My pipeline looks like this:
detect bad electrodes with RANSAC
exclude bad electrodes from autoreject (for now)
run ICA
exclude EMG noise with a function similar to the one used in fieldtrip (z threshold and bandpass between 110 to 140)
interpolation of bad electrodes
Many thanks in advance for any help you might give me!
Hello @Catsine, what you’re looking for is what the reject parameter of Epochs is for It allows you to specify limits as to which signal amplitudes are still acceptable in an epoch or, if the amplitude exceeds the limit, when to remove the epoch.
If you do this before creating epochs, the segments marked as “amplitude to high” or “amplitude too low” will be skipped when you epoch the data.
But the most common way to reject problematic epochs before ICA is using the reject parameter I mentioned above.
Many thanks @richard! I had used the reject parameter, however, since it drops epochs also when there is just one electrode that is bad, I was wondering whether it is instead possible to interpolate those epochs where just one electrode or a few misbehave.
This is why I wanted to use autoreject, but I wonder why so many epochs are dropped if I reduce the n_interpolates to 5 and consensus to 5/64, which I thiught it was a reasonable number if you have 64 electrodes. Do you happen to know of other cases where autoreject is maybe too conservative? Am I being too strict with the number of electrodes that I can interpolate?
Interpolation on a per-epoch basis is currently not possible in MNE, and autoreject in fact just hacks around these limitations as far as I know
But if a single electrode is causing trouble in many epochs, I would just mark the entire channel as bad and interpolate it across all time points.
Also a general advice is to set the rejection threshold quite high so that you don’t end up rejecting epochs containing blinks before running ICA – after all, you’re using ICA to salvage these epochs!
@mainakjas is the creator of autoreject and might be able to give you some advice here!
Yes, with jump I mean a sudden (abnormal) change in the signal, perhaps due to the pulling of an electrode (my participants had to reach towards a touch screen, so at times some cables could be pulled during the movement) or some other forms of noise that lead to very high voltage in some cases…
Not sure if suggesting something outside MNE is verboten, but…
The clean_rawdata plugin in EEGLAB does a great job of this. Part of what it does is to perform artifact subspace reconstruction on the data. Essentially, it runs a short, moving window PCA across the data, removes components with artifacts, then reconstructs the signal without the noise. I probably couldn’t have completed my study without it, as there were so many movement artifacts in my data (people using a prosthesis to perform a task).
See Makoto’s pipeline.
For my study, I used ASR in two stages (as Makoto suggests), the first run, I used it to reject bad channels. Then rereference to average, then run ASR again, this time to only repair noisy regions in the data. Rereference to average again, and you’re ready to run ICA.
Note that when running ICA, you’ll probably want to specify the number of components as the number of channels you had before interpolation. For example, if you start with 58 channels, 2 are removed (leaving 56), then interpolate back to 58, you would specify the rank as 56 when running ICA. This is a little complicated, but is because removing channels removes information, but interpolation doesn’t add any information. It is only creating data based on what is there.
I have already tried by using autoreject. However, I struggled to properly understand how the function works. It is not clear to me whether having a k higher than rho means that some bad epochs will remain in the data, because only the worst electrodes are interpolated. Why is this so?
K higher than rho does not necessarily mean that some bad epochs will remain in the data. If you have less than rho bad channels in an epoch, all of them will be interpolated. If you have more than rho but less than k, then the rho worse channels will be interpolated. The “bad” channels that are not interpolated are likely to have minimal impact on the average data since we already interpolated the worse ones. Remember that the goal of autoreject is to improve data quality (measured by some metric), not remove every epoch with any artifactual channel. In fact, doing this will be worse since you’re removing good data from many channels.
Ideally one would not keep bad segments of data but rather drop them, even if they could be partially rescued for the worst electrodes…Am I understanding this correctly or I am completely mistaken?
How would you do this in MNE? MNE expects equal number of epochs per channel. Fieldtrip uses Nanmean for instance … the equivalent solution in MNE is to interpolate. But you can’t interpolate if too many channels are bad …
Also, when applying autoreject, I realized that by default the function allows for a maximum number of bad electrodes equal to 32, which for me is really high (I used a cap with 64 electrodes). I therefore reduced the k to 5, but I also used a rho of 5/64 - Ideally, I don’t want to keep any non interpolated epoch if this is bad. However, as a result, I get a very high number of rejected epochs (30%), even when the data looks in principle pretty good.
It allows you to specify limits as to which signal amplitudes are still acceptable in an epoch or, if the amplitude exceeds the limit, when to remove the epoch.
That’s what I would call it
