Discussion: robust, linear and polynomial detrending

External Email - Use Caution

Dear MNE community,

I would like to discuss the following issues:

Data background: Our data (MEG-EEG) is often composed with slow drifts
(mostly due to street artifacts). We are interested in finding out cortical
activity in lower frequency states, therefor a high pass filter option
(>.5-1hz) is not a optimal solution for us since it removes most of the
signals that we are interest in.

Some of the highlighted points:
1. Current high filter setting has set to (.1 hz as lower cutoff), we can
see the spatial map of the drift components in the ICA decomposition but
ICA decomposition for other artifacts (eye blinks/cardiac) is not stable
which we already expect at this point. Removing the drift components
directly from ICA and then rerun the ICA for the second time to remove
other biological artifacts sound suspicious to me. I am not an expert of
ICA decomposition but as far I know, removing PCA components will alter the
linear projection of the sources. How valid is the idea of ruining ICA for
two times?

2. MNE offers a dc and a liner detrending option to remove show drifts at
the epochs/evoked level. But often I have seen linear detrending removes
activity that is coming from cortical sources (i.e., not a optimal fit for
our data), in fact a lower order polynomial fit is actually more
reasonable. A recent work by de Cheveign? A
<https://www.ncbi.nlm.nih.gov/pubmed/?term=de%20Cheveigné%20A[Author]&cauthor=true&cauthor_uid=29448077>
(https://www.ncbi.nlm.nih.gov/pubmed/29448077) [method: weighted robust
detrending] is a nice idea to tackle this kind of issue. The algorithm has
been adapted in 'MEEGKIT' package [
https://nbara.github.io/python-meegkit/auto_examples/example_detrend.html#sphx-glr-auto-examples-example-detrend-py]
but lately I have figured it out that it still needs more testing. Now,
coming back to my question: Is there any way to implement such algorithm
directly at the raw data? So far I am not able to modified the raw mne
structure with synthetic values (after removing the trend with a polynomial
fit) while keeping the same info directory. Any advice or suggestion would
be helpful at this stage.

3. Is there any way to apply a linear detrending only to a sub set of the
data (either meg/eeg even though they are combined at epochs level) ?, so
far its implemented in evoked level as I have seen. It would be ideal that
if I can somehow apply the linear detrending only to a subset of the data
at the epochs level (i.e., meg in my case) instead of doing it at evoked
level. I am willing to contribute in this case.

Best,
Dip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20200224/6299a59e/attachment-0001.html

External Email - Use Caution

Dear Dip,

It is not that hard to create Raw and Epochs data structures with custom
data. You do this using the RawArray and EpochsArray classes, see here:

https://mne.tools/stable/auto_tutorials/simulation/plot_creating_data_structures.html

Note that you can just re-use the .info from the original raw and can
skip the parts on how to create a custom Info object.

Once you get comfortable creating these datastructures, you can explore
different ways to de-trend the data as you please, with the usual
numpy/scipy functions.

best,
Marijn.

External Email - Use Caution

Thanks, Marijn. It was really helpful.