noise covariance matrix

Dear MNE users,

I would very much appreciate your advice on the results I am getting from calculation of the noise covariance matrix. I'm using the "method" option for mne.compute_covariance to do automated regularisation.

Data were collected with Neuromag 306 Vectorview system. My responses are time locked to the onset of the average of ~300 auditorily-presented sentences. I have used the silent (baseline) period -500-0ms before sentence-onset as the time period from which to estimate the noise.

Hi Lucy,

you seem to have some channels with very big drifts. That's why you see
so huge GFP values. If it's acceptable for your type of question you could
high pass a bit to fix this.

HTH
Alex

Dear MNE users,

I would very much appreciate your advice on the results I am getting from
calculation of the noise covariance matrix. I?m using the ?method? option
for mne.compute_covariance to do automated regularisation.

Data were collected with Neuromag 306 Vectorview system. My responses are
time locked to the onset of the average of ~300 auditorily-presented
sentences. I have used the silent (baseline) period -500-0ms before
sentence-onset as the time period from which to estimate the noise.

##################

event_id = None

tmin, tmax = -0.5, 5.5

reject_tmin, reject_tmax = -0.5, 1.5

bmin, bmax = -0.5, 0

epochs = mne.Epochs(raw, events, event_id, tmin, tmax, reject_tmin =
reject_tmin, reject_tmax = reject_tmax, picks=picks, baseline=baseline,
reject=reject, preload=True, add_eeg_ref=True)

noise_cov = mne.compute_covariance(epochs, method =(?shrunk?,
?empirical?), tmin=bmin, tmax=bmax, return_estimators = True)

###################

The plot below is for a single subject (but all my subjects show
similar-looking output) for a period -500 to 5000ms covering the duration
of my sentences.

I have compared my output with that for the examples:

http://martinos.org/mne/stable/auto_examples/visualization/plot_evoked_
whitening.html#sphx-glr-auto-examples-visualization-plot-
evoked-whitening-py

http://martinos.org/mne/stable/auto_tutorials/plot_compute_covariance.html

The result tells me that ?shrunk? is the best method, but from looking at
the output from whitening I?m unsure how this is the case, and in fact
whether either method is working as it should.

*Evoked signals for all channels:*

For the MEG, during the baseline the values are generally within the +/-
1.96 indicated by the red dotted line, so I think this is OK.

Data look quite noisy

*GFP plots for MEG:*

For ?empirical?, the baseline values > 1 whereas for ?shrunk, the baseline
values <1. As I understand it values should be around 1 and therefore both
methods look problematic.

My question is therefore:

*when the baseline GFP is > or < 1 then is this due to problems with
regularisation and where should I go from here?*

With thanks for your thoughts and advice.

Kind regards,

Lucy MacGregor

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170406/2a56e9af/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 37936 bytes
Desc: not available
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170406/2a56e9af/attachment-0001.jpg

Hi Alex,

Many thanks for the suggestion.

I have tried again, now with a HPF of 1Hz (previously was 0.1Hz). The EEG look better (although I realise I have some noisy channels in there). For the MEG the GFP looks much better for empirical, with values around 1. However, for ?shrunk? the values are below 1. But ?shrunk? is chosen as the best method.

Please do you have an explanation for why the GPF<1 for the ?shrunk? method?
Do you have a suggestion as to what I could do (e.g. just choose ?empirical? or make some other changes)?

[cid:image004.jpg at 01D2AF81.F325ABE0]

Many thanks for your advice.

Best wishes,

Lucy

Hi Lucy,

your plots show that you're using SSS. We recently saw some rank estimation
issues in some case at the level of the covariance matrix but not yet fully
understand the situation.
Can I ask you to plot the covariance as follows:

cov.plot(evoked.info)

then you could try to plot the whitening again, this time using our non
public maintenance function.

meg_picks = mne.pick_types(meg=True, eeg=False)
sss_rank = raw.estimate_rank(picks=meg_picks)
mne.viz.evoked._plot_white(evoked, noise_cov, rank=sss_rank)

As a sanity check, the ```sss_rank``` should correspond to the prominent
kink in your eigenvalue spectrum of the covariance.

If the display improves your file also belongs to the mysterious group of
SSS'ed files for which our defaults aren't optimized.

I'm currently working on an improvement of our covariance computation with
regard to the data rank. If you are happy to share an epochs or raw file
with me privately I could use that as another test file for development.

Best,
Denis

Hi Alex,

Many thanks for the suggestion.

I have tried again, now with a HPF of 1Hz (previously was 0.1Hz). The EEG
look better (although I realise I have some noisy channels in there). For
the MEG the GFP looks much better for empirical, with values around 1.
However, for ?shrunk? the values are below 1. But ?shrunk? is chosen as the
best method.

Please do you have an explanation for why the GPF<1 for the ?shrunk?
method?

Do you have a suggestion as to what I could do (e.g. just choose
?empirical? or make some other changes)?

[image: image004.jpg]

Many thanks for your advice.

Best wishes,

Lucy

*From:* mne_analysis-bounces at nmr.mgh.harvard.edu [mailto:
mne_analysis-bounces at nmr.mgh.harvard.edu] *On Behalf Of *Alexandre
Gramfort
*Sent:* 06 April 2017 14:35
*To:* Discussion and support forum for the users of MNE Software
*Subject:* Re: [Mne_analysis] noise covariance matrix

Hi Lucy,

you seem to have some channels with very big drifts. That's why you see

so huge GFP values. If it's acceptable for your type of question you could

high pass a bit to fix this.

HTH

Alex

Dear MNE users,

I would very much appreciate your advice on the results I am getting from
calculation of the noise covariance matrix. I?m using the ?method? option
for mne.compute_covariance to do automated regularisation.

Data were collected with Neuromag 306 Vectorview system. My responses are
time locked to the onset of the average of ~300 auditorily-presented
sentences. I have used the silent (baseline) period -500-0ms before
sentence-onset as the time period from which to estimate the noise.

##################

event_id = None

tmin, tmax = -0.5, 5.5

reject_tmin, reject_tmax = -0.5, 1.5

bmin, bmax = -0.5, 0

epochs = mne.Epochs(raw, events, event_id, tmin, tmax, reject_tmin =
reject_tmin, reject_tmax = reject_tmax, picks=picks, baseline=baseline,
reject=reject, preload=True, add_eeg_ref=True)

noise_cov = mne.compute_covariance(epochs, method =(?shrunk?,
?empirical?), tmin=bmin, tmax=bmax, return_estimators = True)

###################

The plot below is for a single subject (but all my subjects show
similar-looking output) for a period -500 to 5000ms covering the duration
of my sentences.

I have compared my output with that for the examples:

Page Redirection

http://martinos.org/mne/stable/auto_tutorials/plot_compute_covariance.html

The result tells me that ?shrunk? is the best method, but from looking at
the output from whitening I?m unsure how this is the case, and in fact
whether either method is working as it should.

[image: image003.jpg]

*Evoked signals for all channels:*

For the MEG, during the baseline the values are generally within the +/-
1.96 indicated by the red dotted line, so I think this is OK.

Data look quite noisy

*GFP plots for MEG:*

For ?empirical?, the baseline values > 1 whereas for ?shrunk, the baseline
values <1. As I understand it values should be around 1 and therefore both
methods look problematic.

My question is therefore:

*when the baseline GFP is > or < 1 then is this due to problems with
regularisation and where should I go from here?*

With thanks for your thoughts and advice.

Kind regards,

Lucy MacGregor

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170408/1bb4ce51/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 37936 bytes
Desc: not available
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170408/1bb4ce51/attachment-0002.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 37510 bytes
Desc: not available
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170408/1bb4ce51/attachment-0003.jpg

Hi Denis,

Thank you for your help.

I did the following as you suggested, but the whitening visualisation does not seem to change ? the GFP is still <1 for the MEG data.

Noise_cov.plot(evoked.info)

Meg_picks = mne.pick_types(raw.info, meg =True, eeg=False)
Sss_rank = raw.estimate_rank(picks = meg_picks)
Rank = sss_rank
Mne.viz.evoked._plot_evoked_white(evoked, noise_cov[0], rank)

[cid:image004.jpg at 01D2B1F0.AA197C30]

Although the ?shrunk? method seems to be still problematic, is there any reason to suppose there are problems with the ?empirical? method for SSS-ed data? In other words, could I go ahead with ?empirical? method (I think it?s the standard?) or would I be better not using automated regularisation?

I would happily share a data file, but the MRC has a strict policy on this, so I just need to check.

Thank you.

Best wishes,

Lucy

Hi,

Something seems funny about the covariance spectra. The kink seems
different for magnetometers and gradiometers, which after SSS is
unexpected. What is the value of your estimated rank? Are you using
movement compensation? Could you also tell me a bit about the type of data
(how many samples, do you use multiple runs)?
I am asking because movement compensation can artificially inflate the seen
rank. Another thing would be to read out the expected rank from the
measurement info. Have a look at mne.io.proc_history._get_sss_rank, for
which I am currently working on adding a public version
WIP: add public SSS rank getter by dengemann · Pull Request #4177 · mne-tools/mne-python · GitHub.
Probably you are fine with empirical, if it looks the same when using the
public plot_white method. However, this may just be the case for this
dataset. The higher cross-validated negative log likelihood suggests that
the shrunk covariance solution simply generalized better. At this point we
do not know at which level the problem is situated, the estimation or the
application / graphical display. But it is clear that EEG is well treated
and some problems are related to SSS.
If you can find a way to share one of these datasets it would help us a lot
to understand how to ideally treat SSSed data.

Denis

Hi Denis,

Thank you for your help.

I did the following as you suggested, but the whitening visualisation does
not seem to change ? the GFP is still <1 for the MEG data.

Noise_cov.plot(evoked.info)

Meg_picks = mne.pick_types(raw.info, meg =True, eeg=False)

Sss_rank = raw.estimate_rank(picks = meg_picks)

Rank = sss_rank

Mne.viz.evoked._plot_evoked_white(evoked, noise_cov[0], rank)

Although the ?shrunk? method seems to be still problematic, is there any
reason to suppose there are problems with the ?empirical? method for SSS-ed
data? In other words, could I go ahead with ?empirical? method (I think
it?s the standard?) or would I be better not using automated regularisation?

I would happily share a data file, but the MRC has a strict policy on
this, so I just need to check.

Thank you.

Best wishes,

Lucy

*From:* mne_analysis-bounces at nmr.mgh.harvard.edu [mailto:
mne_analysis-bounces at nmr.mgh.harvard.edu] *On Behalf Of *Denis-Alexander
Engemann
*Sent:* 08 April 2017 15:09

*To:* Discussion and support forum for the users of MNE Software
*Subject:* Re: [Mne_analysis] noise covariance matrix

Hi Lucy,

your plots show that you're using SSS. We recently saw some rank
estimation issues in some case at the level of the covariance matrix but
not yet fully understand the situation.

Can I ask you to plot the covariance as follows:


cov.plot(evoked.info)

then you could try to plot the whitening again, this time using our non
public maintenance function.


meg_picks = mne.pick_types(meg=True, eeg=False)

sss_rank = raw.estimate_rank(picks=meg_picks)

mne.viz.evoked._plot_white(evoked, noise_cov, rank=sss_rank)

As a sanity check, the ```sss_rank``` should correspond to the prominent
kink in your eigenvalue spectrum of the covariance.

If the display improves your file also belongs to the mysterious group of
SSS'ed files for which our defaults aren't optimized.

I'm currently working on an improvement of our covariance computation with
regard to the data rank. If you are happy to share an epochs or raw file
with me privately I could use that as another test file for development.

Best,

Denis

Hi Alex,

Many thanks for the suggestion.

I have tried again, now with a HPF of 1Hz (previously was 0.1Hz). The EEG
look better (although I realise I have some noisy channels in there). For
the MEG the GFP looks much better for empirical, with values around 1.
However, for ?shrunk? the values are below 1. But ?shrunk? is chosen as the
best method.

Please do you have an explanation for why the GPF<1 for the ?shrunk?
method?

Do you have a suggestion as to what I could do (e.g. just choose
?empirical? or make some other changes)?

[image: image004.jpg]

Many thanks for your advice.

Best wishes,

Lucy

*From:* mne_analysis-bounces at nmr.mgh.harvard.edu [mailto:
mne_analysis-bounces at nmr.mgh.harvard.edu] *On Behalf Of *Alexandre
Gramfort
*Sent:* 06 April 2017 14:35
*To:* Discussion and support forum for the users of MNE Software
*Subject:* Re: [Mne_analysis] noise covariance matrix

Hi Lucy,

you seem to have some channels with very big drifts. That's why you see

so huge GFP values. If it's acceptable for your type of question you could

high pass a bit to fix this.

HTH

Alex

Dear MNE users,

I would very much appreciate your advice on the results I am getting from
calculation of the noise covariance matrix. I?m using the ?method? option
for mne.compute_covariance to do automated regularisation.

Data were collected with Neuromag 306 Vectorview system. My responses are
time locked to the onset of the average of ~300 auditorily-presented
sentences. I have used the silent (baseline) period -500-0ms before
sentence-onset as the time period from which to estimate the noise.

##################

event_id = None

tmin, tmax = -0.5, 5.5

reject_tmin, reject_tmax = -0.5, 1.5

bmin, bmax = -0.5, 0

epochs = mne.Epochs(raw, events, event_id, tmin, tmax, reject_tmin =
reject_tmin, reject_tmax = reject_tmax, picks=picks, baseline=baseline,
reject=reject, preload=True, add_eeg_ref=True)

noise_cov = mne.compute_covariance(epochs, method =(?shrunk?,
?empirical?), tmin=bmin, tmax=bmax, return_estimators = True)

###################

The plot below is for a single subject (but all my subjects show
similar-looking output) for a period -500 to 5000ms covering the duration
of my sentences.

I have compared my output with that for the examples:

Page Redirection

http://martinos.org/mne/stable/auto_tutorials/plot_compute_covariance.html

The result tells me that ?shrunk? is the best method, but from looking at
the output from whitening I?m unsure how this is the case, and in fact
whether either method is working as it should.

[image: image003.jpg]

*Evoked signals for all channels:*

For the MEG, during the baseline the values are generally within the +/-
1.96 indicated by the red dotted line, so I think this is OK.

Data look quite noisy

*GFP plots for MEG:*

For ?empirical?, the baseline values > 1 whereas for ?shrunk, the baseline
values <1. As I understand it values should be around 1 and therefore both
methods look problematic.

My question is therefore:

*when the baseline GFP is > or < 1 then is this due to problems with
regularisation and where should I go from here?*

With thanks for your thoughts and advice.

Kind regards,

Lucy MacGregor

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170410/6011fd84/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 37510 bytes
Desc: not available
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170410/6011fd84/attachment-0004.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 37936 bytes
Desc: not available
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170410/6011fd84/attachment-0005.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 80893 bytes
Desc: not available
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170410/6011fd84/attachment-0006.jpg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 37510 bytes
Desc: not available
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20170410/6011fd84/attachment-0007.jpg

Hi,

Yes, I see what you mean about the discrepancy between gradiometers and magnetometers.

Estimated rank = 69.

I had four runs per subject.
In Maxfilter I did movement compensation, down sampled from 1000Hz to 250 Hz and trans-ed data to the first recording run.
In MNE Python I then concatenated the four blocks of data for further processing.

Thanks,

Lucy