on_missing and time-frequency

system · October 1, 2014, 8:48am

Hello,

2 issues:

1. Using mne.Epochs with on_missing='ignore' continues to return an error -
and crashes the script.

The error is returned irrespective of the argument I put in. It doesn't
work in ipython or in spyder albeit others report it works in ipython but
not in spyder.

2. I was giving a quick shot at TF analysis in sensor space using
"tfr_morlet" - although it runs fine, the returned power and itc are not in
readable format or am I missing someting? e.g. "power.plot_topo" is not
recognized. This was run in spyder with updated mne-python.

Any help appreciated!

Best,
-V

system · October 1, 2014, 10:58am

Hi Virginie,

2014-10-01 10:48 GMT+02:00 Virginie van Wassenhove <
virginie.van.wassenhove at gmail.com>:

Hello,

2 issues:

1. Using mne.Epochs with on_missing='ignore' continues to return an error
- and crashes the script.

thanks for reporting, we'll file a bug report and inquire.

The error is returned irrespective of the argument I put in. It doesn't
work in ipython or in spyder albeit others report it works in ipython but
not in spyder.

2. I was giving a quick shot at TF analysis in sensor space using
"tfr_morlet" - although it runs fine, the returned power and itc are not in
readable format or am I missing someting? e.g. "power.plot_topo" is not
recognized. This was run in spyder with updated mne-python.

In which sense is the format not readable, how does this look like?
What does it mean that "power.plot_topo" is not recognized?

-Denis

Any help appreciated!

Best,
-V

--
Virginie van Wassenhove
Exec Dir NeuroSpin MEG
Group leader, Brain Dynamics
CEA.DSV.I2BM.NeuroSpin - INSERM Cognitive Neuroimaging Unit
B?t 145 Point Courrier 156
Gif s/ Yvette F-91191 FRANCE

+33(0)1.69.08.1667
Virginie.van.Wassenhove at gmail.com
https://sites.google.com/site/virginievanwassenhove/

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20141001/f0fab1f9/attachment.html

system · October 1, 2014, 11:06am

One more thing. When you construct the Epochs with on_missing='ignore' and
you have a crash, how does the error message look like? Can you rule out
that for some reason None of the events in your event_id is found in the
third column of `events`. In that case ignoring would not help.

-D

2014-10-01 12:58 GMT+02:00 Denis-Alexander Engemann <
denis.engemann at gmail.com>:

Hi Virginie,

2014-10-01 10:48 GMT+02:00 Virginie van Wassenhove <
virginie.van.wassenhove at gmail.com>:

Hello,

2 issues:

1. Using mne.Epochs with on_missing='ignore' continues to return an error
- and crashes the script.

thanks for reporting, we'll file a bug report and inquire.

The error is returned irrespective of the argument I put in. It doesn't
work in ipython or in spyder albeit others report it works in ipython but
not in spyder.

2. I was giving a quick shot at TF analysis in sensor space using
"tfr_morlet" - although it runs fine, the returned power and itc are not in
readable format or am I missing someting? e.g. "power.plot_topo" is not
recognized. This was run in spyder with updated mne-python.

In which sense is the format not readable, how does this look like?
What does it mean that "power.plot_topo" is not recognized?

-Denis

Any help appreciated!

Best,
-V

--
Virginie van Wassenhove
Exec Dir NeuroSpin MEG
Group leader, Brain Dynamics
CEA.DSV.I2BM.NeuroSpin - INSERM Cognitive Neuroimaging Unit
B?t 145 Point Courrier 156
Gif s/ Yvette F-91191 FRANCE

+33(0)1.69.08.1667
Virginie.van.Wassenhove at gmail.com
https://sites.google.com/site/virginievanwassenhove/

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you
in error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20141001/422d3ba6/attachment.html

system · October 1, 2014, 12:01pm

Yes, I know in advance that is in some conditions and in some runs, no
events will be found which is the reason why I want to use the "ignore"
option when that happens. Am I understanding this option incorrectly? What
is the "on_missing" referring to o.w.?

the error message is :
"ValueError: no desired events found."

Thanks!
V

system · October 1, 2014, 12:05pm

2. I was giving a quick shot at TF analysis in sensor space using
"tfr_morlet" - although it runs fine, the returned power and itc are not in
readable format or am I missing someting? e.g. "power.plot_topo" is not
recognized. This was run in spyder with updated mne-python.

In which sense is the format not readable, how does this look like?

What does it mean that "power.plot_topo" is not recognized?

I had to close that console, lost the 2 hours computations and I can't tell
you the exact "format" value it was reporting when calling the apparently
non-existent "power" variable despite having computed it for 2 hours. I'll
try that again and be more specific.

Is it possible that "power._getdata()" need to be used to actually load the
results of tfr_morlet in a spyder console?

for "power.pot_topo", I need to recompute and I'll tell you.

Later, thanks!
V

system · October 1, 2014, 12:18pm

Hi Virginie

1. this is in fact an issue with the documentation, not a bug. The 'ignore'
option will avoid crashes if one or many events are missing but at least
one remains. If all events are missing nothing will be ignored, currently
the constructor needs at east one event. We might want to rethink the
semantics. Out of curiosity, what are you doing with epochs for which you
know in advance that events and ids don't match? In other words, would
there be a use case for the behavior you expected from 'ignore'? If it's
just to avoid crashes when no event is found, I would manually check that
and skip computations if nothing is found. For example (in a loop that
iterates over datasets or subjects):

if not np.any(np.in1d(event_ids.values(), events[:, 2])):
continue

2. the second issue sounds mysterious to me, I don't `smell` what might be
going on there.

2014-10-01 14:05 GMT+02:00 Virginie van Wassenhove <
virginie.van.wassenhove at gmail.com>:

2. I was giving a quick shot at TF analysis in sensor space using
"tfr_morlet" - although it runs fine, the returned power and itc are not in
readable format or am I missing someting? e.g. "power.plot_topo" is not
recognized. This was run in spyder with updated mne-python.

In which sense is the format not readable, how does this look like?

What does it mean that "power.plot_topo" is not recognized?

I had to close that console, lost the 2 hours computations and I can't
tell you the exact "format" value it was reporting when calling the
apparently non-existent "power" variable despite having computed it for 2
hours. I'll try that again and be more specific.

So you imply that power = tfr_morlet completed (after 2 hours?) but that
`power` was not found?

Is it possible that "power._getdata()" need to be used to actually load
the results of tfr_morlet in a spyder console?

This should not be necessay, this one is private and should be ignored for
regular usage.

-Denis

for "power.pot_topo", I need to recompute and I'll tell you.

Later, thanks!
V

-Denis

Any help appreciated!

Best,
-V

--
Virginie van Wassenhove
Exec Dir NeuroSpin MEG
Group leader, Brain Dynamics
CEA.DSV.I2BM.NeuroSpin - INSERM Cognitive Neuroimaging Unit
B?t 145 Point Courrier 156
Gif s/ Yvette F-91191 FRANCE

+33(0)1.69.08.1667
Virginie.van.Wassenhove at gmail.com
https://sites.google.com/site/virginievanwassenhove/

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom
it is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you
in error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you
in error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

--
Virginie van Wassenhove
Exec Dir NeuroSpin MEG
Group leader, Brain Dynamics
CEA.DSV.I2BM.NeuroSpin - INSERM Cognitive Neuroimaging Unit
B?t 145 Point Courrier 156
Gif s/ Yvette F-91191 FRANCE

+33(0)1.69.08.1667
Virginie.van.Wassenhove at gmail.com
https://sites.google.com/site/virginievanwassenhove/

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20141001/b27f8f93/attachment.html

system · October 2, 2014, 7:11am

1. this is in fact an issue with the documentation, not a bug. The
'ignore' option will avoid crashes if one or many events are missing but at
least one remains. If all events are missing nothing will be ignored,
currently the constructor needs at east one event. We might want to rethink
the semantics. Out of curiosity, what are you doing with epochs for which
you know in advance that events and ids don't match? In other words, would
there be a use case for the behavior you expected from 'ignore'? If it's
just to avoid crashes when no event is found, I would manually check that
and skip computations if nothing is found. For example (in a loop that
iterates over datasets or subjects):

if not np.any(np.in1d(event_ids.values(), events[:, 2])):
continue

The reason is purely pragmatic for batch processing (all runs of all
participants; in some runs there may be some conditions missing due to the
nature of the task)

2. the second issue sounds mysterious to me, I don't `smell` what might be
going on there.

I might have found the bug.
-V

system · October 2, 2014, 8:46am

The reason is purely pragmatic for batch processing (all runs of all
participants; in some runs there may be some conditions missing due to the
nature of the task)

either we have it crash at epochs creation or it will crash later.

I think you should detect it and address the problem
on your side when running your batch script.

ALex

system · October 2, 2014, 12:52pm

Ok thanks. You may want to clarify the "no_missing" option on the wiki.
Cheers,
V

system · October 2, 2014, 1:20pm

ok bug fixed, tfr_morlet works great along with the superb figure suite
tools!
very nifty!
-V

quick test outcome on 1 participant: some lovely parietal alpha and low
gamma
40 min with n_jobs= 4, 2s epoch, decimfactor=3 (333 Hz)
[image: Inline image 1]

system · October 2, 2014, 4:30pm

nice to hear it works

how would you phrase the doc? It is currently:

    on_missing : str
        What to do if an event id is not found in the recording.
        Valid keys are 'error' | 'warning' | 'ignore'
        Default is 'error'. If on_missing is 'warning' it will proceed but
        warn, if 'ignore' it will proceed silently.

Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20141002/e38ecb57/attachment.html

system · October 2, 2014, 8:42pm

simply specify that "if no event id are found in the data, an error will be
automatically generated irrespective of the str"

however, i am still unclear about what "no_missing" means: how would
mne.epochs know in advance whether an event should be there or not (hence
that it is missing)?
-V

system · October 3, 2014, 8:59am

Thanks Virginie,

I opened a PR, which is based on your suggestion.

2014-10-02 22:42 GMT+02:00 Virginie van Wassenhove <
virginie.van.wassenhove at gmail.com>:

simply specify that "if no event id are found in the data, an error will
be automatically generated irrespective of the str"

however, i am still unclear about what "no_missing" means: how would
mne.epochs know in advance whether an event should be there or not (hence
that it is missing)?

The mismatch is produced as follows:

You pass an event_id that will be a dict in most cases, e.g. {'left': 1,
'right':2}.

Second you pass events at which to segment the continuous recordings.
For simplicity let's assume we only have two of them.

np.array([[1, 0, 14], [1, 0, 2]])

In this case the event id 'left' is not found in the events (not in the
data) and 'on_missing' becomes relevant.

For the sourcecode see:

github.com

mne-tools/mne-python/blob/main/mne/epochs.py#L708


      
              if hasattr(self, "events"):
                  assert len(self.selection) == len(self.events)
                  assert len(self.drop_log) >= len(self.events)
              assert len(self.selection) == sum((len(dl) == 0 for dl in self.drop_log))
              assert hasattr(self, "_times_readonly")
              assert not self.times.flags["WRITEABLE"]
              assert isinstance(self.drop_log, tuple)
              assert all(isinstance(log, tuple) for log in self.drop_log)
              assert all(isinstance(s, str) for log in self.drop_log for s in log)
          
          def reset_drop_log_selection(self):
              """Reset the drop_log and selection entries.
          
              This method will simplify ``self.drop_log`` and ``self.selection``
              so that they are meaningless (tuple of empty tuples and increasing
              integers, respectively). This can be useful when concatenating
              many Epochs instances, as ``drop_log`` can accumulate many entries
              which can become problematic when saving.
              """
              self.selection = np.arange(len(self.events))
              self.drop_log = (tuple(),) * len(self.events)

If no event id is found at all and 'on_missing' is set to 'ignore' it will
crash here:

github.com

mne-tools/mne-python/blob/main/mne/epochs.py#L731


      
          def load_data(self):
              """Load the data if not already preloaded.
          
              Returns
              -------
              epochs : instance of Epochs
                  The epochs object.
          
              Notes
              -----
              This function operates in-place.
          
              .. versionadded:: 0.10.0
              """
              if self.preload:
                  return self
              self._data = self._get_data()
              self.preload = True
              self._do_baseline = False
              self._decim_slice = slice(None, None, None)
              self._decim = 1

-Denis

-V

nice to hear it works

how would you phrase the doc? It is currently:

     on_missing : str
        What to do if an event id is not found in the recording.
        Valid keys are 'error' | 'warning' | 'ignore'
        Default is 'error'. If on_missing is 'warning' it will proceed but
        warn, if 'ignore' it will proceed silently.

Alex

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

The information in this e-mail is intended only for the person to whom it
is
addressed. If you believe this e-mail was sent to you in error and the
e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
MyComplianceReport.com: Compliance and Ethics Reporting . If the e-mail was sent to you in
error
but does not contain patient information, please contact the sender and
properly
dispose of the e-mail.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20141003/ea291354/attachment.html

Topic		Replies	Views
Epochs.save() ✉️ Mailing List Archive (read-only) list-archive	2	153	May 21, 2014
[mne-python] tfr_morlet to return single epochs ✉️ Mailing List Archive (read-only) list-archive	4	230	August 30, 2014
time frequency analysis ✉️ Mailing List Archive (read-only) list-archive	1	184	January 9, 2019
plotting topograph issue with channel type ✉️ Mailing List Archive (read-only) list-archive	1	206	November 25, 2013
loading -ave.fif between mne versions ✉️ Mailing List Archive (read-only) list-archive	5	181	February 17, 2017

on_missing and time-frequency

Related topics