many epochs dropped due to 'NO_DATA'

Mailing List Archive (read-only)
1

Hi,

Thanks for the hint; It must have something to do with it.

I must be overlooking something here, but I can't get it right:
Somehow it looks as if the epochs method looks for timestamps from
before the cropping of the data from somewhere (?), although I'm not
sure how that is possible

I first crop my data with raw.crop, save that data, and load it again
from file.
I previously crop the file at 232 seconds - it loses the first 44 epochs.
Other files I don't crop don't have this problem.

I can't figure out what's wrong about the time indices:

the resulting raw structure has the correct raw.time (from 0 to 618, at
sfreq=1000) and time indices:
raw[0][1] goes from 0 to 618;
len(raw[0][1]): 618001

My events array goes like this:
In [37]: events
Out[37]:
array([[ 9279, 0, 1],
       [ 18677, 0, 2],
       [ 28475, 0, 3],
       [ 38491, 0, 3],
       [ 49491, 0, 1],
       [ 58705, 0, 3],
       [ 69505, 0, 1],
       [ 79905, 0, 1],
       [ 89736, 0, 1],
       [ 99134, 0, 2],
       [108933, 0, 1],
       [118948, 0, 1],
       [128146, 0, 2],
       [138361, 0, 3],
       [147759, 0, 2],
       [157558, 0, 3],
       [167774, 0, 1],
       ......
       [595400, 0, 1]])

Notably, I lose the first epochs, not the last epochs:

epochs = mne.Epochs(raw,events, event_id, tmin=tmin, tmax=tmax,
proj=True, picks=picks_plan2,
                    baseline=None, preload=True,
                    reject=None)

In [35]: 60 matching events found
Applying baseline correction (mode: mean)
0 projection items activated
Loading data for 60 events and 1001 original time points ...
44 bad epochs dropped

In [36]: epochs.drop_log
Out[36]:
[['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
['NO_DATA'],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[],
[]]

Any Ideas?
Many Thanks,

cheers Tristan

2

have a look at raw.first_samp

it can interfere. It is basically an offset in samples

A

3

That was it; when adding raw.first_samp to all my events I lose no epochs.
I guess a cleaner way is to reset the first_samp property.

For others with this problem, instead of:
raw.first_samp=0

do this (found by asking ipython for the definition with:
raw.first_samp?? ):
raw._first_samps[0]=0

May I suggest a switch in the crop method (e.g.: keep_offset=True)?
Thanks for the help!

Tristan

4

I'm not sure if it is a good idea to set the first_samp property by hand.
It could have some side effects elsewhere. In general, you shouldn't have
to change the private attributes yourself. I don't know your analysis pipe
line, but the events should line up properly if you do mne.find_events to
the cropped data. This way the events outside the cropping window will be
excluded.