External Email - Use Caution
Hi all,
I've written a previous email explaining in more detail the process leading
up to this epoching issue, but it seems that the email has been hung up
waiting for moderator approval. I'll start from the beginning:
Here is where I load up the raw data, which was imported from the CMI
database in a .raw format:
raw=mne.io.read_raw_egi (raw_file_path, montage=
mne.channels.read_montage(kind= 'GSM-HydrocCel-129'), preload=True,
verbose= True)
Output:
Reading EGI header from [raw_file_path] ....
Reading events..
Assembling measurement info...
Synthesizing trigger channel 'STI 014' ...
Excluding events {101,102,103.....[etc] } (the events listed as being
excluded here have the same labels as the nonsense channels talked about
below-- not sure what to make of that)
Reading 0 .... 168248 = 0.000 ... 336.496 secs ....
Upon importing the raw file there are quite a few channels that seem to be
not 'real' scalp or external electrodes included in the raw object. One of
them, 'STI 014,' is useful for reading the eyes closed/eyes open events but
there are about 50 other channels that contain no data and are randomly
labeled.
raw.info['ch_names'][129:] yields a list of these nonsensical channels: [
'101' , '102' , '3' , '11' ,........]
I split up and re-concatenate the raw dataset into its respective eyes-open
and eyes-closed sections, and proceed from there with 2 sets of raw data.
After going through several standard preprocessing steps (filtering,
re-referencing, ICA) without a hitch, I run into my issue with epoching the
data into 2 seconds.
The sample frequency is 500 Hz, so I construct an array of trigger events
by sampling every 1000th index and appending it to the array with a made-up
trigger code of 7.
print(epoch_array) yields [ [ 0, 0, 7], [1000, 0, 7], [2000, 0, 7], [ 3000,
0, 7].....]
I use this array to construct the epoch object:
twoSec_epoch = mne.Epochs(eyes_closed_raw, events=epoch_array, tmin=0,
tmax=0, event_id={'twoSec':7}, reject=None, flat=None,
reject_by_annotation=False)
Output:
93 matching events found
Applying baseline correction (mode: mean)
Not setting metadata
0 projection items activated
And of the 93 epochs in twoSec_epoch, the first 28 of them are being
dropped seemingly without reason.
twoSec_epoch.get_data()
Output:
Loading data for 93 events and 1001 original time points ...
28 bad epochs dropped
This happens whether or not the empty channels are dropped from the raw
object before creating the epoch object.
twoSec_epoch.drop_log[:28] yields [ ['NO_DATA'], ['NO_DATA'],
['NO_DATA'].....]
But I know that the 'real' electrodes (with the random fake channels
dropped) do contain data for these epochs:
raw_data_mat= eyes_closed_raw.get_data()
first_28_epochs= raw_data_mat[:128, 0:28*1000]
print(sum(sum(np.isnan(first_28_epochs))))
print(sum(sum(first_28_epochs==0)))
yields 0 and 0, ie there is data for each of the 128 channels.
I am unsure whether the issue is with the epoch_array I'm making or with
something else about this particular dataset. Any help/guesses would be
greatly appreciated!
Thank you and best wishes,
Dillan Cellier