How do I add event_dict as label to raw data

[TTyb]

I want to add the event_dict in raw to the edf data as a label. My code is as follows:

import mne
import os
import pandas as pd

pd.options.display.max_rows = 1000
pd.options.display.max_columns = 10000
pd.options.display.width = 10000

testing_data_folder = mne.datasets.testing.data_path()
eeglab_raw_file = os.path.join("C:/Users/taoya/mne_data/MNE-testing-data/EEGLAB/test_raw.set")
eeglab_raw = mne.io.read_raw_eeglab(eeglab_raw_file)

event_from_annot, event_dict = mne.events_from_annotations(eeglab_raw)
print(event_from_annot)
print(event_dict)

event_dict result is {'rt': 1, 'square': 2}.

and than conver raw to dataframe show is :

![image|690x74](upload://rJCJUtKcgvt517E1KloWAj7OU7N.png)

this shape is (30504, 33)。

when I use mne.Epochs show shape is (14014, 35):

epochs = mne.Epochs(eeglab_raw, events=event_from_annot,  event_id=event_dict,
                    preload=True, baseline=None)

epochs_df = epochs.to_data_frame()
print(epochs_df.head(5))
print(epochs_df.shape)

this dataframe show is :

Now I have two questions:

1. Why does mne.Epochs reduce shape? Is it because of tmax and tmin。and how to keep shape?
2. If I don’t use mne.Epochs, how do I add event_dict as a list of labels to the eeglab_raw dataframe

Thank you!

[richard (Richard Höchenberger)]

Hello,

@TTyb

Why does mne.Epochs reduce shape? Is it because of tmax and tmin

there are several potential reasons why you might end up with fewer epochs than events. It could be that an epoch might extend across the beginning or end of the raw data; it could be that parts of the raw data have been annotated as bad; etc.

To find out what’s happening, run

epochs.plot_drop_log()

This will display a visualization of the reason(s) that lead to dropped epochs, and the frequency of this happening.

@TTyb

1. If I don’t use mne.Epochs, how do I add event_dict as a list of labels to the eeglab_raw dataframe

I don’t understand that question, could you elaborate?

Best wishes,
Richard

[TTyb]

When I use the epochs.plot_drop_log() method, I get this:
Figure(640x480)

We have this problem (1)，the dataframe I get when I parse the edf is:

eeglab_raw_df = eeglab_raw.to_data_frame()
print(eeglab_raw_df.head(5))

the epochs dataframe I get when I parse the edf is:

epochs = mne.Epochs(eeglab_raw, events=event_from_annot,  event_id=event_dict,
                    preload=True, baseline=None)

epochs_df = epochs.to_data_frame()
print(epochs_df.head(5))

now, I want to add epochs_df’s column condition to eeglab_raw_df,but I don’t know how to do that

Looking forward to your reply,thank you !

[CarinaFo (Carina)]

Hello,

if I understand your question, you want to access the event array:
You can access it like this:

epochs.events

epochs.events.shape  # the shape should be the number of epochs x 3

# the 3 dimensions per epoch are the timestamp, the event duration and the event ID, e.g. 

array([[   392,      0,      1],
       [  1461,      0,      1],
       [  2464,      0,      1]

# you can access the event ID by indexing the 3rd dimension per epoch: 

epochs.events[:,2]

Hope this helped,

Carina

[TTyb]

Hello,thank you for your reply,I print epochs.events and epochs.events.shape show:

(154, 3)
[[  128     0     2]
 [  217     0     2]
 [  267     0     1]
 [  602     0     2]
...

that event_dict is {'rt': 1, 'square': 2}.

OK,but when I show epochs.to_data_frame(), it‘s dataframe column name is：

and show epochs.to_data_frame().head(200):

now, 'the 3 dimensions per epoch are the timestamp, the event duration and the event ID, e.g. ', if first column is timestamp, epochs.events show first raw is 128 0 2

epochs.to_data_frame().head(200) index 0 to 181 is square , I don’t know that why epochs.events show first timestamp is 128 and square in the range of index 0 to 181

Looking forward to your reply, thank you !

[richard (Richard Höchenberger)]

@larsoner @agramfort I believe what’s needed here is a way to retain the raw’s sample numbers when constructing the Epochs DataFrame. This would allow to later join the Epochs and the Raw DataFrame using the sample number as index.

But I’m not sure if it’s worth having.

I think we discussed something related before the introduction of Epochs metadata, but I’m not sure what we ended up doing…

Thoughts?

[larsoner (Eric Larson)]

@richard

I believe what’s needed here is a way to retain the raw’s sample numbers when constructing the Epochs DataFrame.

This sounds like something that could fairly easily be done by the end user, e.g., by adding events[epochs.selection, 0] to epochs.metadata after creation, no?

[richard (Richard Höchenberger)]

@larsoner

This sounds like something that could fairly easily be done by the end user, e.g., by adding events[epochs.selection, 0] to epochs.metadata after creation, no?

I meant the sample numbers relating to all data points in the raw, not just the ones relating to events.

[TTyb]

Yes ，I think this is necessary, and I found some bugs in it：

event_from_annot, event_dict = mne.events_from_annotations(eeglab_raw)
data, times = eeglab_raw[:, :]
eeglab_raw_df["real_time"] = times
print(event_from_annot[0:100,:])
print(eeglab_raw_df[["time","real_time"]].head(500))


annotations_list = []
for i in range(len(eeglab_raw.annotations.onset)):
    annotations_list.append((eeglab_raw.annotations.onset[i],eeglab_raw.annotations.description[i],eeglab_raw.annotations.duration[i]))
print(annotations_list)

event_from_annot[0:100,:] show:

[[  128     0     2]
 [  217     0     2]
 [  267     0     1]
 [  602     0     2]
 [  659     0     1]
 [  987     0     2]
 [ 1372     0     2]
 [ 1447     0     1]
 [ 1757     0     2]
 [ 1807     0     1]
 [ 2142     0     2]
 [ 2200     0     1]
 [ 2527     0     2]
...

annotations_list show:

[(1.000068, 'square', 0.0), (1.695381, 'square', 0.0), (2.082407, 'rt', 0.0), (4.703193, 'square', 0.0), (5.148224, 'rt', 0.0), (7.711006, 'square', 0.0), ...]

eeglab_raw_df[["time","real_time"]].head(500) show:

     time  real_time
0       0   0.000000
1       8   0.007812
2      16   0.015625
3      23   0.023438
...
264  2062   2.062500
265  2070   2.070312
266  2078   2.078125
267  2086   2.085938
268  2094   2.093750
269  2102   2.101562
270  2109   2.109375
271  2117   2.117188
...

I Know from eeglab_raw.annotations.onset that ‘rt’ starts at 2.082407 seconds,

I Know from event_from_annot[0:100,:] that ‘rt’ index is 267,

but I get eeglab_raw_df 267 is 267 2086 2.085938？

so,I just want using event as label,raw as tarin data to train my model,but I don’t know how to join event label to raw data.

So,can you solve it as soon as possible, or is there any other way?