How to maintain sets of marker in epochs

[balandongiv]

Given a continuous signal, I would like to convert into epoch.

Each epoch made from three marker (i.e., deviation onset, response onset, response offset), as illustrated below.

The Extract epochs functions in EEGLAB can automatically identified the 3 events and create an epoch, out of these markers as shown below.

As shown above, the 3 markers were preserved.

However, in mne

events = mne.events_from_annotations (raw)
epochs = mne.Epochs(raw, events[0], preload=True)
#or
#epochs = mne.Epochs(raw, events[0], event_id=events[1],preload=True)
epochs.plot()

While the epochs were successfully created, the 3 marker in the epochs is diminish as shown below.

I’m curious whether it is possible/ or how to achieved preserved this marker as in EEGLAB?
The corresponding .set file can be downloaded via the link: https://figshare.com/ndownloader/files/14249795

Also, may I know what is the proper setting to offset , so that we extract, 1 sec before the deviation onset, and 1 sec after the response offset?
Something like

epochs = mne.Epochs(raw, events[0], preload=True,tmax=1,tmin=1)

Since along the pipeline I would like to do some analysis for t-deviation onset, ResponseOnset-DeviationOnset, ResponseOffset-ResponseOnset, and t-ResponseOffset, I wonder whether it is efficient to create epochs as I desire to achieve above? Or It is better to create separate epochs for each of t-deviation onset, ResponseOnset-DeviationOnset, ResponseOffset-ResponseOnset, and t-ResponseOffset. For example, epoch for time in between ResponseOnset to DeviationOnset, and so on.

Thanks in advance for any insight/confirmation

[richard (Richard Höchenberger)]

Hello @balandongiv,

what you describe is unfortunately currently not possible with MNE-Python. I suppose it could be easily implemented using epochs metadata, however it would definitely require some adjustments to our plotting function to make use of this.

@drammock WDYT?

Edit:
Do you, for now, only care about the data itself, or is the visualization the crucial part? (I only considered the latter when I wrote my post. Retaining the data would definitely be possible already today using epochs metadata. Let me know if that would suffice for now, then I can point you to some useful resources.)

[balandongiv]

Thanks for the quick response @richard .

Im more concern for the analysis pipeline later. I know the events = mne.events_from_annotations (raw) may separate each of the events.

However, Im not so sure how to tune the mne.Epochs so that it will extract the following

1. With reference to Deviation Onset marker
   tmin=1,tmax=0

2. With reference to Response onset
   tmin=0, tmax=1

or with the aid of visualization, it will be something like

Please note the text 1 sec

@balandongiv

#epochs = mne.Epochs(raw, events[0], event_id=events[1],preload=True)

I believe with the current implementation, the epoch is created by using the default setting tmin=-0.2, tmax=0.5,

@richard

Do you, for now, only care about the data itself,

Ultimately, I more concern of epoch creation as I would like to, later, for example, calculating psd separately for time interval t-deviation onset , ResponseOnset-DeviationOnset , ResponseOffset-ResponseOnset , and t-ResponseOffset. Please let me know if this requirement is not clear.

[richard (Richard Höchenberger)]

So epochs should begin 1 s before deviation onset and end 1 s after response offset, is this correct?or are these two different types of epochs you want to create?

[balandongiv]

Hi @richard, I had edit the figure above to make things clear.

@balandongiv

or with the aid of visualization, it will be something like

@richard

1 s before deviation onset and end 1 s after response offset, is this correct

Im afraid we are not on the same. There should be 3 epochs. I think the amended figure explain better now

[richard (Richard Höchenberger)]

Oh, so you want the following 3 epochs per trial

1. deviation onset – 1 s → deviation onset
2. deviation onset → response offset
3. response offset → response offset + 1 s

Is this correct?

[balandongiv]

Hi @richard , thanks for the support.

@richard

Oh, so you want the following 3 epochs per trial

Yes, you are correct. This is the definition per trial.

Based on this requirement, I have the impression that we can achieve this as follow, albeit there is some error at mne.Epochs

In summary, the code below create separate epochs for each of the trial. But, Im not sure this is the recommended way or not.

Appreciate for any suggestion

import mne
import numpy as np
import pandas as pd

raw = mne.io.read_raw_eeglab(fpath,preload=False).crop(tmax=160)

montage = mne.channels.make_standard_montage ( 'standard_1020' )
raw.set_montage ( montage, match_case=False )
events = mne.events_from_annotations (raw)
nfreq=500
arr=events[0]
trange = np.where(arr[:,2] == 3)[0]
val=np.array([arr[trange, 0],arr[trange, 2],(arr[trange, 0]-arr[trange-1, 0])/nfreq,
          (arr[trange+1, 0]-arr[trange, 0])/nfreq]).T


trange= np.where(arr[:,2] != 3)[0]
val_oth=np.array([arr[trange, 0],arr[trange, 2],arr[trange, 2],arr[trange, 2]]).T
df = pd.DataFrame(np.vstack((val,val_oth)),columns=['timepoint','event','pre','post'])
df['duration']=0
df["event_id"] = df["event"].map(dict((value,key) for key,value in events[1].items()))
df['event'] = df.event.astype('int')
df['timepoint'] = df.timepoint.astype('int')

df.loc[df.event==4, ['pre','post']] = [0,6]
df.loc[df.event.isin([1,2]), ['pre','post']] = [6,0]
df.loc[df.pre>0,'pre']=df['pre']*-1
df.sort_values(by='timepoint', ascending=True,inplace=True)
df.reset_index(drop=True,inplace=True)
ls_epochs=[]

for x in range(len(df)):

    my_event=np.array(df.loc[x,['timepoint','duration','event']].values.tolist()).reshape(-1, 3)
    tmin=df.loc[x,['pre']].values[0]
    tmax=df.loc[x,['post']].values[0]
    event_type=df.loc[x,['event']].values[0]
    event_idx=df.loc[x,['event_id']].values[0]

    epochs_=[]
    try:
        epochs_ = mne.Epochs(raw, my_event,preload=True,tmin=tmin,tmax=tmax,event_id={event_idx:event_type})
    except ValueError:
        epochs_ = mne.Epochs(raw, my_event,preload=True,tmin=tmin,tmax=tmax,baseline=(0, 0),event_id={event_idx:event_type})
    ls_epochs.append(epochs_)

all_epoch=ls_epochs

@balandongiv

or with the aid of visualization, it will be something like

Kindly refer this figure for the events defination (i.e., 1,3,4). Please note on the number in red bracket.

[drammock (Dan McCloy)]

each epoch within an Epochs object in MNE-Python must all have the same duration. So, it will be possible to create one-second-long deviation_onset_epochs using event code 1 and tmin=-1, tmax=0 (labeled “epoch1” in your diagram). It is also possible create one-second-long response_offset_epochs using event code 4 and tmin=0, tmax=1 (“epoch 3” in your diagram).

But I’m guessing that the time between deviation onset and response offset is not always the same duration (since you didn’t label that duration in your diagram). If so, that means you cannot easily create an Epochs object for “epoch 2”. Either you have to store them as a list of separate Raw objects suitably cropped, or store them as a list of separate Epochs objects of different duration (each containing only one epoch), or compute in advance what the longest duration will be and use that as your tmax when creating the “epoch 2” object (and know that it will contain more data than you want on certain trials).

[balandongiv]

Thanks for the clarification and suggestion @drammock . I reckon the order (sequence of epoch per trial) will be mess up if I am to create the Epochs object for “epoch 2” (I maybe wrong about this assumption at this stage). Nevertheless, I understand in principle your concern.

Worse case is to Extract epochs the continuous signal in EEGLAB and subsequently apply epochs=mne.io.read_epochs_eeglab`.

[richard (Richard Höchenberger)]

Could you explain to me (I’m not an EEGLAB user) how Extract epochs would handle your case for epoch 2? Would all epochs have different durations?

[balandongiv]

Thanks for the continue support @richard.

@richard

Could you explain to me (I’m not an EEGLAB user)

I dont have enough knowledge to explain about your question. But, if you can refer to the figure I attached above (shortcut below), the duration for epoch2 is automatically detected to be in between the marker deviation onset and response onset.

@balandongiv

The Extract epochs functions in EEGLAB can automatically identified the 3 events and create an epoch, out of these markers as shown below.

I believe this epoching that I desired is similar to how other epoch the signal for n-back . I tried comb the net, but to no available .

[balandongiv]

Hi @drammock ,

I think, I am able to epoched the continuous signal.

However, along the way, mne keep pop an ValueError

ValueError: Baseline interval is only one sample. Use `baseline=(0, 0)` if this is desired.

Accordingly, I catch this error and as recommended used baseline baseline=(0, 0) as shown below

@balandongiv

epochs_ = mne.Epochs(raw, my_event,preload=True,tmin=tmin,tmax=tmax,baseline=(0, 0),event_id={event_idx:event_type})

However, I am not so sure why and what does baseline=(0, 0) means here?

Can you kindly give some hints about this. I dig into the main file, but limited information is attached.

@drammock

If so, that means you cannot easily create an Epochs object for “epoch 2”.

As per your explanation,

epochs_standard = mne.concatenate_epochs(ls_epochs)

Return an error

return less_equal(abs(x-y), atol + rtol * abs(y))

ValueError: operands could not be broadcast together with shapes (2075,) (3001,)

[drammock (Dan McCloy)]

@balandongiv

I am not so sure why and what does baseline=(0, 0) means here?

I’ll give a few examples. baseline can be:

• (-0.5, 1.2) means use all samples between time=-0.5 and time=1.2
• (None, -0.1) means use all samples occuring before or at time = -0.1
• None means do not baseline at all
• (0.3, 0.3) means use just the single sample that occurs closest to time=0.3

You’re getting the warning probably because tmin=0 and the default value for baseline is (None, 0), so there is only one sample before or at time=0. The warning says that if you really wanted to use a single sample as the baseline then pass baseline=(0,0). I’m guessing though that you do not actually want that? You should instead explicitly pass in a value for baseline because the default value is not appropriate when tmin=0.

[balandongiv]

@drammock , thanks for the detail explanation + accompanying example.

To interested reader, further read about what is baseline correction.

[yaqing]

Hello everyone, I’m very new to MNE and would really love to follow up this discussion two years later.

I have a similar situation in my MEG experiment–I play single sentences to my subjects, and each sentence has three events: sentence onset, and onsets of two keywords.

Since the keyword onsets vary across sentences, I thought it would be better to first epoch response into sentences, do baseline correction using the silent period before sentence onset, then epoch within each sentence around the two keywords.

However, once I epoch the sentences, the keyword events are lost… Has there been any update about this quite particular problem? Would you suggest some other ways around?

Many thanks,
Yaqing