Grand Averaging from Text Events Files

Mailing List Archive (read-only)
1

Hello MNE users,
I am having an odd issue that I am having trouble conceptualizing.

I am averaging data that is split into 3 raw data files, then taking the
events files generated from these averages and modifying these events files
to include only events where the subject responded correctly. I am then
taking these corrected events files and grand averaging the raw data to end
up with a single .fif average which is corrected for our behavioral data.

The issue I am having is related ot the initial offsets in the events
files. In all cases, there is an initial offset greater than zero for the
2nd and 3rd raw data files. In some cases, there is also an initial offset
of a few seconds in the events file for the first of the three raw data
files. It seems from the log files that the offset is being utilized
differently for the two cases (a few seconds offset vs a whole dataset
offset). For the log from the first file, the offset is added to the times,
while for the other files, the offset is not added. For example, with an
initial offset of 4 seconds, the log for the uncorrected average starts at
29.555 seconds, while for the corrected data, the log starts at 33.555
seconds. The logs are identical for the second and third data files. This
is also apparent in the averages, as there is no signal in the corrected
average of the first raw file. It seems that the offsets are necessary for
the later files (data files 2 and 3), however, as I get no averages without
them.

My question is: Should I eliminate these offsets from the first text events
file when averaging from events files? Is there something I have not
considered here?

Thanks in advance!!

-Brian Coffman, M.S.
Research Associate, The Mind Research Network
Doctoral Candidate, The University of New Mexico

Phone: (505) 336-0606
Fax: (505) 272-7011

"I used to think that the brain was the most wonderful organ in my body.
Then I realized who was telling me this."
  - Emo Phillips
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20120125/a0ac56e6/attachment.html

2

Hi Brian,
   Is the normal averaging (as opposed to grand averaging) working as
expected for the individual runs with the behavioral corrected event
files?

Regards,
Hari

3

Brian,

I'm not sure why you need to change the events by an offset. I'll explain
my approach to the same problem (rewriting new .eve files based on subject
criteria eg. correctness). From your description, I am not sure what you
mean by offsets, and I think if your triggers were at the right times you
don't want to rewrite or readd any offsets just use the original event
timing. (If you do want to change the timing of triggers that is another
problem my lab has to deal with. We have a solution but haven't tested it
out yet)

I contain all of my subject (and study) variables in structures, hopefully
their syntax is intuitive so I don't need to explain it. I do this in
Matlab, but if you use a different programming environment I may be able
to help you translate.

Presume we have already extracted event files from each of the raw data
blocks.

event_file = sprintf('%s/triggers/%s_%s.eve',...
  subject.meg.dir, subject.name, block);
events = load(event_file);

Matlab's load command loads them perfectly. Now events is a N row by 4
column matrix representing the data from the .eve file. Then my code goes
line by line, finding events that correspond to triggers we sent and the
next user input and evaluates their action. For events that we will use in
the analysis I reassign by:

events(i, 4) = some_code;

I do not delete events I will not use. I presume you still could by doing:

events(i, :slight_smile: = ;

or just make a new events matrix for ones you will include

new_events(end + 1, :slight_smile: = events(i, :);

just make sure to include the initial line before you do that

new_events(1, :slight_smile: = events(1, :);

Then I make a new event_file and reprint every line based on the events
matrix.

event_file = sprintf('%s/triggers/%s_%s_applied.eve',...
  subject.meg.dir, subject.name, block);
fid = fopen(event_file);
for i = 1:size(events, 1)
  fprintf(fid, '%6d %3.3f %6d %3d\n',...
    events(i, 1), events(i, 2), events(i, 3), events(i, 4));
end
fclose(fid);

I hope that makes some sense and could be useful. Good luck with it!

- Conrad Nied
Neuropsychology Lab @ MGH, David Gow
617 724 8846
conrad at martinos.org

4

Hi Hari,
No, the normal averaging is doing the same thing. Just tested this.

5

Hi Conrad,
Thanks for your help. This looks a bit complex for what we are doing. We
already get the events we are interested when averaging without corrections
by specifying --eventsout within mne_process_raw. These files come out with
an initial skip (offset) value in the form of "seconds milliseconds 0 0."
It seems after a bit more testing that this initial skip is not accounted
for when averaging from the events files that come out of MNE.

I have started to use the log file timing for the first run (log files seem
to contain timing relative to the start of the .fif file, while the events
files contain timing for the entire dataset, with an initial skip at the
beginning of the file) and this works fine. This doesn't work for the 2nd
and 3rd file, though, as MNE looks for the dataset event time rather than
the fif file event time. Using the events file timing for the second and
third runs seems to work just fine, with or without the initial skip.

It seems very strange that MNE would write events in a way that can't be
read for averaging. Is this possibly related to the version of MNE I am
using (2.7.3)?

6

Hi Conrad,
Thanks for your help. This looks a bit complex for what we are doing. We
already get the events we are interested when averaging without corrections
by specifying --eventsout within mne_process_raw. These files come out with
an initial skip (offset) value in the form of "seconds milliseconds 0 0." It
seems after a bit more testing that this initial skip is not accounted for
when averaging from the events files that come out of MNE.

How did you test this? We need to understand what tests you are doing
to understand why you think it is "not accounted for."

I have started to use the log file timing for the first run (log files seem
to contain timing relative to the start of the .fif file, while the events
files contain timing for the entire dataset, with an initial skip at the
beginning of the file) and this works fine. This doesn't work for the 2nd
and 3rd file, though, as MNE looks for the dataset event time rather than
the fif file event time. Using the events file timing for the second and
third runs seems to work just fine, with or without the initial skip.

It seems very strange that MNE would write events in a way that can't be
read for averaging. Is this possibly related to the version of MNE I am
using (2.7.3)?

How is MNE writing events in a way that can't be used for averaging?
Can you explain what this means?

Dan

7

Hello again,

> Hi Conrad,
> Thanks for your help. This looks a bit complex for what we are doing. We
> already get the events we are interested when averaging without
corrections
> by specifying --eventsout within mne_process_raw. These files come out
with
> an initial skip (offset) value in the form of "seconds milliseconds 0
0." It
> seems after a bit more testing that this initial skip is not accounted
for
> when averaging from the events files that come out of MNE.
How did you test this? We need to understand what tests you are doing
to understand why you think it is "not accounted for."

When averaging directly from the events file that MNE has generated (using
--eventsout) in a file with an initial skip of 4 seconds, I get a .fif and
a .log file that together suggest that MNE is averaging 4 seconds later
than it should, as if the first line of the events file is disregarded
during averaging.

> I have started to use the log file timing for the first run (log files
seem
> to contain timing relative to the start of the .fif file, while the
events
> files contain timing for the entire dataset, with an initial skip at the
> beginning of the file) and this works fine. This doesn't work for the 2nd
> and 3rd file, though, as MNE looks for the dataset event time rather than
> the fif file event time. Using the events file timing for the second and
> third runs seems to work just fine, with or without the initial skip.
>
> It seems very strange that MNE would write events in a way that can't be
> read for averaging. Is this possibly related to the version of MNE I am
> using (2.7.3)?

How is MNE writing events in a way that can't be used for averaging?
Can you explain what this means?

The event times in the --eventsout file are equal to the event times in the
log file + the initial skip. If the initial skip were taken into
consideration and subtracted from the event times that follow when
averaging, then one would get the correct event times, however, this does
not seem to be the case. The log file from an average using the event file
generated by MNE shows events are off by 4 seconds (the initial skip) and
are in fact the event times listed in the event file. The log file from an
average without specifying an event file shows the correct event times.

8

Hey Brian Dan and MNErs,

I think you may be misinterpreting the event files.

Take for instance the first three lines of a event file I have:

19800 32.966 0 0
20740 34.531 0 36
21446 35.707 0 128

* The first column is the sample number (which is dependent on the sample
rate, for this subject it is 600.615 Hertz) since I pressed the GO button
on the MEG machine (I didn't click the record raw button until 32 seconds
in because I was writing the HPI coordinates, 19800 samples in)
* The second column is the seconds since GO
* The third is the channel STI 014 base (always 0)
* The fourth is the channel's increase to (which should correspond to the
trigger number you gave it).

Check out the manual for more details.

http://martinos.org/mne/manual/browse.html#event-files

Does this explain the weirdness in the files?

- Conrad

9

Hi,

I think I can explain:

Hello again,

> Hi Conrad,
> Thanks for your help. This looks a bit complex for what we are doing. We
> already get the events we are interested when averaging without
> corrections
> by specifying --eventsout within mne_process_raw. These files come out
> with
> an initial skip (offset) value in the form of "seconds milliseconds 0
> 0." It
> seems after a bit more testing that this initial skip is not accounted
> for
> when averaging from the events files that come out of MNE.
How did you test this? We need to understand what tests you are doing
to understand why you think it is "not accounted for."

When averaging directly from the events file that MNE has generated (using
--eventsout) in a file with an initial skip of 4 seconds, I get a .fif and a
.log file that together suggest that MNE is averaging 4 seconds later than
it should, as if the first line of the events file is disregarded during
averaging.

Well, I believe there is a logic to this which follows:
time in an events file should be relative to the raw data, because
that is what it applies to. Therefore the time in an events file is
displayed primarily in samples as you would find in raw.first_samp
(from [raw] = fiff_setup_read_raw(<name of raw file>)).

time in a log file should be relative to the actual data points you
have (as one cannot process non-existent data points). Therefore the
time in a log file starts from the first sample, T, which corresponds
to the sample # S in
T = S - (raw.first_samp +1)

This is a very small detail, but it does have a logic to it.

>
> I have started to use the log file timing for the first run (log files
> seem
> to contain timing relative to the start of the .fif file, while the
> events
> files contain timing for the entire dataset, with an initial skip at the
> beginning of the file) and this works fine. This doesn't work for the
> 2nd
> and 3rd file, though, as MNE looks for the dataset event time rather
> than
> the fif file event time. Using the events file timing for the second and
> third runs seems to work just fine, with or without the initial skip.
>
> It seems very strange that MNE would write events in a way that can't be
> read for averaging. Is this possibly related to the version of MNE I am
> using (2.7.3)?

How is MNE writing events in a way that can't be used for averaging?
Can you explain what this means?

The event times in the --eventsout file are equal to the event times in the
log file + the initial skip. If the initial skip were taken into
consideration and subtracted from the event times that follow when
averaging, then one would get the correct event times, however, this does
not seem to be the case. The log file from an average using the event file
generated by MNE shows events are off by 4 seconds (the initial skip) and
are in fact the event times listed in the event file. The log file from an
average without specifying an event file shows the correct event times.

Can you e-mail me (directly) the three/four files you are talking
about, so I can see the values? To look into the error further.

10

Hello everyone,
I have tested many different things and have decided that the issue was
that I was using the UNIX command "sort" to recombine and sort event lists
for individual events. If I sort the text manually (using a text editing
program) and insert the events by copy and paste into a new text file, the
initial skip is read from the file without error. I still am not sure why
using sort in this way effects the ability of MNE to read the initial skip
at the head of the file, but this seems to be the issue. I have had issues
using this command in other tasks as well and I recommend others be
cautious in using this command in UNIX scripting.

Thanks for your help with this, everyone.