mne_brain_vision2fiff Losing Events?

Hello,

When converting with mne_brain_vision2fiff, the resulting fif files seem to be missing events on occasion. My recordings should all contain 1960 events. mne_brain_vision2fiff says "nevent = 1960" (for the complete output see below), however, when opening the resulting fiff files in mne-python, up to 3 events are missing. The conversion seems to lose events consistently (i.e. when I repeat the command the same number of events get lost):

n events found by mne-python for my different subjects:
1959 R0040_eref3h_raw.fif
1958 R0458_eref3h_raw.fif
1957 R0425_eref3h_raw.fif
1957 R0355_eref3h_raw.fif
1960 R0273_eref3h_raw.fif
1960 R0541_eref3h_raw.fif

The event channel data reflects that (for a recording with 1958 events):

np.unique(data)

array([ 0., 128.])

np.sum(np.diff(data) > 0)

1958

np.sum(np.diff(data) < 0)

1958

I also tried the nightly version of mne with the same result. In case someone can have a look, I uploaded a set of files (10 mb): https://dl.dropbox.com/u/659990/R0458%20eref3h.zip

Thanks

Christian

$ mne_brain_vision2fiff --header R0458_eref3h.vhdr --out R0458_eref3h.fif

mne_brain_vision2fiff version 1.7 compiled at Jan 7 2011 02:26:21

header file : R0458_eref3h.vhdr
output file : R0458_eref3h.fif

Read header file : R0458_eref3h.vhdr
Read header file : R0458_eref3h.vmrk

nchan = 5
sfreq = 500.0 Hz
raw data file = R0458_eref3h.eeg
raw binary = true
raw binary type = 16
raw endian = little
raw nsamp = -1
nevent = 1960
eXimia = false

Data will be split into <= 1907.3 MByte pieces

Writing header to R0458_eref3h_raw.fif ... [done]
Writing raw data to R0458_eref3h_raw.fif ... Number of samples deduced from the raw file size : 671230
[done]
Wrote 1...671230 of 671230 samples total to R0458_eref3h_raw.fif
Adding directory to R0458_eref3h_raw.fif...[done]

hi Christian,

I confirm that I find 1958 events in your file with both
mne_process_raw and mne-python

$mne_brain_vision2fiff --header R0458_eref3h.vhdr --out test.fif

indeed reports 1960 events although with

$mne_process_raw --raw test_raw.fif --events test-eve.fif

and loading test-eve.fif in Python I get only 1958

import mne
ev = mne.read_events('test_raw-eve.fif')
len(ev[ev[:, 2] == 128])

1958

I therefore suspect an issue in the C code maybe while writing the
stim channel to fif.

We'll have to dig in the C code and maybe Matti can give us a hand.

Alex

Christian,

do you know which events are missed?

do you see the stim channel rising from 0 at these time points?

@matti : any idea what may be happening?

Best,
Alex

Hello,

Thanks for looking into this Alex! Based on a comparison of indexes, the events that are missing from the fif file are Mk292 and Mk1582. As far as I can see these events are not different from other events, neither in the text file::

  Mk291=Stimulus,S128,97717,1,0
  Mk292=Stimulus,S128,98000,1,0
  Mk293=Stimulus,S128,98300,1,0
  ...
  Mk1581=Stimulus,S128,531684,1,0
  Mk1582=Stimulus,S128,532000,1,0
  Mk1583=Stimulus,S128,532300,1,0

nor when printing the line content as Python string::

  >>> p = [291, 292, 1582]
  >>> for line in open('/Volumes/Iomega/ereftests/raw_eeg converted vmrk/R0458_eref3h.vmrk'):
  ... if any(line.startswith('Mk%s' % pi) for pi in p):
  ... print repr(line)
  ...
  'Mk291=Stimulus,S128,97717,1,0\n'
  'Mk292=Stimulus,S128,98000,1,0\n'
  'Mk1582=Stimulus,S128,532000,1,0\n'

The Stim channel at these time points is simply flat, at 0 (https://dl.dropbox.com/u/659990/R0458%20eref3h/missing_events.py).

Best,

Christian

hi Christian

after further investigation there is clearly a bug when writing events
in the C code.
Try offsetting by 1 the forgotten events and they will appear magically.
Before we fix the bug in the C code I would use Python to parse the vrmk file
and fix the stim channel with mne-python. Something like

mne_brain_vision_fix_stim.py --raw --vrmk file.vrmk

that would do read/write on the raw after using some raw[stim_idx,
time_idx] = 128.

HTH
Alex