_orig_time issue with annotations

[dsrishyla]

MNE version: 1.0.1
Operating System: Windows 10

I am adding new annotations to a .fif file which already has annotations. When I re-open the plot of the file to edit or add more annotations, I find that the previous annotations have moved in time.
I suspect this has to do with my use of _orig_time. However, I am unable to pinpoint the issue. So I am requesting some assistance. Here is the function I am using to add new annotations:

def add_annot(raw, annot_file):
    if Path(annot_file).exists():
        annotations=mne.read_annotations(annot_file)
        annotations._orig_time = raw.annotations._orig_time
        list_annot_values = [tuple(annot.values()) for annot in annotations 
                                                   if annot["description"].lower() 
                                                   in ['saccade', 'clean']]
        onset, duration, description = list(zip(*list_annot_values))[:3]
        orig_annotations = mne.Annotations(onset,duration,description)
        raw.set_annotations(orig_annotations + raw.annotations)
        
    else:
        warnings.warn(f'Path {annot_file} not existent')

Let me know if I must clarify anything. Thank you for your time.

Best,
Diksha

[richard (Richard Höchenberger)]

I think you simply need to delete that line in your code that sets the private attribute _orig_time. Simply append annotations without an orig_time set, and never manually modify private attributes (those starting with an underscore) manually unless told by a developer

Best wishes,
Richard

[dsrishyla]

Hi Richard,
Thanks for your suggestion. I tried that, and it still doesn’t work. For example, when I add a new annotation to the interactive window at 12s, the saved annotations file shows the onset time at 14.6 s. Any idea why this might be happening?

Best,
Diksha

[dsrishyla]

Hi again,
My supervisor and I seem to have found a solution. We had to separate the annotations that were already in the file, from the new annotations being added. We did this by adding a third argument to the function:

def add_annot(raw, annot_file, orig_annots):
    if Path(annot_file).exists():
        annotations=mne.read_annotations(annot_file)
        
        list_annot_values = [tuple(annot.values()) for annot in annotations 
                                                   if annot["description"].lower() 
                                                   in ['saccade','clean']]
        if len(list_annot_values)==0:
            return 
        onset, duration, description = list(zip(*list_annot_values))[:3]
        file_annots = mne.Annotations(onset,duration,description)
        raw = raw.set_annotations(file_annots + orig_annots)
else:
        warnings.warn(f'Path {annot_file} not existent')

Again, thanks for your time.

Best,
Diksha

[scott-huberty]

I spoke to @christian-oreilly about this, (he helped @dsrishyla troubleshoot the unexpected behavior).

Seems like when working with a raw object that has been cropped with a tmin > 0, unexpected behaviour can occur with mne.Annotations.

1. Let’s say we have a raw file with one annotation in it.

from mne.datasets import testing
import mne

egi_fpath = testing.data_path() / 'EGI' / 'test_egi.mff'

raw = mne.io.read_raw_egi(egi_fpath, preload=True)
raw.filter(1,40)  # just for visualization
test_annot = mne.Annotations(onset=[2],duration=[2], description=['TEST'])
raw.set_annotations(test_annot)
raw.plot()

2. Now if I crop the file and append a new annotation at 6 seconds.

raw2 = raw.copy().crop(tmin=1, tmax=11)  # note that tmin is > 0
raw2.annotations.append(onset=[6], duration=[2], description=['TEST2'])
raw2.plot()  # new annot TEST2 starts at 5 not 6. 

I set the onset of the new annotation at 6, but the onset is actually 5.

3. Then I tried just resetting the annotations.

raw2 = raw.copy().crop(tmin=1, tmax=11) 
new_annot = (onset=[6], duration=[2], description=['TEST2'])
raw2.set_annotations(test_annot + new_annot)
raw2.plot() 

The new annot starts at 6, as expected. The original annotation still occurs at 2 seconds, but is actually shifted 1-second to the right in relation to the signal, compared to where it originally was in the uncropped raw object.

#2 seems like a bug

#3 maybe not a bug per se… set.annotations is doing what you ask - setting the first annotation at 2 seconds. But I could see why someone might try this and get tripped up, especially if annotations.append wasn’t working as expected like in #2. I think this is what @dsrishyla experienced!

EDIT: this example was demonstrated with mne 1.3.0

[scott-huberty]

Follow up:

I just read the warning in the documentation for mne.Annotations:

when raw.info['meas_date'] is None , doing raw.set_annotations(raw.annotations) will not alter raw if and only if raw.first_samp == 0 . When it’s non-zero, raw.set_annotations will assume that the “new” annotations refer to the original data (with first_samp==0 ), and will be re-referenced to the new time offset!

Seems like that could explain the unexpected (to me) behavior described in #2 above, so I must be missing something here…