mne.io.read_raw_eyelink failure: "adjust times" sub-function does not work/can't merge

đź’¬ Support & Discussions
1

Hi there,

I am using the mne.io.read_raw_eyelink function and am encountering an issue for some of my subjects where the adjust times sub function in utils throws an error. I am using:

  • MNE version: 1.7.1
  • operating system: macOS 14.5

When running mne.io.read_raw_eyelink(f'{path}/P7.asc', create_annotations=["messages"]), I get this error:

File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/mne/io/eyelink/eyelink.py", line 62, in read_raw_eyelink
    raw_eyelink = RawEyelink(
  File "<decorator-gen-202>", line 12, in __init__
  File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/mne/io/eyelink/eyelink.py", line 107, in __init__
    eye_ch_data, info, raw_extras = _parse_eyelink_ascii(
  File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/mne/io/eyelink/_utils.py", line 71, in _parse_eyelink_ascii
    raw_extras["dfs"]["samples"] = _adjust_times(
  File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/mne/io/eyelink/_utils.py", line 509, in _adjust_times
    return pd.merge_asof(
  File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/pandas/core/reshape/merge.py", line 708, in merge_asof
    return op.get_result()
  File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/pandas/core/reshape/merge.py", line 1926, in get_result
    join_index, left_indexer, right_indexer = self._get_join_info()
  File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/pandas/core/reshape/merge.py", line 1151, in _get_join_info
    (left_indexer, right_indexer) = self._get_join_indexers()
  File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/pandas/core/reshape/merge.py", line 2239, in _get_join_indexers
    right_values = self._convert_values_for_libjoin(right_values, "right")
  File "/Users/teichmanna2/anaconda3/envs/occ_beh/lib/python3.9/site-packages/pandas/core/reshape/merge.py", line 2182, in _convert_values_for_libjoin
    raise ValueError(f"{side} keys must be sorted")
ValueError: right keys must be sorted

To replicate the error, I uploaded the data here: eyetracking_example - Google Drive

I tried to sort the values for the data frames before merging but am then running into errors down the line.

Any help would be greatly appreciated! :slight_smile:

Thanks,
Lina

2

Hi Lina,

Thanks for reporting and sharing the problem file. This sounds like a bug in our reader. I’ll try my best to look at this today and will respond here.

1 Like
3

Hi @LinaTeichmann1

I think there was an issue/bug in Eyelink when writing this data to file*.

Unfortunately I’m not sure that there is a quick fix for this file. In the meantime, Is it okay for me to share this file with EyeLink support so that I can at least alert them of the issue?

*If anyone wishes to read on, here's the nitty gritty:

Eyelink writes START and END events to the ASCII file to indicate recording periods (where gaze/pupil data are actually being recorded). For us, this is important because lines in the ASCII file that occur outside these START…END recording sections are unstructured and, IMO, are difficult to parse.

One of the recording blocks in this file is missing an “END” event, resulting in a format like:

START
xpos ypos pupil
xpos ypos pupil
END
...
START
xpos ypos pupil
xpos ypos pupil
...
START
xpos ypos pupil
xpos ypos pupil
END

In contrast, EyeLink’s specification indicates that recording blocks should always be demarcated by “START” and “END” lines, as shown below:

START
xpos ypos pupil
xpos ypos pupil
END
...
START
xpos ypos pupil
xpos ypos pupil
END
...
START
xpos ypos pupil
xpos ypos pupil
END
1 Like
4

Sure, thanks for looking into it! About 25% of my files for this experiment seem to have the same issue. Do you think there is a way to add the END after the last trigger message in this block? I read the data in earlier using the Matlab EDF-converter which worked fine but I was going to move the entire pipeline to Python.

thanks again for your help, much appreciated!
Lina

5

Since these are simple text files, then yes one could, but this sounds very tedious! :slightly_smiling_face:

Here are two potential alternative ideas to get you unblocked while we figure this out:

option 1:

I have this python package that can read eyelink EDF files directly. The package includes a helper function that can convert the loaded object directly to MNE (I can give you directions if the documentation is unclear).

I was able to read in your problematic file with this package. However just a couple caveats to keep in mind:

  • This package requires the Eyelink Software Developers kit to be installed
  • I can’t provide as much support as MNE-Python typically does.
  • I’ve never tested this package on Windows…
option 2

Load the data into .mat structures with the MATLAB EDF-converter as you did before, save the .mat files, load them into python with pymatreader, and then convert them to MNE objects.

  • This option is also a little tedious and will require some custom code (speaking from experience). But since I’ve done this before I could provide you with some boiler plate code to work from.

What do you think?

1 Like
6

Great suggestions @scott-huberty – thank you! I will try next week. Thank you so much for the responses and ideas :slight_smile:

1 Like
7

Ok!

In the mean time, I’ve opened a ticket on Github: Researcher file breaks read_raw_eyelink · Issue #12690 · mne-tools/mne-python · GitHub

8

Hi @LinaTeichmann1

Just an FYI that I raised this issue with SR Research here:

1 Like
9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.