Issue with Co-Registration when combining MEG and EEG data into a single file

I tried the exact code from the link that mentions the bug:

mne.viz.plot_alignment(mne.io.read_raw_egi(mne.datasets.testing.data_path() + '/EGI/test_egi.mff').info, show_axes=True, dig=True, eeg=True, coord_frame='head')

but it still doesn’t show the 3 colorful spheres even though it was supposed to.

but it still doesn’t show the 3 colorful spheres even though it was supposed to.

This suggests you are probably not on the latest main branch, but instead still 0.22. You need to be on main branch to see the spheres. For example to uninstall 0.22 and use latest main you can do:

$ pip install https://github.com/mne-tools/mne-python/archive/main.zip

But even better would be to be on my branch from the PR I linked above:

$ pip install https://github.com/larsoner/mne-python/archive/mff.zip

Then everything might even show up in the right place!

I downloaded the version on your branch. The three colorful spheres are showing up right now but my electrodes are not. I tried both the testing_egi.mff and also my recording and both of them gave the same output. Is there another bug? because In the meantime, I am going to install the latest stable version try on that.

I just tried it on the latest stable version(0.22.1) and the old issue(no colorful spheres) persist. It seems like stable version doesn’t provide LPA/Nasion/RPA and the dev version doesn’t provide my electrodes.

I tried both the testing_egi.mff and also my recording and both of them gave the same output. Is there another bug?

Maybe it’s a rendering bug, but if the fiducials show up, the electrodes should, too. This is with eeg=True, right?

If you just do raw.plot_sensors do the sensors show up in reasonable locations?

On my branch for me this code:

import mne
raw = mne.io.read_raw_egi(mne.datasets.testing.data_path() + '/EGI/test_egi.mff')
mne.viz.plot_alignment(raw.info, show_axes=True, dig=True, eeg=True, coord_frame='head')
raw.plot_sensors()

produces:

1 Like

Yes, this is for eeg=true. And when I do raw.plot_sensors() on test_egi.mff, I see the sensor locations just fine like in yours but in the plot_alignment, they don’t show up and I just see 3 fiducials. Would this be related to the mne dependencies? Here are my installed dependencies and their version.

Would this be related to the mne dependencies?

It could be a 3D backend problem, or a problem rendering 3D in general on your system. If you do mne.set_3d_backend('mayavi') and then do the plot_alignment call, does it work?

I’m also a bit confused since plotting sensor locations worked for you earlier, so there is something weird going on.

In any case, it might be worth trying your original code to combine instances while on my branch with the coordinate frame fix (mff), and see if plot_alignment there looks correct. I’m hoping it will!

No, it did not work and python got stuck when used set_3d_backend(‘mayavi’) but regardless, I did the EEG and MEG combination in your branch and did the plot alignment and it looks different than what I had before. It seems like it might have fixed it but the EEG electrodes are not there so I would like to ask for your opinion since you are an expert on this.

Can you crop the raw EEG data processed on my branch (so that electrodes are created properly) and upload it / email it again? That way I can check to see if it’s an issue with your rendering or the electrodes.

I have shared the new data via your personal email.

With this code:

import mne
from collections import Counter
raw_meg = mne.io.read_raw_fif('MEG_temp2.fif', verbose='info')
raw_eeg = mne.io.read_raw_fif('EEG_testnbranch.fif', verbose='info')
for key in ('dev_head_t', 'acq_pars', 'experimenter', 'description',
            'highpass', 'lowpass', 'line_freq', 'proj_name', 'gantry_angle'):
    raw_eeg.info[key] = raw_meg.info[key]
print(len(raw_eeg.info['dig']))
raw_meg.info['dig'] += raw_eeg.info['dig'][3:]  # not LPA/Nasion/RPA
raw_eeg.info['dig'] = None
kwargs = dict(show_axes=True, dig=True, meg=('helmet', 'sensors'), eeg=True)
raw_meg.load_data().add_channels([raw_eeg.load_data()])
mne.viz.plot_alignment(raw_meg.info, coord_frame='meg', **kwargs)
print(Counter(d['kind'] for d in raw_meg.info['dig']))
# Counter({4 (FIFFV_POINT_EXTRA): 507, 3 (FIFFV_POINT_EEG): 257, 2 (FIFFV_POINT_HPI): 5, 1 (FIFFV_POINT_CARDINAL): 3})

I now get:

There are probably some simplifications we should make at the MNE end since you shouldn’t have to mess with most of those entries to merge the two instances. And the alignment isn’t perfect here, probably because template EEG locations are being used. But this still seems like some reasonable progress!

Feel free to open an issue about improving / simplifying the add_channels code such that FIF-MEG+MFF-EEG data are easier to combine

@larsoner I tried the exact code and here is the output I’m getting. The overall result looks much better right now but my output figure is still missing the EEG electrodes.(FYI: I’m running your branch on my macOS device)

this is very weird. One thing you could try is to create a fresh conda environment to reduce the chances of a dependency or installation problem:

curl --remote-name https://raw.githubusercontent.com/mne-tools/mne-python/master/environment.yml
conda env update --file environment.yml --name mne_test
conda activate mne_test
pip install https://github.com/larsoner/mne-python/archive/mff.zip

Then, in the same terminal window, start ipython and run the code that larsoner said was working for him with your data.

@ometett it just occurred to me that you might be hitting the same issue as in WIP,FIX: paintEvent/resizeEvent by GuillaumeFavelier · Pull Request #9313 · mne-tools/mne-python · GitHub. If you take the window and rotate the view with your mouse, do the electrodes then show up? I think we might have some 3D updating problem.

Dear @larsoner @drammock
I apologize for the late response. I have done a fresh update and updated MNE to the latest version 0.23 and ran the code that you gave me above. The EEG electrodes are still not appearing and also when I move around with my mouse electrodes still don’t show up. @larsoner I can send you my code if you would like to take a look at it? I don’t understand why the electrodes don’t appear when I run the exact same code on my computer. I’m starting to think maybe it’s related to my OS. (I’m using macOS)

Coming back to our minimal example:

import mne
raw = mne.io.read_raw_egi(mne.datasets.testing.data_path() + '/EGI/test_egi.mff')
mne.viz.plot_alignment(raw.info, show_axes=True, dig=True, eeg=True, coord_frame='head')

On macOS I get:

@ometett can you just re-confirm that you do not get this? I assume you see only the fiducial (R/G/B) points?

I am really confused how the plot can work for all points except for the last ones added (EEG electrodes). It looks like you have “OpenGL 4.1 Metal …” something – can you copy-paste the output of the pyvista line of your mne sys_info? Mine is:

pyvista:       0.30.dev0 {pyvistaqt=0.5.dev0, OpenGL 4.1 INTEL-16.2.16 via Intel(R) Iris(TM) Plus Graphics OpenGL Engine}

I suspect it’s the Metal vs INTEL graphics difference. The one thing that might be causing problems is the alpha – if you change this line in your mne/viz/_3d.py (for me it’s line 996; if you want to know where the mne files live on your system do print(mne.__file__) which should say something like /Users/larsoner/python/mne-python/mne/__init__.py):

    alphas = [0.8,

to

    alphas = [1.0,

does it work? If so it’s a translucent geometry rendering problem with your graphics card. One workaround would be that we could add a opacity argument to plot_alignment that took keys like 'eeg', so for example you could do plot_alignment(..., opacity=dict(eeg=1.)).

@larsoner my pyvista output is:
pyvista: 0.30.1 {pyvistaqt=0.4.0, OpenGL 4.1 Metal - 71.5.9 via Apple M1}

The weird thing is that when I load the test_egi.mff, it shows the locations but when I load my recording it does not. I changed the alphas from 0.8 to 1.0 just like you suggested, and rerun my script but I still get the same issue. The first image is the output from test_egi.mff which is the same as what you get, and the other image is the output from my .mff recording. When load this recording in brainstorm I see the electrodes nicely.


On my macOS machine, using this code:

import mne
raw_eeg = mne.io.read_raw_fif('EEG_testnbranch.fif')
fig = mne.viz.plot_alignment(
    raw_eeg.info, coord_frame='head', show_axes=True, dig=True, eeg=True)

And the file you sent to me, I see electrodes:

It is even weirder that this works for you on the testing dataset but not your own :frowning:

This is insane. I use the exact same code and I am not getting the electrodes. Do you think this somehow related to the fact that I am using M1 chip macbook? I can also try to install the same versions for the plugins like mayavi etc if you can send me screenshot of your mne sys_info

I am very convinced that it’s related to my computer having M1 and not INTEL because nothing else is different between our runs.