Issue with Co-Registration when combining MEG and EEG data into a single file

Dear all,

I have two simultaneously recorded MEG and HD-EEG recordings and I successfully combine them in MNE software using the triggers that are sent to both machines via trigger box. The combined .fif file looks correct but I seem to have an issue when I try to display the MRI registration in the brainstorm software. The MEG registration in the combined .fif file matches with the single MEG recording but EEG registration looks like it’s shifted when I compare it with the single EEG file(.mff) I suspect this is due to the fact that EEG and MEG use different coordinate systems. Is there a way to fix this issue in MNE? Any ideas are very much appreciated! Thank you!

pinging @larsoner and @alexrockhill who both know a lot about coregistration and coordinate systems in MNE-Python. In the meantime it would help if you could post an image showing the problem, and also try it with mne.gui.coregistration() to see if it’s different than what you see in brainstorm.

The MEG and EEG coordinate frames are not already aligned in your fif file right? So it doesn’t matter if they are split up and each aligned to the fiducial points separately? That would be what I might try first

You could align each separately and then append them back into one fif file also

Indeed all should work properly if you ensure that your coordinate frame definitions conform to MNE conventions (e.g., the MEG<->head transformation is correct, and the EEG electrodes are in the head coordinate frame). I would also use plot_alignment at various stages to check that things are okay.

@alexrockhill @larsoner Thank you very much for your responses. I don’t think I am performing any kind of alignment in my code. Till now, my code only synchronizes the eeg and meg data points based on the triggers that are sent to both and it also takes care of the linear drift. How may I perform such alignment for coregistration? I have tried mne.viz.plot_alignment but I’m not sure if that is the function I use for alignment.
Note: I also don’t have the MRI for this patient so when I am displaying the coregistration, I am using a template

Alignment is generally done with the coregistration graphical user interface (GUI) @drammock mentioned (mne.gui.coregistration() ) , the viz_plot_alignment just plots the alignment that is already done. If you don’t have the MRI, you could just align to the fiducial points of your template and general headshape (often the alignment of the fiducial points has a bit of room for maneuvering while still being generally on the fiducial points due to measurement error and the inexact nature of measuring fiducial points). On the template, you’re just going to get an approximate alignment anyway.

@alexrockhill Apologies for the confusion. I use the template from the brainstorm MNI: ICBM152 when displaying the coregistration there. I tried to use mne.gui.coregistration, and it did not like it when i only entered my source file, it is asking for an MRI. Is there any way I can download a MRI template for use?

Alignment is generally done with the coregistration graphical user interface (GUI)

This is to align the head and MRI coordinate frames, so actually this won’t help here (no MRI).

You need to make sure that you have properly aligned your MEG data – which is supposed to be in the MEG device coordinate frame – with your EEG data – which is supposed to be in the head coordinate frame. This alignment is given by info['dev_head_t']. What is this value for your data? If it’s an identity matrix, it’s probably but not definitely wrong.

the viz_plot_alignment just plots the alignment that is already done

Indeed and this is useful for debugging even without an MRI and with no trans / no mne coreg / mne.gui.coregistration. If you do some variant of plot_alignment(raw.info, meg=('sensors', 'helmet'), eeg=True, dig=True, show_axes=True, coord_frame='meg') and paste the result here, it will actually visually provide a lot of useful diagnostic information about your setup.

@larsoner I just checked my dev_head_t value. Since I had my EEG channel data added into my MEG, my MEG gets updated. I just did MEG.info[‘dev_head_t’] and it’s not the identity matrix.
Here is the value:

Do you think it’s wrong?

@larsoner This is the output from mne.viz.plot_alignment(MEG.info,meg=(‘sensors’, ‘helmet’), eeg=True, dig=True, show_axes=True, coord_frame=‘meg’)
It doesn’t look correct to me. It seems like my EEG coordinates are shifted downwards.

It doesn’t look correct to me. It seems like my EEG coordinates are shifted downwards.

Your dev_head_t has a Z transformation of 0.083, which is quite large. This is consistent with the blue (MEG) and red (head) axes being separated quite a bit vertically. It suggests that something about how you digitized the LPA/Nasion/RPA during MEG acquisition, and/or how you applied the montage to the EEG data when adding it, is inconsistent.

If you can anonymize your data and crop the raw instance to 1 sec and upload it somewhere that would help. Even better if you can do this to the MEG instance alone and the EEG instance alone (before or after the temporal alignment step is fine since here we’re just looking at coordinate frames, not actual data) before doing raw.add_channels. With that, I or someone else could take a look at the data structures to figure out what’s going on.

I would start looking at the instances by themselves and the data they contain in terms of digitization and coordinate frames, with plot_alignment calls on each instance by itself to look and see how the digitization points and sensors align.

I shared MEG and EEG data with you via personal email.

With some quick code to sanity check before any combination:

import mne
raw_meg = mne.io.read_raw_fif('MEG_temp2.fif', verbose='info')
raw_eeg = mne.io.read_raw_fif('EEG_temp2.fif', verbose='info')
kwargs = dict(show_axes=True, dig=True, meg=('helmet', 'sensors'), eeg=True)
mne.viz.plot_alignment(raw_meg.info, coord_frame='meg', **kwargs)
mne.viz.plot_alignment(raw_eeg.info, coord_frame='head', **kwargs)

The MEG file has something that looks correct (head in a reasonable position, digitization lining up with axes):

But the EEG file has something that looks wrong (head coordinate frame not in the center defined by LPA/Nasion/RPA:

So I think the root of the problem is that however you set the montage for your EEG system, it did not correctly define the LPA/Nasion/RPA (or convert to head coordinates when doing raw.set_montage), which is necessary when you eventually combined the instances.

How did you set the electrode locations to create EEG_temp2.fif (or the uncropped version of it) from the MFF recording?

That might be the issue because to create the fif file from the mff format, I have only done the following:
I only read the raw file using:
EEG= mne.io.read_raw_egi(eeg_filename2, preload=True)
and then I just cropped it to a shorter length and did EEG.save(“something.fif”)
I have limited knowledge about this since I am a computer engineering student, so I didn’t know that I have to perform coordinate operations.

I think we have some bug, see BUG: EGI coords read but not transformed ¡ Issue #9250 ¡ mne-tools/mne-python ¡ GitHub. But first it would be good to see LPA/Nasion/RPA show up for your data (colorful spheres), like in:

mne.viz.plot_alignment(mne.io.read_raw_egi('test_egi.mff').info, show_axes=True, dig=True, eeg=True, coord_frame='head')

Are you on 0.22? If you use latest main do you see LPA/Nasion/RPA 0.

here is my output and there is no LPA or Nasian defined.

and my MNE version is 0.22.0

Can you try latest main to see if you see LPA/Nasion/RPA?

Or even better, try on BUG: Fix coordinate frame by larsoner ¡ Pull Request #9251 ¡ mne-tools/mne-python ¡ GitHub and things might work completely