Importing raw data from FieldTrip [Distorted data]

I’m trying to import MEG data from fileldtrip. I’m reading raw/mhd file in Matlab and exporting raw data to fif file:


cfg = [];
cfg.dataset = infile;
hdr = ft_read_header(infile);
bad_channel = find([hdr.orig.ch.flag]==1);
good_channel = setdiff(find([hdr.orig.ch.type]==2),bad_channel);
cfg.channel = [hdr.label(good_channel)];
cfg.continuous = 'yes';
data = ft_preprocessing(cfg);

fiff_file  = 'data_raw.fif';
fieldtrip2fiff(fiff_file, data)

Then I’m reading fif file in MNE:


fname_fif = PATH_fif
raw_fif = mne.io.read_raw_fif(fname_fif)

print(raw_fif.info)

raw_fif.plot()
plt.show()

The file is read (number od channels is right, sampling rate etc) however voltage time traces are completely messed up. Instead of analog signal there is a binary-like signal:

Hi Tomasz,

have you plotted your data in Fieldtrip/MATLAB before exporting? Does is still look okay there?

Best,
Britta

Hi Britta,
thank you for help. Yes, I plotted it in Matlab, MAG channels look normal. Small update: I was able to import part of data through mat. file to Python by using this function:

import h5py

def read_matlab(filename):
    def conv(path=''):
        p = path or '/'
        paths[p] = ret = {}
        for k, v in f[p].items():
            if type(v).__name__ == 'Group':
                ret[k] = conv(f'{path}/{k}')  # Nested struct
                continue
            v = v[()]  # It's a Numpy array now
            if v.dtype == 'object':
                # HDF5ObjectReferences are converted into a list of actual pointers
                ret[k] = [r and paths.get(f[r].name, f[r].name) for r in v.flat]
            else:
                # Matrices and other numeric arrays
                ret[k] = v if v.ndim < 2 else v.swapaxes(-1, -2)
        return ret

    paths = {}
    with h5py.File(filename, 'r') as f:
        return conv()

This returns Python dictionary with keys corresponding to MAG parameters, arrays etc. However to import this to MNE seems to be Sisyphean task…

Mh, I guess that leaves fieldtrip2fiff or read_raw_fif as possible culprits. My guess (really, just a guess!) is that the filetype raw.mhd is maybe not providing all information necessary for writing a valid fif-file.
If you would want to test further: have you tried reading the .fif in again with FieldTrip and if so, does it look okay there?
If it does, then we might look at a potential bug in MNE, and could take further steps to solve it.

As acute help: are you aware of mne.io.RawArray — MNE 1.3.1 documentation ?

1 Like

Good idea with reading fif in fieldtrip. Thanks :slight_smile:

I checked how fieldtrip reads saved .fif file and it looks correct:


cfg_fif = [];
cfg_fif.dataset = fiff_file;
hdr = ft_read_header(fiff_file);
cfg_fif.continuous = 'yes';

data_fif = ft_preprocessing(cfg_fif);

Plot for first channel:

Dear Britta,
you mentioned mne.io.RawArray. How to construct MEG raw file with sensor locations, orientations etc. with it?

Hi Tomasz,

mh, mh, mh. In a coincidence, I was just talking about fieldtrip2fiff with a FieldTrip developer in a different context - the function is probably not quite equipped to handle any raw data but only very common ones (e.g. CTF data etc). But curious that FieldTrip reads the data back correctly. Could still be that there is some aspect of the data that FieldTrip just ignores, but that trips MNE-Python.

Any chance you can share a snippet of the data? Just a few seconds worth? I have never worked with mhd data, so I am just left at guessing here.
If you can share some data, I can give it a look - unless @larsoner might have already a strong idea what is going on?

I actually remembered another function that might be even of more help for you here - check out mne.io.read_raw_fieldtrip — MNE 1.3.1 documentation
that should read from a .mat file. Although, disclaimer, it might be a roundtrip into the same problem, as there is the following info: FieldTrip does not normally store the original information concerning channel location, orientation, type etc. It is therefore highly recommended to provide the info field.

Lastly, before we dive into a potential debugging adventure: I assume you make the detour via FieldTrip because MNE cannot read the raw data directly?

1 Like

You might just be seeing the bit depth of the file (original or round-trip) because your data channel values appear to be on the ~1e4 range (based on your MATLAB plot) and your data are treated as magnetometers which are plotted in fT (1e15 scale factor). If you call raw.plot(scalings=dict(mag=1e4)) or so (and try other values like 1e2, etc.) does it look more reasonable?

If so, a raw.rename_channels then raw.set_channel_types(...) to set them to HbO/HbR properly before raw.plot you result in a plot with scale factors that make more sense.

2 Likes

Thank you Britta, yes I’m exporting fif / mat files from fieldtrip because I cannot read it directly in MNE. Raw / .mhd is from ITAB lab (Chieti): MEG file formats - Brain Imaging Data Structure v1.8.0. I think I can share excerpt of data. Of course it would be great if we could debug MNE reader.

In worst case, or just for now, I can import numpy arrays (timecourses, placements and orientations of sensors etc.) to mne.Raw. However for that I need a detailed knowledge about data structure. If someone can link documentation that would help me a lot.

Thank you! Scaling really helped (raw.plot(scalings=dict(mag=1e4)). Interesting, I would never think it can be the problem of scaling.

What can still be a problem is metadata: the orientations and placements of MEG sensors. So my questions about the data structure of mne.Raw can be still relevant.

Thank you both for help!

PS. Scaling factor has to be 1e4, for smaller scaling plots are discontinuous.

PS2. MEG sensor locations seems to be well imported.

Hi Tomasz, given that the ITAB format is quite local - do you happen to have an estimate of how many people use this format and would like to use MNE-Python?

Hi Britta,
good idea to know that. I hope I will have some estimate next week.

1 Like