write_raw_bids does not rename all data files of the .ds folder

💬 Support & Discussions
1

Hi,
I have encountered an issue when converting my CTF (.ds) MEG data with the mne_bids.write_raw_bids function into BIDS format.
when copying the .meg4 files, the function only renames the first of several MEG data files according to the BIDS naming rules.
I have figured out that the issue lies in the function copyfile_ctf in copyfiles.py (of mne_bids), where certain file extensions are specified, that are to be renamed.

file_types = (
        ".acq",
        ".eeg",
        ".dat",
        ".hc",
        ".hist",
        ".infods",
        ".bak",
        ".meg4",
        ".newds",
        ".res4",
    )

The problem is that the non-renamed files of the .ds folder have the extension “.1_meg4”, “.2_meg4”,…These are all parts of one 1:10h long dataset.
The issue can be solved for me by changing “.meg4” to “meg4” in this mne_bids function. Do more people encounter this issue and should it maybe be changed permanently in the mne_bids code?

Thank you for your work!

Best,
Mara

2

Hi @Mara that sounds like something we can fix in the codebase of mne-bids. Would you be comfortable opening a PR there with changes to the codebase that fix your problem?

Are the ".{integer}_meg4" extensions the only problem?

What’s the highest {integer} that you encounter in your dataset?

Do you know why these files are named that way? … is it because the dataset is particularly long?

3

Great, yes, I will open a PR.

For me, the ".{integer}_meg4" extensions are the only problem, when I rename them, I can re-open the complete dataset and work with it normally.

However, there are certain files within the .ds folder where I’m uncertain about their conformity to BIDS standards that are also copied and not renamed (MarkerFile.mrk, ClassFile.cls, ChannelGroupSet.cfg, BadChannels.exe, and bad.segments.txt).

The highest integer I encounter in my dataset is 4. The number of files depends on the length of the dataset. However, the datasets are not particularly long, 1.5h maximum. The acquisition software saves the continuous recording in chunks of 1,89 GB each. MNEs read_raw_ctf and read_raw_bids functions naturally concatenate them to one dataset without further do when the .ds folder is specified (but only if they have consistent naming).

In the class RawCTF in init they solve it like this:

        # Determine how our data is distributed across files
# [...]
        while True:
            suffix = "meg4" if len(fnames) == 0 else ("%d_meg4" % len(fnames))
1 Like
4

I see, thank you! In my opinion we should simply add to the list of file_types a number of .{integer}_meg4 “extensions” … up to a reasonable {integer} (maybe 20?). And then add some tests to mne-bids … as well as a potential check of a dataset in case there is a .21_meg4 file so that we catch such a case and warn (including a notification to us devs that we should extend our list of acceptable file_types).

I’m uncertain about their conformity to BIDS standards that are also copied and not renamed (MarkerFile.mrk, ClassFile.cls, ChannelGroupSet.cfg, BadChannels.exe, and bad.segments.txt).

I am not an expert in the CTF file format, so I am always happy to come across people using this file format and willing to contribute to both mne-bids and BIDS itself – so please feel free to point out anything in mne-bids or the BIDS spec that you find weird and/or underspecified.

5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.