Mne raises error while reading .dat format EEG data

[semicolon123 (Swati)]

• mne==0.24.1
• operating system: Windows 10 (Google colab)

I have EEG data in .dat format. According to MNE website, I used the code to read the file but it threw the following error.

import os

import mne

sample_data_folder = '/content/drive/MyDrive/data/data_preprocessed_python'

sample_data_raw_file = os.path.join(sample_data_folder,'s01.dat')

raw = mne.io.read_raw_persyst(sample_data_raw_file, verbose=False)

The output error is:
FileNotFoundError: The path you specified, “s01.dat.lay”,does not exist.

I dont know why the code is looking for “s01.dat.lay” whereas I have mentioned “s01.dat”.

Kindly let me know if i am missing anything.
Thanks!

[larsoner (Eric Larson)]

@semicolon123 can you share the data? If so someone might be able to take a look. It might just be a case where whoever implemented Persyst support always had a .lay file to get the electrode positions, and we need to make that optional rather than mandatory

[semicolon123 (Swati)]

Hello @larsoner

Thank you for your reply. Unfortunately i cannot share the patients data in any form outside.
I hope you guys can replicate the issue somehow.

Let me know.

[mmagnuski (Mikolaj Magnuski)]

Hi @semicolon123,
I think you give wrong file name to the function. According to the docs the fname should be “Path to the Persyst header (.lay) file.” - thats why you get “s01.dat.lay”,does not exist.. If you instead do:

mne.io.read_raw_persyst(os.path.join(sample_data_folder, 's01.lay'), verbose=False)

or

mne.io.read_raw_persyst(os.path.join(sample_data_folder, 's01'), verbose=False)

it should work.

[semicolon123 (Swati)]

Hello @mmagnuski

Thank you very much for your suggestions.
I tried both ways that you mentioned. Unfortunately, it still did not work.
Just to let you know my file type is ‘.dat’. I followed their documentation to read a .dat file.

Let me know if you can think of some other solution.

[mmagnuski (Mikolaj Magnuski)]

@semicolon123

Unfortunately, it still did not work.

What error do you see, could you paste it?

@semicolon123

Just to let you know my file type is ‘.dat’.

Are you sure this is the persyst data fromat? If you don’t have a .lay file then it might not be persyst (I don’t know, because I never used that format). Maybe you have a brainvision file? Then you would have to use a different reading function.

[semicolon123 (Swati)]

The error was:

FileNotFoundError: fname does not exist: /content/drive/MyDrive/data/data_preprocessed_python/s01.lay

And this error is obvious since there is no such file named “s01.lay” (Actual file that exists is “s01.dat”)

The data format is .dat & that is all I am aware of. So, naturally, I used the persyst function as mentioned in MNE documentation here:
https://mne.tools/stable/auto_tutorials/io/20_reading_eeg_data.html#persyst-eeg-data-lay-dat

I am not sure if I am missing something or if some system has been updated which is yet to be changed in MNE package.

[larsoner (Eric Larson)]

@semicolon123

The data format is .dat & that is all I am aware of.

Can you ask the person/people that you got the data from about how the file was acquired? It’s pretty important to know how the file was made (acquisition machine, processed afterward, etc.). See the FieldTrip docs where they mention .dat in particular as being problematic because it’s used by multiple software packages:

https://www.fieldtriptoolbox.org/development/module/fileio/

[semicolon123 (Swati)]

Hello,

This is the website of the dataset. I couldn’t find how the file was made except the description of file. Anyway, I will try to contact the authors.

[larsoner (Eric Larson)]

They link to a paper:

tac_special_issue_2011.pdf

It says in the methods section:

EEG and peripheral physiological signals were recorded using a Biosemi ActiveTwo system4 on a dedicated recording PC (Pentium 4, 3.2 GHz).

So I don’t think these are Persyst files, but rather Biosemi ones. I guess we don’t have a reader for that… any suggestions @mmagnuski ?

[mmagnuski (Mikolaj Magnuski)]

Hmm, I think I worked with biosemi data some time ago, but they were in bdf format so I was able to read them in mne (Importing data from EEG devices — MNE 0.24.1 documentation). I never used biosemi .dat format.

[mmagnuski (Mikolaj Magnuski)]

@semicolon123 , I took a look at the data website that you linked. In the data description section they say that the raw data is in bdf format. Only the preprocessed files are in .dat but they provide the following example on how to read them:

import cPickle
x = cPickle.load(open('s01.dat', 'rb'))

[semicolon123 (Swati)]

Yes, I know that.
But I was wondering if I could use the MNE method to read the .dat file directly so as to create the object.

So, in case I will use the code below to read the files, I believe I will have to use some MNE method to convert the 3d array to mne object for further analysis

import cPickle
x = cPickle.load(open('s01.dat', 'rb'))

[cbrnr (Clemens Brunner)]

@mmagnuski

Hmm, I think I worked with biosemi data some time ago, but they were in bdf format so I was able to read them in mne (Importing data from EEG devices — MNE 0.24.1 documentation). I never used biosemi .dat format.

FYI BioSemi uses only BDF to record data, for which of course we do have a reader.

[semicolon123 (Swati)]

Yes, you are right that BioSemi uses BDF to record data. But the patient’s data that has been provided in .dat format is preprocessed & they even re-ordered some readings. Hence, I want to read .dat file instead of reading .bdf & doing the preprocessing myself.

While I can see there is a reader for .bdf files, I cannot seem to get hold of MNE method to read a .dat file.

[semicolon123 (Swati)]

Essentially, you mean that I cannot read Biosemi’s .dat files using the MNE method. Any chances they may release such a reader?

[cbrnr (Clemens Brunner)]

@mmagnuski already suggested what you can do. The data will likely be available as a NumPy array after loading the pickled file, which you can then convert with RawArray (see Creating MNE-Python data structures from scratch — MNE 0.24.1 documentation and Creating MNE-Python data structures from scratch — MNE 0.24.1 documentation).

[semicolon123 (Swati)]

Got it. Thank you Clemens Brunner, @mmagnuski and @larsoner

You guys have been patient to resolve my query. I will try these suggestions on in a day or two & get back to you:)

Thank you!

[jeetrico (Jeet Banerjee)]

Can anybody help me to read the output of that EEG signal (.dat) file through python!!!

I am attaching my testing code as well as the output. The desired result or output is completely different than my code’s output. Please any python developer or expert whoever has the knowledge of this kind of reading of datas through .dat file contains EEG signals, please help!!! Really Urgent!!!

My code —>
import numpy as np
import mne
import matplotlib.pyplot as plt

Load EEG data

with open(r"C:\Users\jeetr\Downloads\EEG_template.dat", ‘rb’) as f:
data = f.read()

data_1 = np.frombuffer(data, dtype=np.float32) # convert bytes to float
data_1 = np.reshape(data_1, (1, -1)) # reshape to

2D array

Check for NaN values

nan_indices = np.isnan(data_1)
nan_count = np.sum(nan_indices)
print(“Number of NaN values:”, nan_count)

Check for Inf values

inf_indices = np.isinf(data_1)
inf_count = np.sum(inf_indices)
print(“Number of Inf values:”, inf_count)

Create MNE RawArray object

sfreq = 255
ch_names = [‘EEG 1’]
info = mne.create_info(ch_names, sfreq, ch_types=‘misc’)
raw = mne.io.RawArray(data_1, info, first_samp=1, verbose=5)

Calculate the number of samples corresponding to 5 seconds

num_samples_5_sec = 5 * sfreq

Plot EEG signal

plt.figure(figsize=(10, 6))
plt.plot(raw._data[0][:num_samples_5_sec])
plt.xlim(0, num_samples_5_sec) # Limit x-axis to 5 seconds
plt.xlabel(‘Time (samples)’)
plt.ylabel(‘EEG Signal’)
plt.title(‘EEG Signal (First 5 Seconds)’)
plt.grid(True)
plt.show()

[wmvanvliet (Marijn van Vliet)]

The problem is that the need to know more about the file other than that the filename ends in .dat. The .dat stands for just “data” so that doesn’t tell us very much about what is inside the file and how to read it. In the case above, the .dat file was a pickled python file and could be loaded as a numpy array using pickle.load, but I have no idea whether that would be the case for you. So, any details you can provide about where this file came from and how it was made would be helpful.