Bugs in down sampling

💬 Support & Discussions
,
1

Hi, I have issue while concatenating the EEG using the following code.
I first realized that there was wierd EEG signals inserted before the EEG recording.
I found that there was a issue in down sampling the EEG data from a edf file (since the original file had a sf of 2000).

Followings are the code that I am using for the concatenation of EEG files.

for filei in range(len(s_filtered_file_names)):

currentEdf=s_filtered_file_names[filei]
edfpath=os.path.join(path, currentEdf)
raw= mne.io.read_raw_edf(edfpath)

# Assuming edfIdx and currentEdfName are defined earlier in your code
# if filei == 1:
#     record_start=raw.info['meas_date']
time_list.append(raw.info['meas_date'])


raw.drop_channels(['POL E', 'POL PG1', 'POL PG2',  'POL TI', 'POL T2', 'POL EOGR', 'POL EKG1', 'POL EKG2', 'POL RESP','POL RESP1', 'POL RESP2', 'POL EOGL', 'POL A32', 'POL DC01', 'POL DC02', 'POL DC03', 'POL DC04', 'POL $A1', 'POL $A2', 'POL A47'], on_missing='warn')
print(raw.ch_names)

new_names = {ch: ch.replace('POL', '').replace(' ', '').replace('EEG', '').replace('$', '').replace('-Ref', '').replace('T7','T3').replace('T8','T4').replace('P7','T5').replace('P8','T6').replace('FZ','Fz').replace('CZ','Cz').replace('PZ','Pz') for ch in raw.ch_names}

raw.rename_channels(new_names)

#down sample all the data
sf=raw.info['sfreq']

if sf>200:
    raw.resample(200)
    sf=raw.info['sfreq']

#sort the channels in order.
targetChannelList =['Fp1', 'Fp2', 'F3', 'F4', 'C3', 'C4', 'P3', 'P4', 'O1', 'O2', 'F7', 'F8', 'T3', 'T4', 'T5', 'T6', 'Fz', 'Cz', 'Pz', 'A1', 'A2']
sort_raw=raw.pick_channels(targetChannelList)

EEG_list.append(sort_raw)

EEG_data = [data.get_data() for data in EEG_list]
concat_data = np.concatenate(EEG_data, axis=1)

concat_raw=mne.concatenate_raws(EEG_list)

if concat_raw.get_data().shape[1]>35460000:
print(‘files are cropped to 48h’)
concat_raw = concat_raw.crop(tmin=0, tmax=172800)

for i in range(len(time_list)):
print(f’{s_filtered_file_names[i]}:{time_list[i]}')

filename=os.path.join(group,filename+‘.edf’)
concat_raw.export(filename, fmt=‘edf’,overwrite=True)

This scaling error happened even when I downsampled the single EEG data.
Would you kindly tell me why this is happening?
Thank you very much for your help.

mneversion: 1.6.0
opereating system WIndows 10

down sampled file

image
image1899×1032 148 KB

original file

image
image1888×1031 214 KB

Atsuro

2

Could you create a more compact reproducible example without unrelated steps (e.g. no loop over a bunch of files, no channel renaming, etc.)? Right now, it’s difficult to say why downsampling is not working, and with a single file we would be able to reproduce it (if you share the file with us).

1 Like
3

Thank you very much.

I finally found that this problem happened when exporting the data in edf.
Down sampling was actually working.

if sf>200:
  raw.resample(200) #when I see this raw the signal is Ok
  sf=raw.info['sfreq']

raw.export(filename, fmt=‘edf’,overwrite=True)

The scaling dramatically changed when I re-upload the new edf.

The file had some problem in the header information, possibly physical maximum (could not read header information), which works under BESA or Persyst but not at EEG lab.

But luckily I could find a solution, to completely clean the EEG (get rid of the non used channels) and run raw.resample and then save it as an edf. Previously there might have been some unused channels remaining before running raw.resample.

This problem also happend when I used mne.concatenate_raws(used in the prior code). (the scale of the out put changes, and there was a wierd EEG segment inserted at the start of the EEG recording)

It was a weird experience so if would be helpful if you could give me the reason.

Thanks.