Export by-subject epochs as a pandas dataframe/csv with all metadata

đź’¬ Support & Discussions
,
1

I would like to export subjects’ epochs as separate csv files, so that I could import them and run some stats in R. I know that the method to_data_frame() is able to convert epochs into long-format pandas.

epochs = mne.Epochs(raw_highpass, raw_events, event_id, -.1, .8, metadata=metadata, baseline=(-.1, 0), preload=True, reject=None, detrend=0, reject_by_annotation=False)

epochsPd = epochs.to_data_frame(long_format=True)
epochsPd.to_csv('~/Desktop/epochsPd.csv')

I would also need export epoch file to store the metadata associated to the mne.epochs object. As far as I could understand from the wiki, the metadata won’t be stored as part of the dataframe. I guess I can just left-join metadata to the epochsPd by calling pandas.concatenate(), but I was wondering if there was a way to do it within the to_data_frame() method.

Thanks!

2

Hello, honestly I would just keep metadata and Epochs data in two separate tables, since metadata only makes sense per epoch, whereas Epochs.to_data_frame() produces one row per sample. You would have to endlessly duplicate the metadata if you joined the two tables, which I don’t think is a great idea…

3

You’re right! :man_facepalming:

Well, the metadata contain some of the variable I will need to run linear regression, so I definitely need those to be matched to each epoch…

4

For those interested in the issue, here’s a solution.

Your Epochs object will retain all metadata (as long as you’ve defined the argument metadata when epoching) of all non-bad epochs. So you can extract the metadata and make it a pandas dataframe and then merge it to the epochs dataframe.

epochsPd = epochs.to_data_frame(long_format=True)

goodWords = epochs.metadata.reset_index() #reset the index to be able to refer to it when merging it with the epochs pandas
epochsPd = pd.merge(epochsPd, goodWords, left_on='epoch', right_index=True)

The merge method will match each row index of the goodWords dataframe with the value of each epoch as stored in the epoch column in the epochsPd.

1 Like
5

Hi,
Thank you for the entry and the solution! I am also trying to save epochs data with its metadata to run lmm. It was quite helpful, but I couldn’t get what pd is. It would be great if you explain.

epochsPd = pd.merge(epochsPd, goodWords, left_on='epoch', right_index=True)

6

@gozemturan
pd is pandas.
import pandas as pd

pd.merge is a dataframe merge - in this case epochsPd and goodWords

-Jeff

3 Likes