df.to_csv is taking so long

đź’¬ Support & Discussions
1

Hi all,

I am trying to export my epoch data frame as csv file to analyze it later on with R. The functions to_data_frame() and to_csv() give what I want, but it takes ages. I was wondering if there is a better and faster way to do it. I am using version 0.24.1 and I have already tried with Windows 10 and Ubuntu 18.04.

Thanks in advance for your help!
Gözem

2

hi,

.csv is a really inefficient file format. It’s slow to read, write and produces too big files.

you should use something else that can be read in R, eg parquet which is a modern format

Alex

3

To add another suggestion, Feather is a lightweight and super-fast data format that works well with Python and R:

4

I’d suggest to save the DataFrame to HDF5: