# IOError: 2GB file size limit reached. Support for larger raw files will be added in the future. **URL:** \ **Category:** Mailing List Archive (read-only)\ **Tags:** list-archive\ **Created:** [March 19, 2014, 8:52am UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697 "2014-03-19T08:52:21Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 8:52am UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/1 "2014-03-19T08:52:21Z")
Dear MNE developers Just out of curiosity, is this going to happen in the near future, or is it a greater challenge than I imagine? Best Lau ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 3:33pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/2 "2014-03-19T15:33:27Z")
It is challenging because the FIFF file format is not designed to handle files larger than that. Adding support for it would require redesigning and rewriting the I/O code fundamental to data analysis, and would likely break backward compatibility with many tools. AFAIK nobody is actively working on the issue. There are different ways of working around the issue. If you have a specific use case, you could open an issue on the `mne-python` Github site, and perhaps we can provide some ideas. Ideally we could document the workarounds somewhere in case people hit this issue in the future, too. Eric ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 3:37pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/3 "2014-03-19T15:37:25Z")
hi Lau, > Just out of curiosity, is this going to happen in the near future, or is > it a greater challenge than I imagine? I am not sure we have the man power to tackle this now. It would also require to change the C FIF library which is not trivial. In the mean time you need to split the files. Best, Alex ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 6:56pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/4 "2014-03-19T18:56:38Z")
Dear Eric and Alex Thanks for your answers. I now understand that it is not a simple issue. The files themselves are not greater than 2.0 GB, but because I have multiple recordings per subject, I read them in as a list. The only reason that I ask is that I would like to save the filtered file (i.e. the one that is greater than 2.0 GB, because it is a list of raw files). The way I work around it now, is simply to save the epoched files and working from there. The only annoying thing is, if/when I realize that I need to do something different for creating the epochs, then I need to do the time-consuming process of filtering again. But I can live with that. Thanks again for a wonderful python toolbox. Best Lau ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 7:41pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/5 "2014-03-19T19:41:14Z")
Hi folks, it just occurs to me that we might want to think about adding a Raw.split method. As a first quick and dirty solution, Lau, you might want think about a function that splits the raw object and saves each split to fif files. Something like this: def split\_save\_raw(raw, splits , fname\_pattern):     for i\_split, (tmin, tmax) in enumerate(splits):         raw.copy().crop(tmin, tmax).save(fname\_pattern % i\_split) split\_save\_raw(raw, [(0, 1), (1, 3), (3, 4)], 'my-split-%i-raw.fif') Re-construction should then be trivial. Cheers, Denis ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 7:44pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/6 "2014-03-19T19:44:04Z")
Thanks, Denis That definitely looks useful. I will try to implement that into the code tomorrow. Thanks again ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 7:45pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/7 "2014-03-19T19:45:39Z")
I suggest the raw.save splits files automatically like the C code. Alex ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 8:24pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/8 "2014-03-19T20:24:22Z")
Hello Denis et al., I just confirm what Eric already said: it's not easy to get rid of the 2-GB file size limit as it is due to the fact the FIFF format internally uses 32-bit file pointers. Switching to 64-bit pointers would not be backwards compatible. The limit is an issue for Elekta, too, and they are looking for a good solution but I don't know about the time frame. In the meantime we could perhaps make use of the "file chaining" FIFF tags that allow logical linking of consecutive files. If the MNE FIFF i/o functions could use these tags and split/assemble files accordingly, reading and writing data larger than 2 GB could be made transparent to the user. Comments? Cheers, Lauri ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 8:34pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/9 "2014-03-19T20:34:11Z")
you can also set the dtype when saving back the raw filtered data. You can use float32 to have half size files or even float16 HTH A ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 9:02pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/10 "2014-03-19T21:02:15Z")
+1e15 I had the same thought. ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 9:22pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/11 "2014-03-19T21:22:22Z")
16-bit floats? I do not know about such a thing. at any rate, do not use 16 bit integers, precision is most likely lost. Also, add the link tags to the files like the C code does. - Matti ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 19, 2014, 10:25pm UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/12 "2014-03-19T22:25:54Z")
Thanks, Lauri --- that would mean, using such link tags, we could load the entire chain using a single filename? That sound like the best we can do given difficulties mentioned. Sounds clean to me. Denis ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [March 20, 2014, 7:47am UTC](https://mne.discourse.group/t/ioerror-2gb-file-size-limit-reached-support-for-larger-raw-files-will-be-added-in-the-future/697/13 "2014-03-20T07:47:47Z")
I created the issue so we don't forget [https://github.com/mne-tools/mne-python/issues/1194](https://github.com/mne-tools/mne-python/issues/1194) best, Alex