Resolving EGIMFF/mffpy Refactor Blockers

Support & Discussions
1

I wanted to share a significant progress update regarding Project 3: Refactoring EGIMFF to use mffpy. Over the last few days, I have been auditing the scott/mffpy branch (#12981) to identify why the test suite was stalled at 18 failures.

Current Status: 15 passed / 8 failed (Improved from 5 passed initially).

Key Fixes:

  1. Broadcasting & ValueError Fixes: I identified that _get_eeg_data was allocating buffers for all channels but only populating EEG chunks. I refactored the indexing logic to be “chunk-aware,” which resolved the primary ValueError crashes.
  2. Digitization Accuracy (137 → 132 points): By auditing the .xml files, I found that cardinal fiducials were being misclassified. I refactored _get_montage to pull from coordinates.xml, correctly identifying “Nasion,” “LPA,” and “RPA”.
  3. Numerical Stability: I resolved the 1-sample drift and “float interval” errors by switching to integer-based sample counts derived from actual chunk dimensions.
  4. Metadata & Datetime: I implemented a sanitizer for mffpy’s 9-decimal timestamps and restored the utc_offset logic, allowing all meas_date tests to pass.

image
image837Ă—533 53.6 KB

I am now focusing on the remaining 8 failures:

Refactoring for Lazy Loading: Currently, the reader always preloads, which fails architectural expectation tests.
PNS Support: Re-integrating physiological channels into the info structure.

Question for Mentors: Given that I have locally resolved the primary blockers from the previous WIP branch, should I proceed with opening a superseding PR to main now, or would you prefer I continue the cleanup on my fork first?

1 Like
2

it would be good to get those other failures sorted out, but there’s no harm in opening what you have so far as a draft PR. That makes it a bit easier to, for example, ask for help on a specific line of code where tests are failing.

1 Like
3

Thanks for the guidance! I’ve opened the superseding PR and moved it to Ready for Review:

PR: MRG, ENH: Refactor EGIMFF reader to use mffpy (Supersedes #12981) by PragnyaKhandelwal · Pull Request #13684 · mne-tools/mne-python · GitHub

I’ve resolved the remaining implementation issues locally, including:

PNS channel integration
Lazy-loading architecture requirements
Strict mffpy dependency behavior and corresponding test handling
Huge thanks to @scott-huberty — his earlier mffpy groundwork helped me quickly pinpoint and fix the broadcasting/digitization blockers.

Most checks are now green. The remaining failing Azure check appears to be CI/infrastructure related and:

“No matching distribution found for numpy>=2.1.0.dev0”
So this failure occurs during dependency installation (before tests run), not in project test logic.

I’d really appreciate a formal review when you or Scott have a moment — and if possible, a maintainer rerun of the failed Azure job.