Discussion: Proposed Technical Roadmap for EGI-MFF Modernization (PR #13684)

Hi @larsoner and @drammock,

Following up on the initial work in PR #13684, I’ve been drafting a potential roadmap to move the EGI-MFF refactor toward a production-ready state. I’m eager to ensure that my proposed approach aligns with MNE-Python’s long-term goals and would value your feedback on the general direction.

I’ve structured a draft plan around 6 incremental milestones (moving from basic header mapping to out-of-core streaming). My goal with this structure is to keep the review surface area manageable for the maintainer team. I’m also proposing a few technical targets, such as achieving \text{Space Complexity} = O(1) for data streaming, to ensure we meet the library’s performance standards.

Seeking Guidance on Architecture:

Before I go further into the details, I was hoping to get your thoughts on a few strategic points:

  • Implementation Strategy: Would you prefer an incremental merge strategy into main for these milestones, or should I look into maintaining a longer-running feature branch?
  • Backward Compatibility: I’m weighing the benefits of a “clean break” (decommissioning legacy helpers) vs. a “phased transition” (keeping the legacy reader as an optional fallback). Which approach do you feel is more sustainable for the MNE user base?

I have a detailed draft document that covers the “Parity Paradox” validation logic and benchmarking designs in more depth. As my university exams begin on March 18th, I was hoping to get a “sanity check” on these high-level points before my study break.

If you have a moment, would it be helpful for me to share the full draft link here, or should I summarize a specific section first?

Best regards,

Pragnya Khandelwal