Automate MEG processing using GNU make or similar data pipeline tool

💬 Support & Discussions
1

Hi,

I’m curious if anyone tried to adopt a tool like GNU make to transform their MEG data analysis project from start to finish into a reproducible automatized pipeline? I can see that mne-python uses make for building itself, but I’m specifically thinking about applying this to automate data analysis and document steps along the way reproducibly. This seems to be standard practice in data science, see e.g., here.

I am aware of MNE-BIDS-pipeline and I am using it, but make would enable to automate and document any processing pipeline.

I would be keen on adopting this practice for my MEG data analysis and I was wondering if anyone is already doing that and/or has any advice?

Thanks,
Máté

2

I use pydoit to achieve this. make is a bit too cumbersome. See me use it here: https://www.youtube.com/watch?v=RWpCWPZqdj8

and for an entire analysis pipeline that uses it:

2 Likes
3

Thanks for suggesting pydoit @wmvanvliet, this is great! Do you have experience running it on python 3.11 (all my conda envs are based on this version)? On their github page they say it is tested up to 3.10.

Thanks,
Máté

1 Like
4

No, but I can’t think of a reason it shouldn’t work.

1 Like
5

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.