# Using "n\_jobs=-1" for faster parallel CPU processing does not increase speed

**URL:** <https://mne.discourse.group/t/using-n-jobs-1-for-faster-parallel-cpu-processing-does-not-increase-speed/6770>\
**Category:** Support & Discussions\
**Created:** [April 20, 2023, 4:32pm UTC](https://mne.discourse.group/t/using-n-jobs-1-for-faster-parallel-cpu-processing-does-not-increase-speed/6770 "2023-04-20T16:32:45Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![moritz-gerster](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/moritz-gerster/32/1166_2.png) [@moritz-gerster](https://mne.discourse.group/u/moritz-gerster)\
**Post date:** [April 20, 2023, 4:32pm UTC](https://mne.discourse.group/t/using-n-jobs-1-for-faster-parallel-cpu-processing-does-not-increase-speed/6770/1 "2023-04-20T16:32:45Z")

</div>

Many mne functions have the keyword argument “n\_jobs” to parallelize processing. My machine (M1 Mac) has 10 cores, but no matter how I change the “n\_jobs” kwarg, the processing speed is always the same.

What am I doing wrong? Please check this [Jupyter notebook](https://www.dropbox.com/s/ir2sqgaka9ozbqf/mne_paralell_processing.pdf?dl=0) for exemplary reproducible code.

Please note, that this does not seem to be an M1 issue. I ran the same notebook on an old Intel iMac and also did not observe any performance enhancements: [Jupyter Notebook Intel Chip](https://www.dropbox.com/s/0vzlr45j6zku5x6/mne_paralell_processing_iMac.pdf?dl=0).

- MNE version: 1.3.0
- operating system: macOS 13.2.1-arm64-arm-64bit /

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [April 20, 2023, 5:12pm UTC](https://mne.discourse.group/t/using-n-jobs-1-for-faster-parallel-cpu-processing-does-not-increase-speed/6770/2 "2023-04-20T17:12:47Z")

</div>

Hello @moritz-gerster,

generally, it’s important to consider that parallelization of data processing often comes with considerable initialization costs, i.e., chunks of data need to be copied to new memory segments and new processes need to be spawned; and after processing is done, the results need to be collected and glued back together. This means that for operations that don’t take long on a single core (i.e., finish in less than a few seconds or sometimes even tens of seconds), trying to parallelize things may even lead to a **slowdown**.

In the examples you posted, we’re talking processing time of about a quarter of a second, which I would say is not suitable for going parallel (at least not with the machinery we currently use in MNE-Python).

Personally, I use parallelization esp. when doing MVPA, and it leads to **significant** speedups on my M1 Mac.

Best wishes,  
Richard

---

<div class="post-metadata">

**Author:** ![moritz-gerster](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/moritz-gerster/32/1166_2.png) [@moritz-gerster](https://mne.discourse.group/u/moritz-gerster)\
**Post date:** [April 20, 2023, 6:19pm UTC](https://mne.discourse.group/t/using-n-jobs-1-for-faster-parallel-cpu-processing-does-not-increase-speed/6770/3 "2023-04-20T18:19:47Z")

</div>

Thank you very much for this instructive answer @richard!

I will try to re-run the script with much larger data.

Short follow-up question: What is MVPA?

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [April 20, 2023, 6:28pm UTC](https://mne.discourse.group/t/using-n-jobs-1-for-faster-parallel-cpu-processing-does-not-increase-speed/6770/4 "2023-04-20T18:28:23Z")

</div>

> [@moritz-gerster](#):
>
> Short follow-up question: What is MVPA?

“Decoding” / machine learning.

See for example this temporal generalization example:  
[https://mne.tools/stable/auto\_tutorials/machine-learning/50\_decoding.html#temporal-generalization](https://mne.tools/stable/auto_tutorials/machine-learning/50_decoding.html#temporal-generalization)

That’s an operation where we’d usually go parallel for a huge speedup. See the (admittedly not super easy-to-read) code in MNE-BIDS-Pipeline:

> <https://github.com/mne-tools/mne-bids-pipeline/blob/b74f056ab1b6e1ac87d1747a977bf2414d169e9f/mne_bids_pipeline/steps/sensor/_03_decoding_time_by_time.py#L150-L156>

> <https://github.com/mne-tools/mne-bids-pipeline/blob/b74f056ab1b6e1ac87d1747a977bf2414d169e9f/mne_bids_pipeline/steps/sensor/_03_decoding_time_by_time.py#L165-L172>