# mne-bids-pipeline basic error preprocessing

**URL:** <https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228>\
**Category:** Support & Discussions\
**Tags:** mne-bids-pipeline\
**Created:** [July 5, 2022, 9:08pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228 "2022-07-05T21:08:45Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![fraenni](https://avatars.discourse-cdn.com/v4/letter/f/e56c9b/32.png) [@fraenni](https://mne.discourse.group/u/fraenni)\
**Post date:** [July 5, 2022, 9:08pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/1 "2022-07-05T21:08:46Z")

</div>

Hello  
I am new to mne python (and python itself). For reproducibility, I have decided to start using the men-bids-pipeline.

I am using MacOS Montgomery 12.2.1.

This is how I have installed mne-bids and the dependencies:  
conda create --name mne --channel conda-forge mne-bids  
pip install -r [https://raw.githubusercontent.com/mne-tools/mne-bids-pipeline/main/requirements.txt](https://raw.githubusercontent.com/mne-tools/mne-bids-pipeline/main/requirements.txt)

It is running on python 3.9.

I have customised the config.py, and it starts to run just fine using the python run.py --config=config.py --steps=preprocessing command. Then I am getting the following error:

```python
    spec.loader.exec_module(custom_cfg)
  File "/Users/Franziska/Dropbox/PredictiveLanguageCoding_EEG/EEG_analysis/mne-bids-pipeline-main-3/config.py", line 2006, in <module>
    spec.loader.exec_module(custom_cfg)
  File "/Users/Franziska/Dropbox/PredictiveLanguageCoding_EEG/EEG_analysis/mne-bids-pipeline-main-3/config.py", line 2006, in <module>
    spec.loader.exec_module(custom_cfg)
  File "/Users/Franziska/Dropbox/PredictiveLanguageCoding_EEG/EEG_analysis/mne-bids-pipeline-main-3/config.py", line 2006, in <module>
    spec.loader.exec_module(custom_cfg)
  File "/Users/Franziska/Dropbox/PredictiveLanguageCoding_EEG/EEG_analysis/mne-bids-pipeline-main-3/config.py", line 1927, in <module>
    coloredlogs.install(
  File "/Users/Franziska/opt/miniconda3/envs/mne/lib/python3.10/site-packages/coloredlogs/ __init__.py", line 537, in install
    handler.setFormatter(formatter_type(**formatter_options))
  File "/Users/Franziska/opt/miniconda3/envs/mne/lib/python3.10/site-packages/coloredlogs/ __init__.py", line 1025, in __init__
    kw = dict(fmt=self.colorize_format(fmt, style), datefmt=datefmt)
  File "/Users/Franziska/opt/miniconda3/envs/mne/lib/python3.10/site-packages/coloredlogs/ __init__.py", line 1081, in colorize_format
    result.append(ansi_wrap(
  File "/Users/Franziska/opt/miniconda3/envs/mne/lib/python3.10/site-packages/humanfriendly/terminal/ __init__.py", line 293, in ansi_wrap
    start_sequence = ansi_style(**kw)
  File "/Users/Franziska/opt/miniconda3/envs/mne/lib/python3.10/site-packages/humanfriendly/terminal/ __init__.py", line 236, in ansi_style
    elif isinstance(color_value, numbers.Number):
  File "/Users/Franziska/opt/miniconda3/envs/mne/lib/python3.10/abc.py", line 119, in __instancecheck__
    return _abc_instancecheck(cls, instance)
RecursionError: maximum recursion depth exceeded in comparison

```

It seems it is looking for a custom\_cfg file. As you know, in the relevant section in the config file it reads:

```python
# For testing a specific dataset, create a Python file with a name of your
# liking (e.g., ``mydataset-template-config.py``), and set an environment
# variable ``MNE_BIDS_STUDY_CONFIG`` to that file

```

.

I don’t know what I have to do here. What does “and set an environment variable `MNE_BIDS_STUDY_CONFIG` to that file.” mean and how am I doing that?

Also is there a detailed documentation of how to work with the pipeline? I am wondering as I am only half way through the config file …

Thanks a lot for your help. Much appreciated!  
Fraenni

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [July 6, 2022, 12:07pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/2 "2022-07-06T12:07:15Z")

</div>

Hello @fraenni, thanks for your interest in the pipeline! I suppose the setup instructions need some more work ☹ What you need to do is:

- create a copy of the original `config.py`
- delete everything below the lines

```python
  ###############################################################################
  # #
  # CUSTOM CONFIGURATION ENDS HERE #
  # #
  ###############################################################################

```

- adjust the configuration options – you can delete everything where you’re not changing the default
- To run the pipeline from the command line, do something like:

```python
MNE_BIDS_STUDY_CONFIG=/path/to/custom_config.py python run.py

```

Best wishes,  
Richard

---

<div class="post-metadata">

**Author:** ![fraenni](https://avatars.discourse-cdn.com/v4/letter/f/e56c9b/32.png) [@fraenni](https://mne.discourse.group/u/fraenni)\
**Post date:** [July 6, 2022, 4:35pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/3 "2022-07-06T16:35:53Z")

</div>

Hello Richard,  
Thank you very much for your advice. Very helpful!

Unfortunately, it is not quite working (yet), but this might be due to a misunderstanding.

I have done the following: I took my adapted my config file, and deleted everything after the marked area.

I then run the preprocessing part of the pipeline from the command line using the following command, as suggested:

`MNE_BIDS_STUDY_CONFIG=/Users//PredictiveLanguageCoding_EEG/EEG_analysis/mne-bids-pipeline-main-3/config.py python run.py --steps=preprocessing`

This produced the following error:

> (mne) mne-bids-pipeline-main-3 % MNE\_BIDS\_STUDY\_CONFIG=/Users/Franziska/Dropbox/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/config.py python run.py --steps=preprocessing  
> ERROR: Missing required flags: {‘config’}  
> Usage: run.py   
> optional flags: --config  
> required flags: --steps | --root\_dir | --subject | --session |  
> –task | --run | --interactive | --n\_jobs
> 
> For detailed information on this command, run:  
> run.py --help

I therefore included `--config=/Users/PredictiveLanguageCoding_EEG/EEG_analysis/mne-bids-pipeline-main-3/config`

Again an error

> ModuleNotFoundError: No module named ‘config’

I installed the module config, and run the full command again, however, now receiving an error, for something that is no longer defined in the config file after deleting everything after the marked area.

> (mne)mne-bids-pipeline-main-3 % MNE\_BIDS\_STUDY\_CONFIG=/Users/Franziska/Dropbox/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/config\_plang.py python run.py --config=/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/config\_plang.py --steps=preprocessing  
> Traceback (most recent call last):  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/run.py”, line 246, in   
> fire.Fire(process)  
> File “/Users/opt/miniconda3/envs/mne/lib/python3.10/site-packages/fire/core.py”, line 141, in Fire  
> component\_trace = \_Fire(component, args, parsed\_flag\_args, context, name)  
> File “/Users/opt/miniconda3/envs/mne/lib/python3.10/site-packages/fire/core.py”, line 466, in \_Fire  
> component, remaining\_args = \_CallAndUpdateTrace(  
> File “/Users/opt/miniconda3/envs/mne/lib/python3.10/site-packages/fire/core.py”, line 681, in \_CallAndUpdateTrace  
> component = fn(\*varargs, \*\*kwargs)  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/run.py”, line 196, in process  
> SCRIPT\_MODULES = \_get\_script\_modules(  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/run.py”, line 71, in \_get\_script\_modules  
> from scripts import init  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/scripts/init/ **init**.py”, line 1, in   
> from . import \_00\_init\_derivatives\_dir  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/scripts/init/\_00\_init\_derivatives\_dir.py”, line 18, in   
> from config import gen\_log\_kwargs, on\_error, failsafe\_run  
> ImportError: cannot import name ‘gen\_log\_kwargs’ from ‘config’ (/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/config.py)

I of course tried the full command including setting it to the environment using the config file without deleting anything. As expected it produced the command as in the first post.

There is probably something obvious I am still doing wrong, if you had some more advice that would be great!  
Thanks & best,  
F.

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [July 6, 2022, 7:21pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/4 "2022-07-06T19:21:51Z")

</div>

I’m sorry, @fraenni; I didn’t take enough time to draft my response and accidentally led you down the wrong path.

This environment variable need **not** to be set. Instead, passing the path to the config file via `--config` is sufficient.

You should be able to run:

```python
python run.py --config my_config.py

```

and that should be it.

Please uninstall the `config` package you installed; it is not required (and may lead to conflicts).

Let me know how it goes!

Best wishes,  
Richard

---

<div class="post-metadata">

**Author:** ![fraenni](https://avatars.discourse-cdn.com/v4/letter/f/e56c9b/32.png) [@fraenni](https://mne.discourse.group/u/fraenni)\
**Post date:** [July 6, 2022, 8:35pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/5 "2022-07-06T20:35:27Z")

</div>

Thanks again, Richard 🙂 I uninstalled config, and run the command:

> (mne) mne-bids-pipeline-main-3 % python run.py --config=/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/config\_plang.py --steps=preprocessing  
> Traceback (most recent call last):  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/run.py”, line 246, in   
> fire.Fire(process)  
> File “/Users/opt/miniconda3/envs/mne/lib/python3.10/site-packages/fire/core.py”, line 141, in Fire  
> component\_trace = \_Fire(component, args, parsed\_flag\_args, context, name)  
> File “/Users/opt/miniconda3/envs/mne/lib/python3.10/site-packages/fire/core.py”, line 466, in \_Fire  
> component, remaining\_args = \_CallAndUpdateTrace(  
> File “/Users/opt/miniconda3/envs/mne/lib/python3.10/site-packages/fire/core.py”, line 681, in \_CallAndUpdateTrace  
> component = fn(\*varargs, \*\*kwargs)  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/run.py”, line 196, in process  
> SCRIPT\_MODULES = \_get\_script\_modules(  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/run.py”, line 71, in \_get\_script\_modules  
> from scripts import init  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/scripts/init/ **init**.py”, line 1, in   
> from . import \_00\_init\_derivatives\_dir  
> File “/Users/PredictiveLanguageCoding\_EEG/EEG\_analysis/mne-bids-pipeline-main-3/scripts/init/\_00\_init\_derivatives\_dir.py”, line 17, in   
> import config  
> ModuleNotFoundError: No module named ‘config’

if I then reinstall the config module, I am back at the error: `ImportError: cannot import name ‘gen_log_kwargs’ from ‘config’ (/Users/PredictiveLanguageCoding_EEG/EEG_analysis/mne-bids-pipeline-main-3/config.py)` described in the last post.

in terms of my installation, I am only using the following, nothing else: python 3.10 (updated) and

> conda create --name mne --channel conda-forge mne-bids  
> pip install -r [https://raw.githubusercontent.com/mne-tools/mne-bids-pipeline/main/requirements.txt](https://raw.githubusercontent.com/mne-tools/mne-bids-pipeline/main/requirements.txt)

Thanks again! BW. F.

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [July 6, 2022, 8:48pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/6 "2022-07-06T20:48:29Z")

</div>

Please **don’t** reinstall the `config` package! Please remove it again!

This is one of the super annoying things about Python packaging, it’s not clear whether a package or module is imported from a “local” (current directory) or “global” (e.g., system-wide Python) installation.

The `config` import the pipeline tries to do is for `config.py` in the pipeline directory. Did you perhaps remove or rename this file? It must not be touched by the user! To create your personalized config file, the original `config.py` must be copied; the copy can be modified; but the original `config.py` **must** remain untouched!

---

<div class="post-metadata">

**Author:** ![fraenni](https://avatars.discourse-cdn.com/v4/letter/f/e56c9b/32.png) [@fraenni](https://mne.discourse.group/u/fraenni)\
**Post date:** [July 7, 2022, 4:20am UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/7 "2022-07-07T04:20:47Z")

</div>

Alright, I got it now. I kept the original config-file as “config.py” in the pipeline folder and stored the customise file in a separate folder with a different name. Now the pipeline is starting (producing other errors, but those I will hopefully figure out myself!)  
Thanks a lot for the help 😃  
BW. F

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [July 7, 2022, 8:28am UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/8 "2022-07-07T08:28:43Z")

</div>

@fraenni Please feel free to post new topics if any other problems come up.

Good luck!

---

<div class="post-metadata">

**Author:** ![allermat](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/allermat/32/43_2.png) [@allermat](https://mne.discourse.group/u/allermat)\
**Post date:** [November 9, 2022, 5:54pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/9 "2022-11-09T17:54:33Z")

</div>

Hi @richard, I also got this error and had to trim my copy of the config file. I couldn’t find this mentioned in the documentation though. I think this should be added to the documentation so people don’t have to dig up this forum thread. Or perhaps are you working on a better solution in the meantime?

Thanks,  
Máté

---

<div class="post-metadata">

**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\
**Post date:** [November 9, 2022, 7:01pm UTC](https://mne.discourse.group/t/mne-bids-pipeline-basic-error-preprocessing/5228/10 "2022-11-09T19:01:08Z")

</div>

Hello @allermat, we’re in the process of fixing this, I hope we’ll have something ready sometime next week! I will update this topic once we’re done.

Thanks!

Richard