# Found array with dim 3. Estimator expected \<= 2.

**URL:** <https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303>\
**Category:** Support & Discussions\
**Tags:** epochs\
**Created:** [July 20, 2022, 1:54pm UTC](https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303 "2022-07-20T13:54:10Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![JaiC96](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/jaic96/32/1036_2.png) [@JaiC96](https://mne.discourse.group/u/JaiC96)\
**Post date:** [July 20, 2022, 1:54pm UTC](https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303/1 "2022-07-20T13:54:10Z")

</div>

Version : mne-1.0.3

I have encountered this error every time i run a pipeline with validate as True. The accuracy of the code significantly changes with this and its not accurate.  
here is the code

‘’’  
raw= mne.io.read\_raw\_edf(‘ST7011J0-PSG.edf’,stim\_channel=‘auto’,preload=True)  
annot = mne.read\_annotations(‘ST7011JP-Hypnogram.edf’)  
raw.set\_annotations(annot, emit\_warning=True)  
events, \_ = mne.events\_from\_annotations(raw, event\_id=event\_id, chunk\_duration=30.)  
tmax = 30. - 1. / raw.info[‘sfreq’] # tmax in included  
epochs=mne.Epochs(raw=raw, events=events,event\_id=event\_id, tmin=0., tmax=tmax, baseline=None)  
‘’’  
\*\*

\*\*  
def eeg\_power\_band(epochs):  
#EEG relative power band feature extraction.

```
This function takes an ``mne.Epochs`` object and creates EEG features based
on relative power in specific frequency bands that are compatible with
scikit-learn.

Parameters
----------
epochs : Epochs
    The data.

Returns
-------
X : numpy array of shape [n_samples, 5]
    Transformed data.
"""

# specific frequency bands
FREQ_BANDS = {"delta": [0.1, 4.5],
              "theta": [4.5, 8.5],
              "alpha": [8.5, 11.5],
              "sigma": [11.5, 15.5],
              "beta": [15.5, 30]}

psds, freqs = psd_welch(epochs_train, picks='eeg', fmin=0.5, fmax=30.)
psds, freqs = psd_welch(epochs_test, picks='eeg', fmin=0.5, fmax=30.)
# Normalize the PSDs
psds /= np.sum(psds, axis=-1, keepdims=True)

global X
X = []

for fmin, fmax in FREQ_BANDS.values():
    psds_band = psds[:, :, (freqs >= fmin) & (freqs < fmax)].mean(axis=-1)
    X.append(psds_band.reshape(len(psds), -1))

return np.concatenate(X, axis=1)
return Xer(l_freq=0, h_freq=30)

```

\*\*

\*\*  
pipe=make\_pipeline(FunctionTransformer(eeg\_power\_band,validate=True),RandomForestClassifier(n\_estimators=20,criterion=‘entropy’,class\_weight=‘balanced’, random\_state=42))  
\*\*

\*\*  
yt\_train = epochs\_train.events[:, 2]  
pipe.fit(epochs\_train, yt\_train)  
\*\*

---

<div class="post-metadata">

**Author:** ![drammock](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/drammock/32/4_2.png) [@drammock](https://mne.discourse.group/u/drammock)\
**Post date:** [July 21, 2022, 1:58pm UTC](https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303/2 "2022-07-21T13:58:08Z")

</div>

could you please try to make this example reproducible, and as short as possible? Right now it loads in files without providing download links for others to get those files, and uses modules/functions/classes that aren’t imported in your posted code. There are also issues with formatting and indentation:

- the way to start and end code blocks on this forum is three backticks (this: ```)
- you seem to be defining a function, but lines following the `def` line aren’t indented
- your function has two `return` lines

There may be other things that also need fixing, this is just what I noticed on a quick read through. Try to make it so that users can copy the whole code block, paste it into a new python interpreter, and have it run to completion (or yield the error that you’re asking about).

---

<div class="post-metadata">

**Author:** ![JaiC96](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/jaic96/32/1036_2.png) [@JaiC96](https://mne.discourse.group/u/JaiC96)\
**Post date:** [July 22, 2022, 1:50pm UTC](https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303/3 "2022-07-22T13:50:54Z")

</div>

Sure. Sorry for the incomplete code

Here it is

```python

import os

import mne
import numpy as np
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
from mne.time_frequency import psd_welch
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import FunctionTransformer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
import matplotlib.pyplot as plt

```

Here is the code for reading edf file and creating epochs

```python
raw= mne.io.read_raw_edf(‘ST7011J0-PSG.edf’,stim_channel=‘auto’,preload=True)
annot = mne.read_annotations(‘ST7011JP-Hypnogram.edf’)
raw.set_annotations(annot, emit_warning=True)
events, _ = mne.events_from_annotations(raw, event_id=event_id, chunk_duration=30.)
tmax = 30. - 1. / raw.info[‘sfreq’] # tmax in included
epochs=mne.Epochs(raw=raw, events=events,event_id=event_id, tmin=0., tmax=tmax, baseline=None)

```

Here is the function for calcualting power spectral density

```python
def eeg_power_band(epochs):
    """EEG relative power band feature extraction.

    This function takes an ``mne.Epochs`` object and creates EEG features based
    on relative power in specific frequency bands that are compatible with
    scikit-learn.

    Parameters
    ----------
    epochs : Epochs
        The data.

    Returns
    -------
    X : numpy array of shape [n_samples, 5]
        Transformed data.
    """
    # specific frequency bands
    FREQ_BANDS = {"delta": [0.5, 4.5],
                  "theta": [4.5, 8.5],
                  "alpha": [8.5, 11.5],
                  "sigma": [11.5, 15.5],
                  "beta": [15.5, 30]}

    psds, freqs = psd_welch(epochs, fmin=0.5, fmax=30.)
    # Normalize the PSDs
    psds /= np.sum(psds, axis=-1, keepdims=True)
    

    X = []
    for fmin, fmax in FREQ_BANDS.values():
        psds_band = psds[:, :, (freqs >= fmin) & (freqs < fmax)].mean(axis=-1)
        X.append(psds_band.reshape(len(psds), -1))

    return np.concatenate(X, axis=1)

```

Here is the pipeline for the function and the data

```python
pipe=make_pipeline(FunctionTransformer(eeg_power_band,validate=True),RandomForestClassifier(n_estimators=20,criterion=‘entropy’,class_weight=‘balanced’, random_state=42))

```

```python
yt_train = epochs.events[:, 2]
pipe.fit(epochs, yt_train)

```

Now when I try to fit the pipeline with validate as True I am getting this error as

```python
Found array with dim 3. Estimator expected <= 2.

```

Thank you for the response 🙂

---

<div class="post-metadata">

**Author:** ![drammock](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/drammock/32/4_2.png) [@drammock](https://mne.discourse.group/u/drammock)\
**Post date:** [July 22, 2022, 2:50pm UTC](https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303/4 "2022-07-22T14:50:40Z")

</div>

thanks! There are still some problems though:

1. we don’t have access to the files you’re using (ST7011J0-PSG.edf and ST7011JP-Hypnogram.edf). They look like physionet sleep data; can they be downloaded with `from mne.datasets.sleep_physionet.age import fetch_data`? If so, include the lines of code that would do that. If not, change your example to use files that are built-in to MNE-Python, or provide us with download links.
2. The second code block has smart-quotes instead of straight quotes, so it’s not copy-paste-runnable
3. When I fix the quotes and substitute the files for ones that are already distributed with MNE-Python, running the second code block yields an error: `NameError: name 'event_id' is not defined`.

In light of that, I’ll repeat myself:

> [@drammock](#):
>
> Try to make it so that users can copy the whole code block, paste it into a new python interpreter, and have it run to completion (or yield the error that you’re asking about).

You should _test this_ by opening a fresh python interpreter from within a new, empty folder, copy-pasting code directly from your (draft) post, and seeing if the code runs.

As a side note: your code looks fairly similar to this tutorial:  
[https://mne.tools/dev/auto\_tutorials/clinical/60\_sleep.html](https://mne.tools/dev/auto_tutorials/clinical/60_sleep.html)

It would help if you mentioned that up front, and tell us what exactly is _different_ between your code and that tutorial. For example, if the _only_ thing that is different is the data files you’re using, that is _really helpful_ information that makes it much faster for us to figure out what is wrong. Alternatively, if your code fails with _the same data files_ used in the tutorial, then it’s way easier for us to debug the problem _with files that we already have_ (because they’re distributed with MNE-Python).

---

<div class="post-metadata">

**Author:** ![JaiC96](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/jaic96/32/1036_2.png) [@JaiC96](https://mne.discourse.group/u/JaiC96)\
**Post date:** [July 22, 2022, 2:59pm UTC](https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303/5 "2022-07-22T14:59:30Z")

</div>

Yes event id is missing.

here it is

```python
event_id = {'Sleep stage W': 1,
                              'Sleep stage 1': 2,
                              'Sleep stage 2': 3,
                              'Sleep stage 3': 4,
                              'Sleep stage 4': 4,
                              'Sleep stage R': 5}

```

The data download command is as follows

```python
wget -r -N -c -np https://physionet.org/files/sleep-edfx/1.0.0/

```

Yes the code is same as the link you have mentioned.

Sorry for this chaos. I’ll keep this things in mind

---

<div class="post-metadata">

**Author:** ![drammock](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/drammock/32/4_2.png) [@drammock](https://mne.discourse.group/u/drammock)\
**Post date:** [July 22, 2022, 4:36pm UTC](https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303/6 "2022-07-22T16:36:51Z")

</div>

OK, so please clarify then:

- if you run the tutorial _as written_ does it work?
- if you run the tutorial _as written, but only change the data files used_ does it work?

Also, I think your `wget` command will download the entire dataset? Can you provide a command to just get the specific data files needed? (this is why we have things like [mne.datasets.sleep\_physionet.age.fetch\_data — MNE 1.1.dev0 documentation](https://mne.tools/dev/generated/mne.datasets.sleep_physionet.age.fetch_data.html), why not use that?)

---

<div class="post-metadata">

**Author:** ![JaiC96](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/jaic96/32/1036_2.png) [@JaiC96](https://mne.discourse.group/u/JaiC96)\
**Post date:** [July 22, 2022, 8:24pm UTC](https://mne.discourse.group/t/found-array-with-dim-3-estimator-expected-2/5303/7 "2022-07-22T20:24:51Z")

</div>

I reran the tutorial with ‘‘validate=true’’ and it gave the error as

```python
ValueError: Expected 2D array, got scalar array instead:
array=<Epochs | 841 events (good & bad), 0 - 29.99 sec, baseline off, ~12 kB, data not loaded,
 'Sleep stage W': 188
 'Sleep stage 1': 58
 'Sleep stage 2': 250
 'Sleep stage 3/4': 220
 'Sleep stage R': 125>.
Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

```

Yes the same error is persisting with the other datasets too.

I had the downloaded the dataset using wget.

you can download it manully by below link under sleep telementry folder

[https://physionet.org/content/sleep-edfx/1.0.0/](https://physionet.org/content/sleep-edfx/1.0.0/)