I have been working with EEG data, and using the scaler function in mne.decoding with the default parameters, where it is supposed to centre the data with zero mean and unit variance for each channel, from what I understand of the documentation. However, if I try it and then check by computing the mean and standard deviation for each channel, they are not 0 and 1 respectively.
I would be grateful if somebody could let me know what is happening - if I am doing something wrong or misunderstanding the method. Here is my software details and a minimal reproducible example:
MNE version: 1.4.2
operating system: e.g. macOS 14.3
import mne
import numpy as np
np.random.seed(1)
n_epochs, n_channels, n_times = 10, 5, 100
data = np.random.randn(n_epochs, n_channels, n_times)
info = mne.create_info(n_channels, 100, ch_types='eeg')
scaler = mne.decoding.Scaler(info)
data_scaled = scaler.fit_transform(data)
# Check mean and std in each channel
for i in range(n_channels):
channel_data = data_scaled[:, i, :]
print(f"Channel {i}:")
print(f" Mean: {channel_data.mean()}")
print(f" Std: {channel_data.std()}")
Based on the API docs and the corresponding tutorial I believe you can achieve what you want by passing the argument scaling='mean'. If I understand it correctly, using scalings=None must simply be dividing all values by a constant.
Thanks for your response. I don’t think the tutorial link you posted is working.
Looking at the API docs, it says " This class scales data for each channel. It differs from scikit-learn classes (e.g., sklearn.preprocessing.StandardScaler) in that it scales each channel by estimating μ and σ using data from all time points and epochs, as opposed to standardizing each feature (i.e., each time point for each channel) by estimating using μ and σ using data from all epochs". And since with_mean and with_std are parameters that are true by default, it follows that the object should subtract the mean and divide by standard deviation in some way that is different from sklearn’s StandardScaler method (which I thought wasn’t ideal for many EEG data analysis applications).
So i’m not sure if either I am misunderstanding it, or the code does something different from what the API docs describe.
For context, I am using this as a final step in my preprocessing pipeline for use in algorithms that analyse spatiotemporal dynamics of EEG data, so I don’t want to destroy the inter-channel variability.
The scaling happens for each time point separately across all epochs within one channel at a time if you feed in 3D data. So in your loop, you must create a second, nested loop across the time dimension.
Best wishes,
Richard
Edit: Reading the docs again – I might be mistaken. Unfortunately I currently don’t have time to look into this any further.
When using the Scaler for your data, it’s crucial to ensure that it is correctly centering your data around zero and scaling it to have a unit variance. For this purpose, the StandardScaler from scikit-learn is commonly used. It should provide the desired results. If you’re still facing issues, verify the data for any anomalies and check the scaler’s configuration. Additionally, if you are integrating this process with applications like [[loklok shorts], ensure that the data preprocessing steps align with the application’s requirements for optimal performance.
@henrylucas It seems your response was generated via an LLM (which is fine) and is not very useful (which is very bad). Please don’t do this again, or we’ll ban you for spamming. Thanks.
Thank you, this script highlights the differences well. I am surprised at how unclear/potentially misleading the API doc is for this, given this must be quite a commonly used feature - someone should probably update it.
