CSP for Multi-Classification

✉️ Mailing List Archive (read-only)
1

External Email - Use Caution

Hi everyone,
CSP (from mne.decoding import CSP) (Common Spatial Patterns) for
Multi-Classification gives clearly worse accuracy than the binary
classification. Would you suggest a better solution for the case of more
than two classes? Maybe another kind of CSP? Another kind of Machine
Learning methods? or even Deep Learning?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20190223/c22e73b0/attachment.html

2

External Email - Use Caution

Hi!

Could you please be more specific? What have you tried for the multi-class case (how many classes?) and which method yielded better results? Which performance metric did you consider? Which data (cross-validation)? CSP is a spatial filter, and you can train filters e.g. in a 1-vs-rest scheme.

Clemens

3

External Email - Use Caution

Hi Clemens,
Thank you very much for your quesitons. I tried 2,3,4,6, and 18 classes. I
found out that the best accuracy is in case of 2 classes and that by
increasing the number of classes the accuracy decreases.
I used cross_validation. for all cases (2,3,4,6, or 18 classes) Following
is part of my code as I copied some from
https://mne-tools.github.io/0.13/auto_examples/decoding/plot_decoding_csp_eeg.html

from mne.decoding import CSP
from sklearn.cross_validation import cross_val_score

X_train, X_test, y_train, y_test = train_test_split(my_array, y,
test_size=0.25,random_state=0)
csp = CSP(n_components=15, reg=None, log=True, norm_trace=False)
log_reg = LogisticRegression()
clf = Pipeline([('CSP', csp), ('Reg', log_reg)])
my_score = cross_val_score(clf, X_train, y_train, cv=10, n_jobs=1,
scoring='accuracy')

Is it right this way for multi-classification? What would you suggest? What
do you mean by CSP is a spatial filter, and you can train filters e.g. in
a 1-vs-rest scheme

Many thanks for your kind help

El s?b., 23 feb. 2019 a las 23:20, Brunner, Clemens (
clemens.brunner at uni-graz.at) (<clemens.brunner at uni-graz.at>) escribi?:

        External Email - Use Caution

Hi!

Could you please be more specific? What have you tried for the multi-class
case (how many classes?) and which method yielded better results? Which
performance metric did you consider? Which data (cross-validation)? CSP is
a spatial filter, and you can train filters e.g. in a 1-vs-rest scheme.

Clemens

*From:* mne_analysis-bounces at nmr.mgh.harvard.edu <
mne_analysis-bounces at nmr.mgh.harvard.edu> *On Behalf Of *tr rt
*Sent:* Saturday, 23 February 2019 11:59
*To:* mne_analysis at nmr.mgh.harvard.edu
*Subject:* [Mne_analysis] CSP for Multi-Classification

* External Email - Use Caution *

Hi everyone,

CSP (from mne.decoding import CSP) (Common Spatial Patterns) for
Multi-Classification gives clearly worse accuracy than the binary
classification. Would you suggest a better solution for the case of more
than two classes? Maybe another kind of CSP? Another kind of Machine
Learning methods? or even Deep Learning?
_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
Mne_analysis Info Page

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20190224/feca6312/attachment.html

4

External Email - Use Caution

Hi!

If you use accuracy as your performance measure, note that the chance level depends on the number of classes (among other factors such as class balance). For 2 balanced classes, chance level is 50%, for 3 classes its 33.33%, for 4 classes 25%, and so on. Therefore, given equally well separable data, accuracy will decrease with increasing number of classes (but there are other definitions on how to average accuracy in the multi class case).

Here's a good summary of performance measures that might be helpful: https://scikit-learn.org/stable/modules/model_evaluation.html. You might want to try Cohen's kappa, which is 0 at chance level and 1 in the best case (irrespective of the number of classes you have).

CSP is not a classifier, it preprocesses the data so that the variance in one class is maximized vs. the other class. In the multi class case, CSP uses a 1-vs-rest scheme. All this happens automatically, so you don't have to worry about that. Your code looks good, you're correctly using a combination of CSP and a classifier.

Clemens