GeneralizingEstimator with incremental learning

External Email - Use Caution

Hi Giulia,

I recently needed to incorporate SGDClassifier into some MNE pipeline and
ran into the same problem when working with the LinearModel class. I ended
up subclassing the MNE class and overloaded / wrote my own "fit" method
with my own, then called that one instead.

If you write your own class and inherit from GeneralizingEstimator, give it
its own fit function (in which you write your code that calls partial_fit
on the SGDClassifier instance until convergence or whatever your criteria
are), then it should work.

Best
Alex Murphy

Send Mne_analysis mailing list submissions to
        mne_analysis at nmr.mgh.harvard.edu

To subscribe or unsubscribe via the World Wide Web, visit
        https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
or, via email, send a message with subject or body 'help' to
        mne_analysis-request at nmr.mgh.harvard.edu

You can reach the person managing the list at
        mne_analysis-owner at nmr.mgh.harvard.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Mne_analysis digest..."

Today's Topics:

   1. GeneralizingEstimator with incremental learning /
      .partial_fit (Giulia Gennari)
   2. Re: GeneralizingEstimator with incremental learning /
      .partial_fit (Jean-R?mi KING)
   3. Re: GeneralizingEstimator with incremental learning /
      .partial_fit (Giulia Gennari)

----------------------------------------------------------------------

Message: 1
Date: Wed, 5 Aug 2020 18:38:35 +0200
From: Giulia Gennari <giulia.gennari1991 at gmail.com>
Subject: [Mne_analysis] GeneralizingEstimator with incremental
        learning / .partial_fit
To: mne_analysis at nmr.mgh.harvard.edu
Message-ID:
        <CAMn-mVzF+1URRkCr3jf=
9rwneju9Nh_LUJzvgAEFYc0DtG_vSA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

        External Email - Use Caution

Hi!

I would need to try decoding with incremental learning (EEG data).
I was planning to use logistic regression by means of the SGDClassifier
<
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html
>
.
I would then need to call .partial_fit to make my estimator learn on each
of my training sets.
However:

'GeneralizingEstimator' object has no attribute 'partial_fit'

Same issue for SlidingEstimator.
Is there a way to work around this limitation?

Thank you so so much in advance!

Giulia Gennari
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20200805/2ceed796/attachment-0001.html

------------------------------

Message: 2
Date: Wed, 5 Aug 2020 19:17:24 +0200
From: Jean-R?mi KING <jeanremi.king at gmail.com>
Subject: Re: [Mne_analysis] GeneralizingEstimator with incremental
        learning / .partial_fit
To: Discussion and support forum for the users of MNE Software
        <mne_analysis at nmr.mgh.harvard.edu>
Message-ID:
        <
CAOcgdchCrC+ibarOSLBmaEr7tSgVFk71dxj1s5zAMekBZnvgCA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

        External Email - Use Caution

Hi Giulia,

I think you should be able to change the method:

model = sklearn.linear_model.SGDClassifier()
model.fit = model.partial_fit
slider = mne.decoding.SlidingEstimator(model)
for X, y in train_batches:
    slider.fit(X, y)

Best

JR

> External Email - Use Caution
>
> Hi!
>
> I would need to try decoding with incremental learning (EEG data).
> I was planning to use logistic regression by means of the SGDClassifier
> <
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html
>
> .
> I would then need to call .partial_fit to make my estimator learn on each
> of my training sets.
> However:
>
> 'GeneralizingEstimator' object has no attribute 'partial_fit'
>
> Same issue for SlidingEstimator.
> Is there a way to work around this limitation?
>
> Thank you so so much in advance!
>
> Giulia Gennari
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20200805/f78eff5e/attachment-0001.html

------------------------------

Message: 3
Date: Thu, 6 Aug 2020 15:54:37 +0200
From: Giulia Gennari <giulia.gennari1991 at gmail.com>
Subject: Re: [Mne_analysis] GeneralizingEstimator with incremental
        learning / .partial_fit
To: Discussion and support forum for the users of MNE Software
        <mne_analysis at nmr.mgh.harvard.edu>
Message-ID:
        <CAMn-mVydRwoZTBtuTj_UhQNX7TV=
ufd99xsTfSmFymF+1ZaAWA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

        External Email - Use Caution

Dear Jean-R?mi,

Thank you for the nice suggestion!

Just to make sure that this is working (I apologize for my ignorance):

When I run:
model = SGDClassifier(loss='log', class_weight='balanced')
model.fit = model.partial_fit
slider1 = SlidingEstimator(model, scoring='roc_auc')
slider1.fit(X_train, y_train)

or

clf = make_pipeline(Vectorizer(), StandardScaler(), model)
slider2 = SlidingEstimator(clf, scoring='roc_auc')
slider2.fit(X_train, y_train)

I do not get any error, while I would expect:

ValueError: class_weight 'balanced' is not supported for partial_fit.
In order to use 'balanced' weights, use
compute_class_weight('balanced', classes, y). Pass the resulting
weights as the class_weight parameter.

Since this is what I get with:
model.fit(X_train[:,:,single_time_point], y_train)

Is there a good reason for that? E.g. class weights are computed internally
beforehand by SlidingEstimator?

Thank you again!

Giulia

> External Email - Use Caution
>
> Hi Giulia,
>
> I think you should be able to change the method:
>
> model = sklearn.linear_model.SGDClassifier()
> model.fit = model.partial_fit
> slider = mne.decoding.SlidingEstimator(model)
> for X, y in train_batches:
> slider.fit(X, y)
>
> Best
>
> JR
>
>
>> External Email - Use Caution
>>
>> Hi!
>>
>> I would need to try decoding with incremental learning (EEG data).
>> I was planning to use logistic regression by means of the SGDClassifier
>> <
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html
>
>> .
>> I would then need to call .partial_fit to make my estimator learn on
each
>> of my training sets.
>> However:
>>
>> 'GeneralizingEstimator' object has no attribute 'partial_fit'
>>
>> Same issue for SlidingEstimator.
>> Is there a way to work around this limitation?
>>
>> Thank you so so much in advance!
>>
>> Giulia Gennari
>> _______________________________________________
>> Mne_analysis mailing list
>> Mne_analysis at nmr.mgh.harvard.edu
>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
>
> _______________________________________________
> Mne_analysis mailing list
> Mne_analysis at nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20200806/3f539252/attachment.html

------------------------------

_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis

End of Mne_analysis Digest, Vol 151, Issue 4
********************************************

<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&gt;
Virus-free.
www.avg.com
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&gt;
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20200806/c10bff67/attachment-0001.html