Answering both proposals with the same word: Maybe.
'Non-parametric' doesn't mean 'without assumption' (not even 'without
distributional assumption') -- the assumptions just tend to be quite
different. Even the bootstrap has assumptions. You have to check whether
those assumptions conflict with the inherent boundedness of performance
scores. It's the boundedness of the scores that's the core of the problem.
There are several parametric tests that could model scores bounded on
[0,1] -- binomial GLMs, and perhaps even the usual t-test if the
variable is transformed appropriately (e.g. (x - 0.5) * 2 to move the
score to [-1, 1], followed by the Fisher transformation), converting AUC
to d', etc.
Phillip
Hi JR,
Is there a reference for that? i.e., that non-parametric stats aren't
subject to the same inferential issue as t-tests?
Thanks, Andy
While this paper raises an important subtlety (a significant t-test
over subjects' decoding accuracies indicates that there "are some
people in the population whose fMRI data carry information about the
experimental condition ? but" [...] not that there is "an effect
that is typical in the population"), my understanding is that this
particular issue is not a problem for non-parametric statistics.
Do be careful when doing group-level statistics via inferential
statistics on accuracy scores -- Allefeld at al 2016 show some
of the problems with the naive approach using things like
t-tests or ANOVA. You could use a Binomial/Bernoulli regression
model to get around some of points they raise without needing to
use their minimum information statistic.
Best,
Phillip
Dear Yi-hui
Decoding is generally not really adapted for mix-designed, as
the models are traditionally fit at the single subject level -
i.e. your model cannot be easily optimized to look for an
across-subject effect.
You can however compare the decoding scores across
subjects/conditions as a first approximation, and specify
individual subjects' score as your random variable.
For multifactorial within-subject effects, a simple approach
can be to implement RSA; we recently added this example in MNE:
https://mne-tools.github.io/stable/auto_examples/decoding/decoding_rsa.html
I will refer you to Kriegoskorte's RSA papers to see how you
adapt this analysis to your specific needs,
Kindest regards,
Jean-R?mi
Hello MNE experts,
I have MEG data with two within-subject factors (each
having 2 and 3 levels) and one between-subject factor (2
levels). I performed decoding analysis on my MEG data by
using the function " time_decod.fit". The question is how
to perform group analysis in subject's decoding data in
MNE (or outside MNE) for my mixed design (2 x 3 x 2
factorial design). Besides, I have another dependent
variable by using "predict_proba" function to get the
predicting probability. I want to test whether the
distribution of predicting probability differ according to
my design. Whether the difference of the distribution
continues in time (e.g., 200-300ms after stimulus onset)
does not matter. Suggestions will be appreciated.
_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
The information in this e-mail is intended only for the
person to whom it is
addressed. If you believe this e-mail was sent to you in
error and the e-mail
contains patient information, please contact the Partners
Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was
sent to you in error
but does not contain patient information, please contact
the sender and properly
dispose of the e-mail.
_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
<mailto:Mne_analysis at nmr.mgh.harvard.edu>
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
The information in this e-mail is intended only for the person to
whom it is
addressed. If you believe this e-mail was sent to you in error and
the e-mail
contains patient information, please contact the Partners Compliance
HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to
you in error
but does not contain patient information, please contact the sender
and properly
dispose of the e-mail.
--
Andrew R. Dykstra, PhD
Department of Neurology
Ruprecht-Karls-Universit?t Heidelberg
andrew.dykstra at med.uni-heidelberg.de
<mailto:andrew.dykstra at med.uni-heidelberg.de>
Europe: +49.157.7028.2162, North America: +1.786.263.9742
_______________________________________________
Mne_analysis mailing list
Mne_analysis at nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/mne_analysis
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
Url : http://mail.nmr.mgh.harvard.edu/pipermail/mne_analysis/attachments/20171219/dcde32f2/attachment-0001.bin