Does anyone know how to judge the goodness of fit of an MEG localization, in particular with regard to TF-MxNE? RMSE between the measured sensor data and that predicted by the localization seems to be a popular choice, but has limited value except in direct comparisons, and no test statistic.
I am tempted to use Wilk's Lambda, defined as Det(SS_error)/Det(SS_total), where SS_error and SS_total are the SSCP matrices as defined in a multivariate regression. In this case the data would be the array M of the sensor observations (#sensors x #samples), which is modeled by the G*Z*Phi-Hermetian term. Rao's-F then provides an approximate p-value. Unfortunately neither SS_error nor SS_total are full rank on my data and thus the determinants are not available. Additionally, I am struggling with the validity of this on a nonstationary system. (Ranks are both ~30, in SSCP matrices of 306x306, corresponding to a Neuromag scanner).
Phys. Rev. Lett. 100, 018701 (2008) - Estimating Granger Causality from Fourier and Wavelet Transforms of Time Series Data
Experiments in many fields of science and engineering yield data in the form of time series. The Fourier and wavelet transform-based nonparametric methods are used widely to study the spectral characteristics of these time series data. Here, we extend the framework of nonparametric spectral methods to include the estimation of Granger causality spectra for assessing directional influences. We illustrate the utility of the proposed methods using synthetic data from network models consisting of interacting dynamical systems.
Read more...<http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.100.018701>
usually one run tests between conditions or across subjects.
I am not sure what you want to test. Do you have in mind
chi2 tests as used for dipole fitting?
Maybe have a look at "discrepancy principle" sometimes used
for inverse problems but I am not sure it will solve your problem.
Sorry for the delay on this question. I would like to report a goodness of fit between the evoked response (at the sensors) derived from my experimental data, and the evoked field (at the sensors) as modeled by the tf-mxne solution. I often see this associated with dipole fits (on the mne_analyze dipole list?) and it is usually reported at "Goodness of Fit" as a percentage. Is this the Chi-Sq you mention, and do you know of a useful reference to it?
Sorry for the delay on this question. I would like to report a goodness of fit between the evoked response (at the sensors) derived from my experimental data, and the evoked field (at the sensors) as modeled by the tf-mxne solution. I often see this associated with dipole fits (on the mne_analyze dipole list?) and it is usually reported at "Goodness of Fit" as a percentage. Is this the Chi-Sq you mention, and do you know of a useful reference to it?
maybe somebody else can point you to some refs when using dipole fits.
for tf-mxne GOF makes sense on whitened data unless you have one
sensor type (eg. gradiometers). I am not even sure how neuromag graph
reports GOF for combined sensors.
Any hint from somebody?
so option one is to report GOF or R2 coef of determination on let's
say only gradiometers.
or compute these metrics on whitened data. I'd also recommend you show
the butterfly
plots of explained data. It is a nice way to visually demonstrate that
your sources
explain the data correctly.
R^2 would be perfect, since it is easy to interpret and has the accompanying F-test. However, R^2 only applies to a single outcome variable, being calculated as SS_reg/SS_total. This is where I turned to Wilk's Lambda, being the multivariate extension of R^2, or det(SSCP_res)/det(SSCP_total). Unfortunately the determinants cannot be calculated on data that is not full rank, which leads to the questions I am asking about PCA in the other thread.
Am I missing something? I am not operating on the whitened data.
Agreed that butterfly plots are an excellent visual verification of fit, and I am already producing them.