I have some questions about good choices for the regularization parameter
lambda^2 in high noise systems. I?m using MNE-Python to run simulations of
theoretical high-density solid-state MEG arrays that operate with much
higher sensor noise than low-Tc SQUID sensors. These arrays contain a small
number of sensors with SNRs > 1, but the average SNR is usually << 1.
I?ve been analyzing dipole localization error (DLE) and spatial dispersion
(SD) on a resolution matrix with sensor noise injected in the middle (M @ N
@ G, for M: inverse operator matrix, G: forward operator matrix, N: sensor
noise matrix). I?m using MNE-Python to create my inverse and forward
matrix.
For a given system at a given sensor noise level, my measurements of DLE
and SD are predictably quite sensitive to choice of lambda^2. It seems that
the recommended choice of lambda^2 = 1/avg_snr (where avg_snr is the
average power snr of the system) is not close to the best choice for
lambda^2 found through tuning.
As I sweep lambda^2 values, there is a clear minimum in the SD function
that coincides with a knee of the curve for DLE. This seems like the right
value to choose for optimal overall system performance. Most often, the
tuned lambda^2 ends up being one or more orders of magnitude higher than
the recommended value, with the scale factor increasing with more noise in
the system.
Does using a large lambda^2 value lead to simulation results that are
valid, or is this a numerical exercise that introduces some bias that I
don?t yet understand? Does it make sense that the tuned value for lambda^2
would be much larger than 1/avg_snr for high noise systems, or is this
likely indicating a bug somewhere?
I?ve been analyzing dipole localization error (DLE) and spatial dispersion
(SD) on a resolution matrix with sensor noise injected in the middle (M @ N
@ G, for M: inverse operator matrix, G: forward operator matrix, N: sensor
noise matrix). I?m using MNE-Python to create my inverse and forward
matrix.
You might consider using the same measures implemented in MNE directly
nowadays, which will also give you some possibly complementary information:
For a given system at a given sensor noise level, my measurements of DLE
and SD are predictably quite sensitive to choice of lambda^2. It seems that
the recommended choice of lambda^2 = 1/avg_snr (where avg_snr is the
average power snr of the system) is not close to the best choice for
lambda^2 found through tuning.
As I sweep lambda^2 values, there is a clear minimum in the SD function
that coincides with a knee of the curve for DLE. This seems like the right
value to choose for optimal overall system performance. Most often, the
tuned lambda^2 ends up being one or more orders of magnitude higher than
the recommended value, with the scale factor increasing with more noise in
the system.
In addition to testing some other metrics (above), you could also try
looking to see if there is any relationship to the estimated SNR like in:
Does using a large lambda^2 value lead to simulation results that are
valid, or is this a numerical exercise that introduces some bias that I
don?t yet understand? Does it make sense that the tuned value for lambda^2
would be much larger than 1/avg_snr for high noise systems, or is this
likely indicating a bug somewhere?
I am not sure. It might also be worth (reading this and) pinging the
authors of this paper separately as they might have ideas, and I'm not sure
how actively they monitor the list:
Tuning lambda (or generally hyperparameters) in inverse problems is a
known challenge.
The MNE recommendation has shown to be a sensible default but surely
would be optimized
for certain studies / systems. Various strategies exist to do this:
cross-validation, "Bayesian
Ridge", L-Curve etc.
do not hesitate to share your experimental findings here.