RuntimeError: Matrix is badly conditioned

Hello,
I am able to run the prepare_emptyroom, but when I try the maxfilter command on the prepared data, I get this error. I tried a set of 7 subject data files and 7 empty room files, and each created the error.

  • MNE version: 1.7.0
  • operating system: Ubuntu 22.04.4 LTS

Here is some of the data Iā€™m working with:
empty_ready is the empty room raw file that went through

In [9]:     empty_ready =  mne.preprocessing.maxwell_filter_prepare_emptyroom(empty_raw, raw =raw, bads ='union' , verbose=True)

In [10]: empty_ready.info
Out[10]:
<Info | 22 non-empty values
 acq_pars: ACQch001 110113 ACQch002 110112 ACQch003 110111 ACQch004 110122 ...
 acq_stim: 1 10.000000 1000.000000
 bads: 2 items (MEG0443, MEG1622)
 ch_names: MEG0113, MEG0112, MEG0111, MEG0122, MEG0123, MEG0121, MEG0132, ...
 chs: 204 Gradiometers, 102 Magnetometers, 4 Stimulus
 custom_ref_applied: False
 description: Vectorview system
 dev_head_t: MEG device -> head transform
 dig: 135 items (3 Cardinal, 4 HPI, 128 Extra)
 events: 1 item (list)
 experimenter: MEG (meg)
 highpass: 0.1 Hz
 hpi_subsystem: 3 items (dict)
 line_freq: 60.0
 lowpass: 330.0 Hz
 meas_date: 2024-01-10 19:25:58 UTC
 meas_id: 4 items (dict)
 nchan: 310
 proj_id: 1 item (ndarray)
 proj_name: test
 projs: mag_ssp_upright.fif : PCA-v1: off, mag_ssp_upright.fif : PCA-v2: ...
 sfreq: 1000.0 Hz
 subject_info: 6 items (dict)
>
In [11]:         empty_proc =  maxwell_filter(empty_ready,origin = (0,0,40), int_order=8, ext_order=3, calibration=cal_data, cross_talk= ct_data, st_duration =6, st_correlation=0.
   ...: 95 ,  coord_frame = 'head', head_pos=head_pos)
   ...:
Maxwell filtering raw data
102 of 102 magnetometer types replaced with T3.
    Bad MEG channels being reconstructed: ['MEG0443', 'MEG1622']
    Processing 204 gradiometers and 102 magnetometers
    Using fine calibration sss_cal_3046_2009-04-29.dat
        Adjusting non-orthogonal EX and EY
        Adjusted coil positions by (Ī¼ Ā± Ļƒ): 0.5Ā° Ā± 0.4Ā° (max: 2.4Ā°)
    Using origin 0.0, 0.0, 40000.0 mm in the head frame
    Processing data using tSSS with st_duration=6.0
/export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:2349: RuntimeWarning: invalid value encountered in scalar divide
  f"bits/sample ({100 * I_tots[lim_idx] / max_info:0.1f}% of peak "
        Using 95/95 harmonic components for    0.000  (80/80 in, 15/15 out)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[9], line 1
----> 1 empty_proc =  maxwell_filter(empty_ready,origin = (0,0,40), int_order=8, ext_order=3, calibration=cal_data, cross_talk= ct_data, st_duration =6, st_correlation=0.95 ,  coord_frame = 'head', head_pos=head_pos)

File <decorator-gen-358>:12, in maxwell_filter(raw, origin, int_order, ext_order, calibration, cross_talk, st_duration, st_correlation, coord_frame, destination, regularize, ignore_ref, bad_condition, head_pos, st_fixed, st_only, mag_scale, skip_by_annotation, extended_proj, verbose)

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:406, in maxwell_filter(raw, origin, int_order, ext_order, calibration, cross_talk, st_duration, st_correlation, coord_frame, destination, regularize, ignore_ref, bad_condition, head_pos, st_fixed, st_only, mag_scale, skip_by_annotation, extended_proj, verbose)
    384 logger.info("Maxwell filtering raw data")
    385 params = _prep_maxwell_filter(
    386     raw=raw,
    387     origin=origin,
   (...)
    404     extended_proj=extended_proj,
    405 )
--> 406 raw_sss = _run_maxwell_filter(raw, **params)
    407 # Update info
    408 _update_sss_info(raw_sss, **params["update_kwargs"])

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:688, in _run_maxwell_filter(raw, skip_by_annotation, st_duration, st_correlation, st_only, st_when, ctc, coil_scale, this_pos_quat, meg_picks, good_mask, grad_picks, head_pos, info, _get_this_decomp_trans, S_recon, update_kwargs, ignore_ref, reconstruct, copy)
    660 def _run_maxwell_filter(
    661     raw,
    662     skip_by_annotation,
   (...)
    686     # The time it takes to recompute S and pS themselves is roughly on par
    687     # with the np.dot with the data, so not a huge gain to be made there.
--> 688     S_decomp, S_decomp_full, pS_decomp, reg_moments, n_use_in = _get_this_decomp_trans(
    689         info["dev_head_t"], t=0.0
    690     )
    691     update_kwargs.update(reg_moments=reg_moments.copy())
    692     if ctc is not None:

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:1259, in _get_decomp(trans, all_coils, cal, regularize, exp, ignore_ref, coil_scale, grad_picks, mag_picks, good_mask, mag_or_fine, bad_condition, t, mag_scale, mult)
   1257 msg = "Matrix is badly conditioned: %0.0f >= 1000" % cond
   1258 if bad_condition == "error":
-> 1259     raise RuntimeError(msg)
   1260 elif bad_condition == "warning":
   1261     warn(msg)

RuntimeError: Matrix is badly conditioned: 18132836229591732 >= 1000

I found the previous thread about this error, which seemed to indicate issues with head position measurement.
I checked the head position from the participant data:
M87104027_head_pos

Is there anything else I can check to troubleshoot this? Any other info I can provide? Does this head position include too large a change?

Thanks for your help,
Megan

Hello,
Just wanted to try to see if anyone can help troubleshoot this issue. I will try to attend the office hours next week.
Here is another example, where the head position looks pretty fine to me from the original run:

mne.viz.plot_head_positions(head_pos, mode="traces")

M87104027_visit2_run1_headpos


In [16]:  empty_proc =  maxwell_filter(empty_ready,origin = (0,0,40), int_order=8, ext_order=3, calibration=cal
    ...: _data, cross_talk= ct_data, st_duration =6, st_correlation=0.95 ,  coord_frame = 'head', head_pos=head
    ...: _pos)
    ...:
Maxwell filtering raw data
102 of 102 magnetometer types replaced with T3.
    Bad MEG channels being reconstructed: ['MEG0443']
    Processing 204 gradiometers and 102 magnetometers
    Using fine calibration sss_cal_3046_2009-04-29.dat
        Adjusting non-orthogonal EX and EY
        Adjusted coil positions by (Ī¼ Ā± Ļƒ): 0.5Ā° Ā± 0.4Ā° (max: 2.4Ā°)
    Using origin 0.0, 0.0, 40000.0 mm in the head frame
    Processing data using tSSS with st_duration=6.0
/export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:2349: RuntimeWarning: invalid value encountered in scalar divide
  f"bits/sample ({100 * I_tots[lim_idx] / max_info:0.1f}% of peak "
        Using 95/95 harmonic components for    0.000  (80/80 in, 15/15 out)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[16], line 1
----> 1 empty_proc =  maxwell_filter(empty_ready,origin = (0,0,40), int_order=8, ext_order=3, calibration=cal_data, cross_talk= ct_data, st_duration =6, st_correlation=0.95 ,  coord_frame = 'head', head_pos=head_pos)

File <decorator-gen-358>:12, in maxwell_filter(raw, origin, int_order, ext_order, calibration, cross_talk, st_duration, st_correlation, coord_frame, destination, regularize, ignore_ref, bad_condition, head_pos, st_fixed, st_only, mag_scale, skip_by_annotation, extended_proj, verbose)

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:406, in maxwell_filter(raw, origin, int_order, ext_order, calibration, cross_talk, st_duration, st_correlation, coord_frame, destination, regularize, ignore_ref, bad_condition, head_pos, st_fixed, st_only, mag_scale, skip_by_annotation, extended_proj, verbose)
    384 logger.info("Maxwell filtering raw data")
    385 params = _prep_maxwell_filter(
    386     raw=raw,
    387     origin=origin,
   (...)
    404     extended_proj=extended_proj,
    405 )
--> 406 raw_sss = _run_maxwell_filter(raw, **params)
    407 # Update info
    408 _update_sss_info(raw_sss, **params["update_kwargs"])

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:688, in _run_maxwell_filter(raw, skip_by_annotation, st_duration, st_correlation, st_only, st_when, ctc, coil_scale, this_pos_quat, meg_picks, good_mask, grad_picks, head_pos, info, _get_this_decomp_trans, S_recon, update_kwargs, ignore_ref, reconstruct, copy)
    660 def _run_maxwell_filter(
    661     raw,
    662     skip_by_annotation,
   (...)
    686     # The time it takes to recompute S and pS themselves is roughly on par
    687     # with the np.dot with the data, so not a huge gain to be made there.
--> 688     S_decomp, S_decomp_full, pS_decomp, reg_moments, n_use_in = _get_this_decomp_trans(
    689         info["dev_head_t"], t=0.0
    690     )
    691     update_kwargs.update(reg_moments=reg_moments.copy())
    692     if ctc is not None:

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:1259, in _get_decomp(trans, all_coils, cal, regularize, exp, ignore_ref, coil_scale, grad_picks, mag_picks, good_mask, mag_or_fine, bad_condition, t, mag_scale, mult)
   1257 msg = "Matrix is badly conditioned: %0.0f >= 1000" % cond
   1258 if bad_condition == "error":
-> 1259     raise RuntimeError(msg)
   1260 elif bad_condition == "warning":
   1261     warn(msg)

RuntimeError: Matrix is badly conditioned: 17743322965512802 >= 1000

Please let me know if I can provide any additional info.
Thanks,
Megan

Some additional info: if I try to replicate my participant processing as I did using MEGINā€™s command line maxfilter, I get the same error. So Iā€™m just not using maxwell_filter correctly, it seems.

My original maxfilter command:

/neuro/bin/util/maxfilter-2.2 -v -force -autobad 60 -badlimit 20 -movecomp inter -st 6 -corr 0.95 -f M87104027+Study20240405+avmmn_visit2_run1_raw.fif -o /M87104027_avmmn_visit2_run1_tsss_mc.fif -origin 0 0 40 -frame head -format short -hp M87104027_avmmn_visit2_run1_tsss_mc_headpos.asc 

What I tried in mne (I used the asc file created from my command line maxfilter-2.2 run for the head_pos):

In [12]: raw_proc =  maxwell_filter(raw,origin = (0,0,40), int_order=8, ext_order=3, calibration=cal_data, cros
    ...: s_talk= ct_data, st_duration =6, st_correlation=0.95 ,  coord_frame = 'head', head_pos=head_pos)
Maxwell filtering raw data
102 of 102 magnetometer types replaced with T3.
    No bad MEG channels
    Processing 204 gradiometers and 102 magnetometers
    Using fine calibration sss_cal_3046_2009-04-29.dat
        Adjusting non-orthogonal EX and EY
        Adjusted coil positions by (Ī¼ Ā± Ļƒ): 0.5Ā° Ā± 0.4Ā° (max: 2.4Ā°)
    Using origin 0.0, 0.0, 40000.0 mm in the head frame
    Processing data using tSSS with st_duration=6.0
/export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:2349: RuntimeWarning: invalid value encountered in scalar divide
  f"bits/sample ({100 * I_tots[lim_idx] / max_info:0.1f}% of peak "
        Using 95/95 harmonic components for    0.000  (80/80 in, 15/15 out)
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[12], line 1
----> 1 raw_proc =  maxwell_filter(raw,origin = (0,0,40), int_order=8, ext_order=3, calibration=cal_data, cross_talk= ct_data, st_duration =6, st_correlation=0.95 ,  coord_frame = 'head', head_pos=head_pos)

File <decorator-gen-358>:12, in maxwell_filter(raw, origin, int_order, ext_order, calibration, cross_talk, st_duration, st_correlation, coord_frame, destination, regularize, ignore_ref, bad_condition, head_pos, st_fixed, st_only, mag_scale, skip_by_annotation, extended_proj, verbose)

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:406, in maxwell_filter(raw, origin, int_order, ext_order, calibration, cross_talk, st_duration, st_correlation, coord_frame, destination, regularize, ignore_ref, bad_condition, head_pos, st_fixed, st_only, mag_scale, skip_by_annotation, extended_proj, verbose)
    384 logger.info("Maxwell filtering raw data")
    385 params = _prep_maxwell_filter(
    386     raw=raw,
    387     origin=origin,
   (...)
    404     extended_proj=extended_proj,
    405 )
--> 406 raw_sss = _run_maxwell_filter(raw, **params)
    407 # Update info
    408 _update_sss_info(raw_sss, **params["update_kwargs"])

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:688, in _run_maxwell_filter(raw, skip_by_annotation, st_duration, st_correlation, st_only, st_when, ctc, coil_scale, this_pos_quat, meg_picks, good_mask, grad_picks, head_pos, info, _get_this_decomp_trans, S_recon, update_kwargs, ignore_ref, reconstruct, copy)
    660 def _run_maxwell_filter(
    661     raw,
    662     skip_by_annotation,
   (...)
    686     # The time it takes to recompute S and pS themselves is roughly on par
    687     # with the np.dot with the data, so not a huge gain to be made there.
--> 688     S_decomp, S_decomp_full, pS_decomp, reg_moments, n_use_in = _get_this_decomp_trans(
    689         info["dev_head_t"], t=0.0
    690     )
    691     update_kwargs.update(reg_moments=reg_moments.copy())
    692     if ctc is not None:

File /export/research/analysis/human/jstephen/shared/programs/python/anaconda_072020/new/envs/mne1.7/lib/python3.12/site-packages/mne/preprocessing/maxwell.py:1259, in _get_decomp(trans, all_coils, cal, regularize, exp, ignore_ref, coil_scale, grad_picks, mag_picks, good_mask, mag_or_fine, bad_condition, t, mag_scale, mult)
   1257 msg = "Matrix is badly conditioned: %0.0f >= 1000" % cond
   1258 if bad_condition == "error":
-> 1259     raise RuntimeError(msg)
   1260 elif bad_condition == "warning":
   1261     warn(msg)

RuntimeError: Matrix is badly conditioned: 15210861592565222 >= 1000

Thanks for any eyes on this,
Megan

@larsoner Would you have any idea whatā€™s going on here?

Typically this is triggered by implausible (or very far out of helmet) head positions. But your position estimates look reasonable.

I think itā€™s origin = (0,0,40). Keep in mind the units are SI meters in MNE, not millimeters like in maxfilter. Can you see if origin=(0., 0., 0.04) fixes the problem?

1 Like

Hi all,
YES. Thank you for noticing that.
I read preprocessing doc , but I didnā€™t see anything about meters, there. It would be great to add it to info about the origin parameter. I also donā€™t see anything in the tutorial article.

If I look closely in the output, using origin = (0,0,0.04), I see

    Using origin 0.0, 0.0, 40.0 mm in the head frame

and if I use origin (0,0,40), I see:

Using origin 0.0, 0.0, 40000.0 mm in the head frame

ā€¦too bad I didnā€™t notice that sooner.

Anyway. Thatā€™s sorted. Thanks for your help!
Megan

1 Like

The documentation you link states for the origin argument:

origin : array_like, shape (3,) | str
Origin of internal and external multipolar moment space in meters. [ā€¦]

There is nothing to change here.

Mathieu

1 Like

Maybe there should be a warning if the values are unusually large

Hi all,
Youā€™re right. I read it, but I didnā€™t notice.

It would be nice to include a hint in the error output to check head position and head origin.

Thanks again for your help!
Megan

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.