rendering source localization plots without plot function

  • MNE version: 0.24.0
  • operating system: Windows 11

hello, this might be a weird question, but I’ve been searching and haven’t found anything.

so i would like to render whatever the stc.plot function renders, like the main brain model and colors and stuff, but without using the function, i would like to do it in another language with the data that is created from mne, like source estimates and other stuff

i tried calling stc.save_as_surface, and that made some files, one of which looked like a 3D surface, i converted it to a 3d model, and looked at it in blender, and the base model was different from what i saw using stc.plot, so is it not the model? am i getting something wrong?

is there an easier way to do this?

i would like to control some of the rendering operations and stuff and do it in another language and graphics api, sorry if the question is stupid, I’m new to using mne

Yes — for a surface source estimate, you can render it elsewhere by exporting the mesh and the source values separately.

First check which surface you are comparing. In MNE 0.24, stc.plot() defaults to surface="inflated". A white or pial surface will look different even for the same subject. You can read the matching FreeSurfer mesh from <subjects_dir>/<subject>/surf/lh.inflated (and rh.inflated) with mne.read_surface, which returns vertex coordinates and triangle indices.

The activation values are in stc.data, with their surface vertex IDs in stc.vertices: left hemisphere first, then right. Preserve that mapping when exporting — the source estimate usually covers only a subset of the mesh vertices. Matching MNE’s appearance also requires matching the smoothing, colormap and colour limits; the mesh alone does not contain those settings.

Could you share your stc.plot(...) and export calls, plus type(stc)? I couldn’t find save_as_surface in the MNE 0.24 SourceEstimate API, so seeing the exact call would help identify what was exported.

sorry, my bad, the version was wrong, i thought it was the latest version in the default text when you make a topic, i have version 1.12.1, i’m using fsaverage for the source space

by appearance i meant the brain geometry alone, not the colors, the brain geometry that i got was more detailed than the one that is show with stc.plot()

code for source space:

         self.space = mne.read_source_spaces(
   str(fs_dir / "bem" / "fsaverage-ico-5-src.fif")

        )

code for plotting

duration = 2.0

events = mne.make_fixed_length_events(

            self.raw,

            duration=duration

        )

epochs = mne.Epochs(

            self.raw,

            events,

            tmin=0,

            tmax=duration,

            baseline=None,

            preload=True,

            verbose=False,

            proj=True,

            picks='all',

        )

noise_cov = mne.compute_covariance(

            epochs, tmax=0.0, method=\["shrunk", "empirical"\], rank=None,                 verbose=True)

        evoked = epochs.average().pick("eeg")

        del epochs

        self.inverse_operator = make_inverse_operator(

            evoked.info, self.forward, noise_cov, loose=0.2, depth=0.8

        )

        method = "dSPM"

        snr = 3.0

        lambda2 = 1.0 / snr\*\*2

        stc, residual = apply_inverse(

            evoked,

            self.inverse_operator,

            lambda2,

            method=method,

            pick_ori=None,

            return_residual=True,

            verbose=True,

        )

        vertno_max, time_max = stc.get_peak(hemi=None)

        surfer_kwargs = dict(

            hemi="both",

            subjects_dir=self.subjects_dir,

            views="lateral",

            initial_time=time_max,

            time_unit="s",

            size=(800, 800),

            smoothing_steps=10,

        )

        stc.save("save", overwrite=True)

        stc.save_as_surface("asFile", self.space)

        brain = stc.plot(\*\*surfer_kwargs)

        brain.add_text(

            0.1, 0.9, "dSPM (plus location of maximal activation)", "title", font_size=14

        )

and this code for converting from .gii to .glb for the geometry:

import nibabel as nib

import numpy as np

import trimesh

def gifti_to_glb(input_file, output_file):

\# Load GIFTI

img = nib.load(input_file)

vertices = None

faces = None

vertices = np.array(\[el.data for el in img.get_arrays_from_intent("NIFTI_INTENT_POINTSET")\])

faces = np.array(\[el.data for el in img.get_arrays_from_intent("NIFTI_INTENT_TRIANGLE")\])

vertices = vertices.reshape(-1, vertices.shape\[-1\])

faces = faces.reshape(-1, faces.shape\[-1\])

vertices = vertices.astype(np.float32)

faces = faces.astype(np.int64)

mesh = trimesh.Trimesh(

    vertices=vertices,

    faces=faces,

    process=False

)

mesh.export(output_file, file_type="glb")

print(f"Converted: {input_file}")

print(f"Vertices:  {len(vertices):,}")

print(f"Faces:     {len(faces):,}")

print(f"Output:    {output_file}")

file_input = input()

file_output = input()

gifti_to_glb(file_input, file_output)

and type(stc) = <class ‘mne.source_estimate.SourceEstimate’>

Thanks, the code makes the difference clear. And yes, save_as_surface is available in 1.12.1 — it was added in 1.7.

save_as_surface("asFile", self.space) exports the geometry from your source space, whereas your stc.plot() call uses surface="inflated" by default. So you are comparing different surface geometries, not just different file formats. smoothing_steps=10 smooths the activation overlay, not the brain mesh.

For the inflated geometry, you can skip the GIFTI conversion and export the FreeSurfer surfaces directly:

from pathlib import Path
import mne
import trimesh

# fs_dir is your fsaverage directory
for hemi in ("lh", "rh"):
    vertices, faces = mne.read_surface(
        Path(fs_dir) / "surf" / f"{hemi}.inflated"
    )
    mesh = trimesh.Trimesh(
        vertices=vertices, faces=faces, process=False
    )
    mesh.export(f"fsaverage-{hemi}-inflated.glb")

This exports geometry only. The coordinates returned by read_surface are in millimetres; scale them by 0.001 if your renderer expects metres.

Does that give you the brain shape you were aiming for?

so from what i understood from the .time.gii file in the output of save_as_surface, it stores a matrix of time points and vertices, where each element is the activation for that vertex at that time point, and i tried rendering that onto the mesh i had from converting the .gii file to .glb, and i just used that as the color by mapping it using this hlsl code:

float SampleAtTimeIndex(int vertID)
{
    return TimeSeries[TimingIndex * VertexCount + vertID];
}

float NormalizeSample(float sample)
{
    return (sample - MinActivation) / (MaxActivation - MinActivation);
}

float4 CalculateColorFromNormalized(float normalized)
{
    return lerp(ColorMin, ColorMax, normalized);
}

is this correct? what kind of mapping does mne use?

also, is it ok to display the solution that was made with a forward operator that was made with a white surface, onto an inflated surface? both logically and code wise? like will an inflated and white surface always have the same amount of vertices so that the time course of a white surface matches that of an inflated surface?

Yes, displaying a white-surface solution on the corresponding inflated surface is fine. Inflation changes the display coordinates, not the source estimates; you do not need to recompute the forward solution for that. There are two indexing details worth checking, though.

Time samples: in MNE 1.12.1, each .time.gii contains one data array per time point, for one hemisphere. Your TimingIndex * VertexCount + vertID is correct if you pack those arrays consecutively into a time-major buffer. For example:

import nibabel as nib
import numpy as np

gii = nib.load("asFile-lh.time.gii")
values = np.stack([a.data for a in gii.darrays], axis=0)
# values.shape == (n_times, n_vertices)
buffer = values.ravel(order="C")

That also assumes the GLB export/import preserves the vertex ordering. If the renderer reorders or duplicates vertices, you need to carry the source-vertex mapping through that step.

Surface vertices: the standard FreeSurfer white and inflated surfaces for the same subject and hemisphere have corresponding vertex indices. However, your ico-5 source space uses only a subset of the full surface vertices. My previous GLB example exported the full inflated geometry, so it is not a direct one-value-per-vertex match for your ico-5 time series.

For a matching source-space mesh, take the full inflated coordinates at src[h]["vertno"] and use the reindexed triangles from mne.get_decimated_surfaces(src)[h]["tris"]. Check that stc.vertices[h] equals src[h]["vertno"] first (h=0 for left, 1 for right). Alternatively, keep the full mesh and map values using stc.vertices[h], then interpolate/smooth over the surface as MNE does. Equal vertex counts alone are not enough to establish correspondence.

Colors: your two-color interpolation is a valid custom scale, but it does not reproduce MNE’s default. With clim="auto", MNE uses the 96th, 97.5th and 99.95th percentiles as three control points, not the minimum and maximum. The automatic colormap is hot for nonnegative data and a diverging mne map for signed data; the latter uses absolute values for those percentiles. Transparency and surface smoothing also affect the result. For your own scale, clamp the normalized value with saturate() and handle MaxActivation == MinActivation explicitly.

Are you aiming to reproduce the smooth stc.plot() appearance, or to display the values directly on the ico-5 mesh?

ok so i decided to use setup_source_space instead, with ico5 and white as its arguments, and i called save_as_surface using that, and then converted the resulting .gii into glb, and it seemed to work, the vertex counts from .time.gii and the model were the same, so I can’t understand why the indices wouldn’t have correspondence

i didn’t quite understand the part about getting a source space mesh

my aim is to just reproduce the output of stc.plot as it is, with the optional given surface( white, inflated)

this is the code i’ve got to to make and write out the stc:

def setup_source_localization(spacing : str = "ico4") -> None:
    global fsaverage_subjects_dir
    global source_spaces
    global bem_solution
    global head_to_mri_trans
    global forward_solution

    fsaverage_subjects_dir = mne.datasets.fetch_fsaverage(
        verbose=True
    )

    fsaverage_subjects_dir = str(fsaverage_subjects_dir)

    # for surf in ['white', 'pial', 'inflated']:
    #     source_spaces[surf] = mne.setup_source_space(
    #             subject=fsaverage_subject,
    #             spacing=spacing,
    #             surface=surf,
    #             subjects_dir=fsaverage_subjects_dir,
    #             add_dist=False,
    #             verbose=True,
    #             n_jobs=-1
    #     )

    print("-------------------------------------------------\nCreating source space")

    source_spaces["white"] = mne.setup_source_space(
                subject=fsaverage_subject,
                spacing=spacing,
                surface="white",
                add_dist=False,
                verbose=True,
                n_jobs=-1
    )

    print("-------------------------------------------------\nCreated source space")
    print("-------------------------------------------------\nCreating bem")
    

    bem_model = mne.make_bem_model(
        subject=fsaverage_subject,
        ico=4,
        conductivity=(0.3, 0.006, 0.3),
        verbose=True
    )

    bem_solution = mne.make_bem_solution(
        bem_model,
        verbose=True
    )

    print("-------------------------------------------------\nCreated bem")


     
    print("-------------------------------------------------\nCreating forward solution")

    coreg = mne.coreg.Coregistration(
        info=raw.info,
        subject=fsaverage_subject,
        fiducials="auto"
    )

    coreg.fit_fiducials(
        verbose=True
    )
    
    head_to_mri_trans = coreg.trans

    forward_solution = mne.make_forward_solution(
        info=raw.info,
        trans=head_to_mri_trans,
        src=source_spaces["white"],
        bem=bem_solution,
        eeg=True,
        meg=False,
        mindist=5.0,
        verbose=True
    )

    print("-------------------------------------------------\nCreated forward solution")


def make_stc(tmin : float = 0.0, tmax : float | None = None, SNR : float = 3.0, method : str = "dSPM") -> None:
    global noise_covariance
    global inverse_operator
    global stc

    lambda2 = 1.0 / (SNR * SNR)

    if tmax is None:
        tmax = raw.times[-1]

    # ------------------------------------------------------------------
    # Create noise covariance
    # ------------------------------------------------------------------

    # This is a pragmatic covariance estimate for continuous EEG.
    #
    # For a proper experimental source-localization pipeline, it is
    # preferable to estimate the noise covariance from a baseline period
    # or from empty-room / prestimulus data where appropriate.

    events = mne.make_fixed_length_events(
                    raw,
                    duration=tmax-tmin
                )
    
    epochs = mne.Epochs(
        raw,
        events,
        tmin=tmin,
        tmax=tmax,
        baseline=None,
        preload=True,
        verbose=False,
        proj=True,
        picks='all',
    )

    noise_covariance = mne.compute_covariance(
        epochs, tmin=tmin, tmax=tmax, method=["empirical"], rank=None, verbose=True
    )

    evoked = epochs.average().pick("eeg")
    del epochs
    
    # ------------------------------------------------------------------
        # Create inverse operator
        # ------------------------------------------------------------------
        
    inverse_operator = mne.minimum_norm.make_inverse_operator(
        info=evoked.info,
        forward=forward_solution,
        noise_cov=noise_covariance,
        loose=0.2,
        depth=0.8,
    )
    
    # ------------------------------------------------------------------
    # Apply inverse to continuous EEG
    # ------------------------------------------------------------------
    
    stc = mne.minimum_norm.apply_inverse(
        evoked,
        inverse_operator,
        lambda2,
        method=method,
        pick_ori=None,
        verbose=True,
    )

def get_stc_dir_from_path(p : str) -> str:
    return str(Path(p) / "source_estimates")

def get_stc_file_name() -> str:
    return "stc-white"

def gifti_to_glb(input_file, output_file):
    # Load GIFTI
    img = nib.load(input_file)

    vertices = None
    faces = None

    vertices = np.array([el.data for el in img.get_arrays_from_intent("NIFTI_INTENT_POINTSET")])
    faces = np.array([el.data for el in img.get_arrays_from_intent("NIFTI_INTENT_TRIANGLE")])

    vertices = vertices.reshape(-1, vertices.shape[-1])
    faces = faces.reshape(-1, faces.shape[-1])

    vertices = vertices.astype(np.float32)
    faces = faces.astype(np.int64)

    mesh = trimesh.Trimesh(
        vertices=vertices,
        faces=faces,
        process=False
    )

    mesh.export(output_file, file_type="glb")

def write_stc_to_folder(p : str) -> None:
    global stc
    global source_spaces

    from pathlib import Path

    output_path = Path(get_stc_dir_from_path(p))

    output_path.mkdir(parents=True, exist_ok=True)

    # Already have white source space

    file_name = str(output_path) + f"/{get_stc_file_name()}"

    stc.save_as_surface(file_name, source_spaces["white"])

    stc.save(file_name, overwrite=True)

    gifti_to_glb(file_name + "-lh.gii", file_name + "-lh.glb")
    gifti_to_glb(file_name + "-rh.gii", file_name + "-rh.glb")

and this is the code for reading a .time.gii file:

private byte[] GZipBase64ToUncompressedBytes(string gzipBase64)
        {
            // Step 1: Decode the Base64 text into the raw gzip-compressed bytes
            byte[] gzipped = Convert.FromBase64String(gzipBase64);

            // Step 2: Decompress with GZip to recover the original bytes
            using var input = new MemoryStream(gzipped);
            using var gzip = new ZLibStream(input, CompressionMode.Decompress);
            using var output = new MemoryStream();
            gzip.CopyTo(output);
            return output.ToArray();
        }

        private float[][] LoadTimeCourseData(string path)
        {
            XDocument timeSeries = XDocument.Load(path);

            var dataArrays = timeSeries.Root!.Elements("DataArray");

            var datas = dataArrays.Select((el) =>
            {
                string compressedData = el.Element("Data").Value;
                var uncompressed = GZipBase64ToUncompressedBytes(compressedData);

                float[] data = new float[uncompressed.Length / 4];

                for (int i = 0; i < uncompressed.Length; i += 4)
                {
                    float val;

                    if (el.Attribute("Endian")!.Value == "LittleEndian")
                    {
                        val = BinaryPrimitives.ReadSingleLittleEndian(
                            uncompressed.AsSpan(i, 4));
                    }
                    else
                    {
                        val = BinaryPrimitives.ReadSingleBigEndian(
                            uncompressed.AsSpan(i, 4));
                    }

                    data[i / 4] = val;
                }

                return data;

            }).ToArray();

            Console.WriteLine($"{datas.Length}");

			return datas;
        }

ok so i got this working, this is the vertex shader used for rendering it, and the HotLUT is the same as the hot colormap mne uses, i just saved it as a png and loaded it as a texture, and there are no negative activations:

StructuredBuffer<float> TimeSeries : register(t0);

Texture2D<float4> HotLUT;
SamplerState HotSampler;

float3 Hot(float x)
{
    x = saturate(x);

    return HotLUT.SampleLevel(
        HotSampler,
        float2(x, 0.5), 0
    ).rgb;
}

float SampleAtTimeIndex(int vertID)
{
    return TimeSeries[TimingIndex * VertexCount + vertID];
}

VSOutput VSMain(VSInput input, uint id : SV_VertexID)
{
    VSOutput output;

    float4 world =
        mul(float4(input.Position, 1.0), Model);

    output.WorldPos = world.xyz;

    output.Position =
        mul(world, View);

    output.Position =
        mul(output.Position, Projection);

    output.Normal =
        normalize(
            mul(
                input.Normal,
                (float3x3)Model));

    float sample = SampleAtTimeIndex(id);

    float magnitude = abs(sample);

    float normalized;

    if (magnitude <= P96)
    {
        normalized = 0.0;
    }
    else if (magnitude <= P975)
    {
        normalized = (magnitude - P96) /
                     max(P975 - P96, 1e-6);

        normalized *= 0.5;
    }
    else
    {
        normalized = 0.5 +
            0.5 * (magnitude - P975) /
            max(P9995 - P975, 1e-6);

        normalized = saturate(normalized);
    }

    float3 color = Hot(normalized);

    output.Color = float4(color.x, color.y, color.z, 1.0);

    return output;
}