How to install only CORE functionality of MNE?

I am using MNE in my light weight pyqt eeg annotation desktop app. I use it to read edf data, do some processing (add low/high frequency filtering), and some channel manipulations like channel relabeling, changing montage etc. But outside of that, MNE has huge dependencies for visualization, rendering etc. Is there any way to install very light-weight version of mne like headless version, or something?

I did some search, and indeed, mne allows installation on headless environments, but only difference is that it bypasses display detection, but all the dependencies are still there.

Note:
I use mne 1.12.1, and install it using pip install mne

The base dependencies of MNE are already pretty light. What specific dependencies are being installed in your app that you are trying to avoid?