Hello
I am trying to read some EEG files using MNE and so far I can read almost all of the data and annotations, but can’t read the content of the comments. Unfortunately I can not share the EEG file but here is the complete description of the problem. I appreciate any input on this and thanks in advance:
**
CODE:**
import mne
print(mne.__version__)
raw = mne.io.read_raw_nihon("FJ00231Z.EEG", preload=False) # OR preload=True
for ann in raw.annotations:
print(ann)
This will output the annotations as expected, but it does not read the comments content. Here is a sample output of the code:
```
The problem is that those annotations with 'description': np.str_('P_COMMENT') contain comments such as this image and I have no idea on how to read those:
Reading .11D, .CMT, .CN2, and .EDF files is currently not supported.
I assume .CMT stands for “comment”? Searching the open issues for “nihon” reveals some other lacking functionality.
Is this a format used in a hospital? I’m not familiar with it myself and perhaps none of the regular developers of MNE-Python even have access to a collection of nihon files to implement these features. MNE-Python is a community project mostly consisting of unpaid volunteers. If you can program a little Python and are interested in adding support for this yourself, we would very much welcome you opening an issue stating your intent of adding this, followed by a pull request that adds the functionality to MNE-Python. We would gladly help you by answering questions about the codebase and the various procedures involved in this.
@wmvanvliet Thanks for your quick reply. And i found the notes I was looking for in that .CMT files, thanks for pointing out. I will try writing some code to read it and hopefully i will do a pull request to add this feature.
Regarding your question: “Is this a format used in a hospital?”
It is used in many hospitals in Iran where I live. We use some outdated devices, software, and formats though (due to USA sanctions). The latest update of this software I used for opening files dates back to 2015. The devices are much older. And we have a massive dataset here all with this file format (at least 2000 long term monitoring recordings - each ~48 hour EEG).
Here is the code to read the comments (Tested only on EEG 2100 devices). I will do my best to do a pull request soon, i think i need to open an issue first.
import re
import string
from dataclasses import dataclass
@dataclass
class Comment:
timestamp: int
text: str
TS_RE = re.compile(rb"(\d{20})")
PRINTABLE = set(bytes(string.printable, "ascii"))
def clean_bytes(b: bytes) -> str:
# keep printable ASCII, including space and newline
cleaned_byte = bytes(c if c in PRINTABLE else ord(" ") for c in b)
cleaned_str = cleaned_byte.decode("ascii", errors="ignore").strip()
cleaned_str = cleaned_str[10:].lstrip().lstrip("\t\n\x0b\r\x0c")
return cleaned_str
def parse_cmt(path: str):
data = open(path, "rb").read()
matches = list(TS_RE.finditer(data))
records = []
for i, m in enumerate(matches):
ts = int(m.group(1).decode("ascii"))
start = m.end()
end = matches[i + 1].start() if i + 1 < len(matches) else len(data)
raw_text = data[start:end]
text = clean_bytes(raw_text)
if text:
records.append(Comment(timestamp=ts, text=text))
return records
records = parse_cmt("ROOT_PATH/NKT/EEG2100/FJ00231Z.CMT")
for r in records:
print("📝")
print(r.timestamp)
print(r.text)
Glad to read that you figured out how to load the comments yourself. Even even happier to hear you would be willing to contribute this to MNE-Python!
Yes, first submit a “feature request” issue in which you describe the problem and how you propose it could be addressed (you can re-use the explanation you give in this forum topic). The idea is that we can discuss the overall approach before actual work starts, so you don’t end up doing a lot of work just for us to say “actually, this way is better” and having to start all over. In your case, the feature is quite straightforward, as you show in your initial implementation above. Although I would ask about text encoding: your implementation forces plain ASCII, while the MNE-Python nihon reader takes the encoding (defaults to utf-8) as a parameter for the user to specify.
This makes for a perfect first contribution I think. The style doesn’t quite match the rest of the MNE-Python codebase yet, but we can go back and forth on that in the actual PR.
I will wait for the response and see what happens.
Also i just noticed that Nihon Kohden comments can actually contain image links. I didn’t cover that aspect in my code snippet.
