I have continuous sleep EEG data spanning hours and there are multiple bad spans of data resulting from movement, interference etc. However, I have a huge subject sample size and manually annotating these bad spans is very time consuming. (Im refering to these random periods of body movement, electrode pop etc. I seemed to have removed ocular and heartbeat artifact with ICA no problem)
There was a recent paper and accompanying video explaining that artifiact removal produces no to modest improvements. However these are suggestions for ERP design. The information for continuous data in this regard is rather sparse.
So I was wondering about the necessity of removing these bad data spans, especially if my goal is to analyse the power spectrum.
In the meantime, I do understand that hoping to skip this step altogether is poor scientific practice (and wishful thinking ). So if anyone knows any resource for automated bad span removal would be extremely helpful.
As far as “how important is it to exclude these” I think that depends on what fraction of the data has this problem and how huge the artifacts are. If you’re analyzing an 8-hour recording where about 2 hours worth in total are affected by motion artifacts, that seems like a lot to ignore (25% of the data!). But if in an 8-hour recording you have ~10 minutes of artifacts (more like 2% of the data) that seems more reasonable.
A common approach is to quickly scan through all the data (or at least a reasonable fraction of it), pick a small representative subset to manually annotate, then (at least partially) analyze it with and without the annotations to see how much the results change. If they are more or less indistinguishable, now you have a reason to justify your choice to ignore artifacts in the rest of the data.
). So if anyone knows any resource for automated bad span removal would be extremely helpful.