Removing artefacts using ICA for attention study

I am working with EEG signals for attention study.

While the existing literature and general steps for EEG processing state that ICA should be applied after segmenting data into epochs, before further classification, my signal of interest is usually between the small window of stimulus points.

I have applied ICA to the raw data(this raw data has already been bandpass filtered and downsampled), there are still some existing muscle artifacts that exist…but since they exist mainly outside the window of interest for my experiment, I was wondering about applying cleaning methods to the signal.

Are there any resources(mainly books) which I can refer to for understanding EEG epoching and artifact removal better? I am an undergrad student who is fairly new to EEG signal study.

Thank you!