# Change in data shape after the picking **URL:** \ **Category:** Support & Discussions\ **Created:** [April 24, 2024, 3:35pm UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702 "2024-04-24T15:35:03Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![NK221088](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/nk221088/32/3620_2.png) [@NK221088](https://mne.discourse.group/u/NK221088)\ **Post date:** [April 24, 2024, 3:35pm UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/1 "2024-04-24T15:35:03Z")
My group and I are doing a bachelors project using multiple datasets including the **[BIDS-NIRS-Tapping](https://github.com/rob-luke/BIDS-NIRS-Tapping)** made by Doktor Luke. We have made a data loading/preprocessing class which has worked just fine until now. However, suddenly our second dimension of the data was reduced from 56 to 40. We haven’t changed any code, and believe the change stem from the section copied below. Could this change be due to an update, or could it be us having made a change without being aware? We tried running an earlier version of the script, which also returns the new data format… We use version 1.6.1 on a Windows 11 operating system. Thanks in advance! > if self.short\_channel\_correction: > raw\_od = mne.preprocessing.nirs.optical\_density(raw\_intensity) > raw\_od = mne\_nirs.signal\_enhancement.short\_channel\_regression(raw\_od) > > ``` > picks = mne.pick_types(raw_intensity.info, meg=False, fnirs=True) > dists = mne.preprocessing.nirs.source_detector_distances( > raw_intensity.info, picks=picks > ) > raw_intensity.pick(picks[dists > 0.01]) > else: > picks = mne.pick_types(raw_intensity.info, meg=False, fnirs=True) > dists = mne.preprocessing.nirs.source_detector_distances( > raw_intensity.info, picks=picks > ) > raw_intensity.pick(picks[dists > 0.01]) > raw_od = mne.preprocessing.nirs.optical_density(raw_intensity) > > ``` ---
**Author:** ![mscheltienne](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/mscheltienne/32/827_2.png) [@mscheltienne](https://mne.discourse.group/u/mscheltienne)\ **Post date:** [April 25, 2024, 9:46am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/2 "2024-04-25T09:46:40Z")
I’m not sure what you think is wrong here. Doing `raw_intensity.pick(sel)` will pick the channels in `sel` in-place, meaning that it will reduce the dimension of the data array in `raw_intensity` to `(n_sel_channels, n_samples)`. I’m guessing `picks[dists > 0.01].size == 40`. which is why you get 40 channels in `raw_intensity`. Mathieu ---
**Author:** ![NK221088](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/nk221088/32/3620_2.png) [@NK221088](https://mne.discourse.group/u/NK221088)\ **Post date:** [April 26, 2024, 7:28am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/3 "2024-04-26T07:28:47Z")
That would make sense. What is worrying is that this has not happened in the three months we have been doing the project. From one day to the next (Without us changing any code) the dimensions were reduced. ---
**Author:** ![cbrnr](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/cbrnr/32/1409_2.png) [@cbrnr](https://mne.discourse.group/u/cbrnr)\ **Post date:** [April 26, 2024, 7:34am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/4 "2024-04-26T07:34:41Z")
Can you verify with older versions to see when this change was introduced? ---
**Author:** ![NK221088](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/nk221088/32/3620_2.png) [@NK221088](https://mne.discourse.group/u/NK221088)\ **Post date:** [April 26, 2024, 8:17am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/5 "2024-04-26T08:17:19Z")
Our first step in troubleshooting this issue was to check our Git repository from last week and run the code from that version. But even in that older version the dimensions were reduced. This is what led us to the assumption that the functionality of “picks” had been changed or fixed. ---
**Author:** ![mscheltienne](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/mscheltienne/32/827_2.png) [@mscheltienne](https://mne.discourse.group/u/mscheltienne)\ **Post date:** [April 26, 2024, 8:19am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/6 "2024-04-26T08:19:57Z")
The functionality of `picks` has not been changed or modified. I don’t believe any recent bugfix would have resulted in that. One way to troubleshoot is to create an environment with: - your git repository from a week ago (or older) - MNE version 1.3, or 1.4, or 1.5 ---
**Author:** ![cbrnr](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/cbrnr/32/1409_2.png) [@cbrnr](https://mne.discourse.group/u/cbrnr)\ **Post date:** [April 26, 2024, 8:24am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/7 "2024-04-26T08:24:55Z")
I meant older versions of MNE, not older versions of your code. ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [April 26, 2024, 8:30am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/8 "2024-04-26T08:30:04Z")
I believe there was a recent change to picking behavior in that we now include bad channels by default or so…? Could you maybe try with `pick(..., exclude="bads")`? Or maybe this happens during `pick_types()`. I would try with different combinations of `exclude="bads"` and `exclude=()` passed to `pick_types()` and `pick()`. I might be entirely mistaken, though. ---
**Author:** ![NK221088](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/nk221088/32/3620_2.png) [@NK221088](https://mne.discourse.group/u/NK221088)\ **Post date:** [April 26, 2024, 11:11am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/10 "2024-04-26T11:11:30Z")
Thanks, we just tried this, but it doesn’t seem to explain it… ---
**Author:** ![mscheltienne](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/mscheltienne/32/827_2.png) [@mscheltienne](https://mne.discourse.group/u/mscheltienne)\ **Post date:** [April 26, 2024, 11:35am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/11 "2024-04-26T11:35:21Z")
Well, if you tried to downgrade MNE to an older and version and to run your code which was _not_ reducing the dimensionality, you replicated the condition which were _not_ reducing the dimensionality. The only logical explanations are that (1) you did not notice previously that the dimension was reduced by the `pick` operation, or (2) that a parameter in your code was changed leading to a different `pick` selection (e.g. this `dists > 0.01` condition). Both options are coherent with the `pick` mechanism which always selected the channels in-place, and thus reduce the dimensionality of the underlying array. Mathieu ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [April 26, 2024, 11:40am UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/12 "2024-04-26T11:40:15Z")
Can you try to run your code unmodified but with MNE 1.5 maybe? To see if you get back the old behavior? ---
**Author:** ![NK221088](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/nk221088/32/3620_2.png) [@NK221088](https://mne.discourse.group/u/NK221088)\ **Post date:** [April 26, 2024, 12:07pm UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/13 "2024-04-26T12:07:19Z")
We’ve identified the bug! It turns out that after implementing the short channel correction, we never tried to reset the parameter back to False. Upon doing so, we discovered that we had mistakenly reversed the order of the correction and channel picking, resulting in continued data processing where the short channels remained unremoved. Apologies for the inconvenience, but thank you very much for your contributions and assistance! ---
**Author:** ![richard](https://yyz2.discourse-cdn.com/free1/user_avatar/mne.discourse.group/richard/32/15_2.png) [@richard](https://mne.discourse.group/u/richard)\ **Post date:** [April 26, 2024, 12:31pm UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/14 "2024-04-26T12:31:05Z")
Hah! Great to hear you figured it out eventually! ---
**Author:** ![system](https://global.discourse-cdn.com/free1/uploads/mne/original/1X/85cc6bd2b69cb698a166dc6d880fb550510d0144.jpeg) [@system](https://mne.discourse.group/u/system)\ **Post date:** [May 3, 2024, 12:31pm UTC](https://mne.discourse.group/t/change-in-data-shape-after-the-picking/8702/15 "2024-05-03T12:31:07Z")
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.