(minor) compatibility issues with datalad

Support & Discussions
1

Dear all,

I don’t think this qualifies as bug report, and I am also not sure whether it is a feasible feature request, so I simply report something that I encountered when working with datalad and mne-bids.

When I try to convert a recording, the associated empty room recording cannot be found, because it is not a regular file, but a symlink. The reason that it is a symlink is that I use datalad to store the dataset on a remote server, while having the dataset hierarchy available locally. The advantage being, that I don’t actually have to use my local machine for storage, but can still access the data whenever I need it (by first pulling it from the remote server).

The way datalad is representing files that are part of the dataset, but not physically present, is with symlinks.

What I do in the moment to bypass the problem is to skip the last two lines of path.find_empty_room(), which assert that the file actually exists. Certainly not the best strategy, but better than nothing.

Perhaps this is of interest to someone.

Eduard

2

it would be nice to be able to support that usecase, but off the top of my head I don’t know exactly how to do it. If you have an idea, we’d welcome a PR. @adswa might know?

3

Hey, I couldn’t find the lines of code that were referred to in a quick search, so apologies if I don’t have a good grasp of the problem. Just from reading the description, it reminds me of a similar compatibility issue with pybids: Any dangling JSON file symlink for subject B would cause a crash on operations done on subject A (Dangling symlinks to JSON files crashes BIDSLayout indexing · Issue #631 · bids-standard/pybids · GitHub).
It was fixed by lazily loading JSONs and silently skipping missing metadata when the file doesn’t exist
(FIX: Lazily load metadata, skip when missing data files are missing sidecars by effigies · Pull Request #711 · bids-standard/pybids · GitHub). I can try to take a look, but would appreciate a pointer to the right place in the source code :slight_smile:

4

Oh apologies, I think I must have muted the discussion, at least I haven’t received notification that there were replies.

The line that makes the difference to me is this one here:

I made it work for me by removing the .resolve() in line 608, but not sure whether that would have some adverse effects in other situations, so far I don’t have experiences issues though. If I remember right, mne-python is also resolving symlinks, so it is here not necessary.