14

I am facing a very confusing behaviour of ls that I cannot search for. It displays that there are contents in a directory, but only when I'm in the directory where these were created from.

Let me show you:

ciprian Documents $ pwd
/Users/ciprian/Documents

ciprian Documents $ ls ../Downloads/rss22/
22rss-USB/


ciprian Documents $ ls ../Downloads/rss22/22rss-USB/
HTML/


ciprian Documents $ cd ../Downloads/rss22


ciprian rss22 $ ls


ciprian rss22 $ ls 22rss-USB/
gls: cannot access '22rss-USB/': No such file or directory

After I cded to ../Downloads/rss22, its contents are displayed as empty. It is also shown empty if I cd ~/Desktop and then I ls ../Downloads/rss22/ , like the first case here.

To me, this indicates that there might be a folder named ../Downloads/rss22 inside Documents. But I cannot figure out how to display it. ls -a ~/Documents does not show anything related to these folders.

What is going on?

The files were created by trying a partial extraction from an archive: unzip 22rss-USB.zip "22rss-USB/HTML/**/*" -d ../Downloads/rss22/

For reference, I am on macOS, though I do not think this is relevant (it's a Unix, right?). I am using Bash 5.1.16 (changed from default zsh).

Output of type pwd: pwd is a shell builtin.

It turns out that if I do cd -P ../Downloads/igarss22/ then it shows the contents that I expect. Where can I see more about this? man cd doesn't show anything about -P.

Now, after cd -P ../Downloads/igarss22/

ciprian Documents $ cd -P ../Downloads/igarss22

ciprian igarss22 $ pwd
/Users/ciprian/Library/CloudStorage/OneDrive/Downloads/igarss22

Right. So I forgot this ; my ~/Documents is a symlink to a folder under my OneDrive:

$ ll ~/ | grep Doc
lrwxr-xr-x    1 ciprian   38 May 19  2022 Documents -> /Users/ciprian/OneDrive/Documents

Which, due to some magic and changes in macOS v12 (Monterey), actually lives under /Users/ciprian/Library/CloudStorage/OneDrive.

I'm still not sure what exactly is going on

Ciprian Tomoiagă
  • 345

2 Answers2

29

The difference is physical vs. logical treatment of ..:

You apparently have two separate Downloads directories:

  • one is /Users/ciprian/Downloads
  • the other is /Users/ciprian/Library/CloudStorage/OneDrive/Downloads

Some shells, including bash and all POSIX shells, have the option of treating .. in the cd command as "take the current path exactly as the user expressed it, cut off the right-most element and change to the resulting path". Across symbolic links this will work as a logical "go back to where we came from", returning from /Users/ciprian/Library/CloudStorage/OneDrive/Documents (symlinked to, and referred by user as ~/Documents) to /Users/ciprian (so the new path at this point is just ~) and then from there to /Users/ciprian/Downloads. If you use cd -L, this is explicitly the behavior you'll get.

Other programs typically won't do this: they will instead follow the physical paths, so for them, changing directories to ../Downloads starting from /Users/ciprian/Library/CloudStorage/OneDrive/Documents will always mean changing to /Users/ciprian/Library/CloudStorage/OneDrive/Downloads. This is also what you'll get with cd -P.

If you don't like this (mis)feature of the shell, there is probably a shell-specific way to disable it. With the bash shell, adding set -P or set -o physical to your ~/.bashrc or similar shell start-up script would make bash behave like all cd commands had the -P option unless the -L option is explicitly used.

Note. The documentation of -P is in man bash (section SHELL BUILTIN COMMANDS) because cd is an internal command of Bash.

Fjor
  • 187
telcoM
  • 96,466
  • Could this also explain https://unix.stackexchange.com/questions/729252/path-resolution-depending-on-pwd-symlinked-dirs ? – MC68020 Jan 12 '23 at 10:43
1

Another possibility relates to OneDrive's Files On-Demand and Folder Redirection options, which are available on MacOS as well as Windows, although possibly limited to OneDrive for Business.

Both of these options provide features that remove local copies of folders and files within One Drive enabled directory trees, in order to save space on local volumes.

Local files can be manually removed by using 'Clear Space' actions available in the One Drive client but I believe that one or both of the above features will remove local copies of files/folders that have not been accessed in 3 x months period.

Pointers to the cloud copy of such resources are retained, and when accessed using native file management applications (e.g. Explorer/Finder) or the OneDrive client, OneDrive will perform a just-in-time download to make them available locally.

The problem is that this does not work well with most *nix shells, which will list files & folders that only exist as cloud pointers, but won't trigger synchronisation when actions are preformed on them, such as opening a file or traversing a directory.

This is an issue I encounter fairly frequently when accessing OneDrive for Business enabled directories from bash shells in WSL.

Such files and folders need to be re-synchronised back to the local device before you need to process them in a shell or you should use the Always keep on this device option to ensure that they are available locally and not removed to optimise space.

DoubleDipped
  • 11