AWK - Create new subdir and move files into it based on file name

Question

I have loads (around 6000!) mp3's which I'd like to just dump into sub directories, based on the name of the file. Currently they're all in a single subdir. (There are already some existing directories for some artists, I don't wish to touch the files in these however a duplicate overwriting an existing file is fine).

The simplest way I can see of doing this, is by creating a subdirectory based on the file name, as the files are all named artist.artist.album.track.mp3.

After some searching I've come up with:

for x in ./*.mp3; do awk x="${x}" -F. '{split(x,a,"."); mkdir a[1]; mv x a[1]/}'; done

However it tells me there's a syntax error and unterminated regexp:

awk: cmd. line:1: x=./Dizzee.Rascal.Radio.1.Live.Lounge.03.That.s.Not.My.Name.mp3
awk: cmd. line:1:   ^ syntax error
awk: cmd. line:1: x=./Dizzee.Rascal.Radio.1.Live.Lounge.03.That.s.Not.My.Name.mp3
awk: cmd. line:1:     ^ unterminated regexp
awk: cmd. line:1: x=./Dizzee.Rascal.Ultimate.Streetdance.CD1.03.Flex.Dave.Spoon.Mix.mp3
awk: cmd. line:1:   ^ syntax error
awk: cmd. line:1: x=./Dizzee.Rascal.Ultimate.Streetdance.CD1.03.Flex.Dave.Spoon.Mix.mp3
awk: cmd. line:1:     ^ unterminated regexp

I think this is because the file name isn't given within speech marks but I'm not 100% sure?

Can someone help - and bonus points if you can get it to print out the progress too.

Answer 1

Why awk? OK, let's assume awk…

• x="${x}" before what you want to be a program need -v (i.e. -v x="${x}"), otherwise it's interpreted as a program. This is the reason you get syntax errors.
• Your actual program is not meant to be executed per line of input, but per awk invocation; so you need BEGIN.
• You probably did not mean a[1]; maybe a[2].
• The unquoted / (i.e. unquoted in the context of awk, not the shell) is wrong.

• mkdir and mv are not actions in awk. I think they are interpreted as (empty) variables. There is system action, you may use it. But since you need to pass an arbitrary file/directory name, these problems arise:

  • proper quoting;
  • command injection vulnerability (a part of filename may be interpreted as a command).

---

I say forget about awk here. The below code assumes you want directories named from the first two dot-separated segments. Notes:

• a file named foo.mp3 will trigger an error from mkdir -p ./foo.mp3/;
• it's quite easy to modify the code so it uses one, three or another fixed number of dot-separated segments.

for x in ./*.mp3; do

   # remove leading ./
   d="${x#./}"

   # first segment
   d1="${d%%.*}"

   # remainder
   d="${d#*.}"

   # second segment
   d2="${d%%.*}"

   #result
   d="./${d1}.${d2}/"

   mkdir -p "$d" && mv "$x" "$d"
done

---

In any case I advise to create a separate directory to work with:

cp -al my_music/ my_music_copy/
cd my_music_copy/

This uses hardlinks, so it's fast and takes almost no additional disk space. Process the directory structure inside my_music_copy/. If you mess up hard, you can always get rid of my_music_copy/ and start over. Only after you're sure everything is in its place inside my_music_copy/, remove my_music/ and rename my_music_copy/ to my_music/.

---

If there is no fixed number of segments and some heuristics is required, try vidir; something like this:

find . -maxdepth 1 -type f -iname '*.mp3' | EDITOR=vi vidir -

Then you can use the "search and replace" feature of the chosen editor to reliably create desired directories (e.g. replace ./Dizzee.Rascal. with ./Dizzee.Rascal/; but ./Mozart. with ./Mozart/ and ./W.A.Mozart. with ./W.A.Mozart/ or maybe with ./Mozart/ as well). See this answer of mine.

Advantages:

• If something goes wrong, just do not save the file. Abort and start over.
• You don't need to process all the files at once. Files (lines) you don't touch will not be moved. Process a small chunk so you're sure this part is right; save, exit the editor. Invoke the command again to process remaining files.
• You can copy data to yet another tool (e.g. a spreadsheet), process there, paste back, save and exit.