Non-logging of .files on cache

I'm actively using the ability to hide files on the cache drive from the mover script, by placing them in dotted directories. For each such file, two lines are added to the syslog for each run of the mover script: one listing the intent to move and one reporting that the move has failed. This both seems gratuitous and puts accumulative stress on the syslog. I propose that this behaviour is changed permanently, ideally by hidden files being ignored by the mover script instead of the current behaviour.

This should probably be posted in feature requests.

You could (if you choose) remove the -v from the mv lines in the mover script.

This should probably be posted in feature requests.

You could (if you choose) remove the -v from the mv lines in the mover script.

First, there is a logic flaw in the mover script.  That is the real reason for your errors.  There is no reason the mover script should be trying to move anything in the directory named with a leading "."

 

The part of the mover script involved looks like this:

 

(cd /mnt/cache ; find . \

  \( -type d  -regex '[.]/[^.].*'                         -exec mkdir -p /mnt/user0/{} \; \) , \

  \( -type f  -regex '[.]/.*/.*'  ! -exec fuser -s {} \;  -exec mv -v {} /mnt/user0/{} \; \)   \

)

You can remove the "-v" and the log entries will not occur.  However... the script itself is slightly broken.

 

It uses -regex to find folders on the cache drive and uses "mkdir" to create duplicate folders in the user shares.  The regular expression matches all folders that do not have a leading period.  It does not check if the folder already exists, therefore, if the folder already exists in the user-shares, the "mkdir" will fail. and an error message will be written to the syslog.

 

Next, it finds files to move, but does not exclude those in the folders with the leading "." characters.   When it attempts to move them, the move fails, since the destination folder was not first created in the user-shares.  That is why you are getting entries for those files in the syslog. In my opinion, the regular expression on the second line of the find command is not correct.  It needs to exclude files in folders with the leading "." characters.

 

Actually, the mv -v log entries are a system crash waiting to happen.   If you do not reboot, and the syslog fills, because you move a lot of files over a period of days/weeks/months,  and uses all available memory, then you will crash, even if nothing else is wrong.

 

Please, remove the "-v" from the mover script.   

 

If you want to try a fix, the script can probably be corrected by using a more specific regular expression for the files.

 

Instead of

find -type f -regex '[.]/.*/.*'       

 

This matched the pattern of

.       <-current directory

/       <- subfolder delimiter

.*      <- any character followed by any number of any character, including a "/" for sub-sub folders.   Note: this does not exclude folders starting with a "."

/       <- subfolder delimiter

.*      <- any character, followed by any number of any character.  This might include subfolders, and includes the file name to be moved. 

 

It should be 

find -type f -regex '[.]/[^.][^\/]*/.*'

 

.       <-current directory

/       <- subfolder delimiter

[^.]   <- any character EXCEPT a leading "."    this matches the first character of the top level directory as long as it is not a "."

             This excludes the folders with the leading "." characters in their names.

[^\/]*  <- any set of characters EXCEPT a directory delimiter "/", in other words, the remainder of the top level folder name only. (all but the first character of the folder name)

/       <- subfolder delimiter

.*      <- any character, followed by any number of any character.  This might include additional subfolders, and includes the file name to be moved. 

 

The fixed section of mover script would then look like this:

(cd /mnt/cache ; find . \

  \( -type d  -regex '[.]/[^.].*'                                         -exec mkdir -p /mnt/user0/{} \; \) , \

  \( -type f  -regex '[.]/[^.][^\/]*/.*'  ! -exec fuser -s {} \;  -exec mv {} /mnt/user0/{} \; \)   \

)

 

Regular expressions are fun  ;D.

 

Joe L.

As always, thanks for the very useful responses.

 

First, there is a logic flaw in the mover script.  That is the real reason for your errors.  There is no reason the mover script should be trying to move anything in the directory named with a leading "."

 

Indeed. Hence, the proposal above, i.e., make something like your (Joe L.'s) adaptation of the mover script part of the standard distribution.

 

 

(NB! This is is not feature request; more like a bug report ... which means it probably should have been posted in the "unRAID Server 4.3" forum.)

As always, thanks for the very useful responses.

 

First, there is a logic flaw in the mover script.  That is the real reason for your errors.  There is no reason the mover script should be trying to move anything in the directory named with a leading "."

 

Indeed. Hence, the proposal above, i.e., make something like your (Joe L.'s) adaptation of the mover script part of the standard distribution.

 

 

(NB! This is is not feature request; more like a bug report ... which means it probably should have been posted in the "unRAID Server 4.3" forum.)

 

I already sent a PM to Tom, to alert him to look at this thread.    I agree, it is more like a bug report.  It is just that very few people are using "." prefaced directories in the cache drive, thus few error reports like this one.    The "mkdir -p" probably also need to be fixed to not try to create folders that already exist, or it needs to send its error out to dev/null.  That way, the error messages for folders that already exist will not end up in the syslog.   I mentioned it above, but did not foocus on it, as you would only get one message per invocation of the mover script.

 

Joe L.

This was not in 4.4.2 either it seems.  Any status on this?

Yes I'll look into this... brain not able to process regex's at the moment 

Yes I'll look into this... brain not able to process regex's at the moment 

Thanks, Tom.

 

One possible solution is in this preceeding post. 

 

Yes, REGEX expressions can make your head hurt a bit, but they are fun.  ;D

 

Joe L.