Running mover does not successfully move items.


Recommended Posts

Running: 5.0-rc12a

 

When I run "Mover" the items are moved but the items persist on the cache drive after the move. I also given the following error codes in the log.

 

Tried running the new permissions script. No change.

Tried changing minimum free space values. No change.

 

Deactivated add-ons. No change.

 

Snippet from syslog:

 

May 14 20:30:45 Tower logger: mover started

May 14 20:30:45 Tower logger: moving TV Shows/

May 14 20:30:45 Tower logger: ./TV Shows/30 Rock/Season 7/30 Rock - 7x03 - Stride of Pride.mkv

May 14 20:30:45 Tower logger: rsync: get_xattr_data: lgetxattr(".","user.org.netatalk.supports-eas.1qmk2g",0) failed: Exec format error (8)

May 14 20:30:45 Tower logger: .d..t...... ./

May 14 20:30:45 Tower logger: .d..t...... TV Shows/

May 14 20:30:45 Tower logger: .d..t...... TV Shows/30 Rock/

May 14 20:30:45 Tower logger: .d..t...... TV Shows/30 Rock/Season 7/

May 14 20:30:45 Tower logger: >f+++++++++ TV Shows/30 Rock/Season 7/30 Rock - 7x03 - Stride of Pride.mkv

May 14 20:31:20 Tower logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7]

May 14 20:31:20 Tower logger: ./TV Shows/30 Rock/Season 7/30 Rock - 7x02 - Governor Dunston.mkv

May 14 20:31:20 Tower logger: rsync: get_xattr_data: lgetxattr(".","user.org.netatalk.supports-eas.1qmk2g",0) failed: Exec format error (8)

May 14 20:31:20 Tower logger: .d..t...... TV Shows/

May 14 20:31:20 Tower logger: .d..t...... TV Shows/30 Rock/

May 14 20:31:20 Tower logger: .d..t...... TV Shows/30 Rock/Season 7/

May 14 20:31:20 Tower logger: >f+++++++++ TV Shows/30 Rock/Season 7/30 Rock - 7x02 - Governor Dunston.mkv

May 14 20:32:02 Tower logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7]

May 14 20:32:02 Tower logger: skipping apps/

May 14 20:32:02 Tower logger: mover finished

Link to comment

Googling "site:lime-technology.com get_xattr_data: lgetxattr" yielded some results..... (try it)

 

You might want to have a look at:

http://lime-technology.com/forum/index.php?topic=16885.0

Here the problem stemmed from a bad SATA cable. But there could be additional drive related issues.

 

Also check: http://lime-technology.com/forum/index.php?topic=23846.0

 

A good thing to do would be to post the details of your unRAID build (like in your signature) and attaching a SYSLOG.

 

All the best!

Link to comment

NO, he has join the exclusive club, running AFP (or at least at some point, and has been through other Beta and/or RC's) and now has corrupted/damaged AFP extended attribute(s) on the drive(s).

 

You will need to email Tom so he can explain how you go about fixing this, as he has not posted a fix in the public forums here.

Me helping everyone fix it doesnt help in getting Tom to explain why/how (e.g. which beta/RC) it happened in and what is the SUPPORTED fix.

 

Since I fixed mine it has not come back since, so it stemmed from one of the Beta/RC's. No matter what you do (example next RC, etc..) nothing will help. The corrupted/damaged extended attribute(s) are written to your drive(s) and mover (rsync) will always take a sh^t once it gets to trying to read those corrupted/damaged extended attribute(s)

 

 

Link to comment

Thats not the issue, he has nothing to delete (in regards to the files he wants to move), he can prove that easily.

 

ex. take any one of those files (30 Rock - 7x03 - Stride of Pride.mkv) and cut and paste it directly to one of the disks that is apart of the TV Shows share, he will have no problem doing so. But mover will NOT.

 

Link to comment

Thats not the issue, he has nothing to delete (in regards to the files he wants to move), he can prove that easily.

 

ex. take any one of those files (30 Rock - 7x03 - Stride of Pride.mkv) and cut and paste it directly to one of the disks that is apart of the TV Shows share, he will have no problem doing so. But mover will NOT.

True. At this time I can definitely copy directly to inidividual disks.

 

The "mover" script will also copy to the disk. However, it will not delete the files from the cache drive. They persist after the move.

 

 

Link to comment

@Madburg

 

I found some of your other suggestions on another thread. Ran the following command via telnet:

 

getfattr -d /mnt/user0/Music

 

Here is the returned results:

 

root@Tower:~# getfattr -d /mnt/user0/Music

/mnt/user0/Music: user.org.netatalk.supports-eas.64JREC: Exec format error

getfattr: Removing leading '/' from absolute path names

# file: mnt/user0/Music

user.org.netatalk.supports-eas.75SCBQ=0seWVzAA==

/mnt/user0/Music: user.org.netatalk.supports-eas.A32lDv: Exec format error

/mnt/user0/Music: user.org.netatalk.supports-eas.mqLlbG: Exec format error

/mnt/user0/Music: user.org.netatalk.supports-eas.qjdyFe: Exec format error

/mnt/user0/Music: user.org.netatalk.supports-eas.sYS42P: Exec format error

/mnt/user0/Music: user.org.netatalk.supports-eas.szGYYR: Exec format error

 

Ran the same command on another share:

 

root@Tower:~# getfattr -d /mnt/user0/Movies

/mnt/user0/Movies: user.org.netatalk.supports-eas.1WXZ6S: Exec format error

/mnt/user0/Movies: user.org.netatalk.supports-eas.IuYQvG: Exec format error

/mnt/user0/Movies: user.org.netatalk.supports-eas.P3YiW7: Exec format error

/mnt/user0/Movies: user.org.netatalk.supports-eas.TWnVjW: Exec format error

/mnt/user0/Movies: user.org.netatalk.supports-eas.UoCUkX: Exec format error

/mnt/user0/Movies: user.org.netatalk.supports-eas.fZ5N1j: Exec format error

/mnt/user0/Movies: user.org.netatalk.supports-eas.ljbjyO: Exec format error

/mnt/user0/Movies: user.org.netatalk.supports-eas.zbGsks: Exec format error

 

 

Does any of this shed light on what may be going on?

 

Thank you very much for the time and assist!

Link to comment

Yes, thats the information you need to email Tom with.

Thanks Madburg. Just curious, in your exchanges with Tom - did he indicate that a fix was coming in the next release?

 

And would you recommend not running AFP in the future?

I know how to fix it, but its MY approach, and should it be one that Tom finds to be unsupported then I don't want anyone to have issues later.

What I can say is, since I have repaired it, it has not come back (i keep checking about once a week). Thus why I have stated (in several forum posts) it seems as it was introduced by a previous beta and/or RC's.

Have you run ANY other version beside 5.0RC12a on these disks?

 

My exchanges with Tom have ended, my last few emails to him have gone unreplied. He doesn't appreciate criticism from paying customers.

I have not seen any posting by him explaining why/how (e.g. which beta/RC) it happened in and what is the SUPPORTED fix.

 

As for your last question, it would be VERY disappointing for many if AFP is not resolved.

 

Link to comment

Looking at the release notes, changes RC12 = shfs: return correct extended attribute value length

So that points to RC/Betas prior to RC12 did not return correct extended attribute value length, so the damage/corruption could have stemmed from there... ?

 

Was it corrected because it surfaced from AFP tests Or Tom may have been playing with Btrfs in preparations for the cache pool feature and noticed that the correct extended attribute value length was not being returned... Or just thrown in for good measure.

 

Btrfs wiki: http://en.wikipedia.org/wiki/Btrfs

 

File system tree [edit]

User-visible files and directories all live in a file system tree. There is one file system tree per subvolume. Subvolumes can nest, in which case they appear as a directory item (described below) whose data is a reference to the nested subvolume's file system tree.

Within the file system tree, each file and directory objects has an inode item. Extended attributes and ACL entries are stored alongside in separate items.

Within each directory, directory entries appear as directory items, whose right-hand key values are a CRC32C hash of their filename. Their data is a location key, or the key of the inode item it points to. Directory items together can thus act as an index for path-to-inode lookups, but are not used for iteration because they are sorted by their hash, effectively randomly permuting them. This means user applications iterating over and opening files in a large directory would thus generate many more disk seeks between non-adjacent files—a notable performance drain in other file systems with hash-ordered directories such as ReiserFS,[43] ext3 (with Htree-indexes enabled[44]) and ext4, all of which have TEA-hashed filenames. To avoid this, each directory entry has a directory index item, whose right-hand value of the item is set to a per-directory counter that increments with each new directory entry. Iteration over these index items thus returns entries in roughly the same order as they are stored on disk.

Besides inode items, files and directories also have a reference item whose right-hand key value is the object id of their parent directory. The data part of the reference item is the filename that inode is known by in that directory. This allows upward traversal through the directory hierarchy by providing a way to map inodes back to paths.

Files with hard links in other directories have multiple reference items, one for each parent directory. Files with hard links in the same directory pack all of the links' filenames into the same reference item. This was a design flaw that limited the number of same-directory hard links to however many could fit in a single tree block. (On the default block size of 4KB, an average filename length of 8 bytes and a per-filename header of 4 bytes, this would be less than 350.) Applications which made heavy use of same-directory hard links, such as git, GNUS, GMame and BackupPC were later observed to fail after hitting this limit.[45] The limit was eventually removed[46] (and as of October 2012 has been merged[47] pending release in Linux 3.7) by introducing spillover extended reference items to hold hard link filenames which could not otherwise fit.

Link to comment
  • 2 weeks later...

That's sad to hear that you didn't get a reply back after 2 emails, makes one wonder what is the level of support.

If it makes you feel better you're not the only one.

 

There is a blurb about this in the rc13 release

shfs: fixed removexattr() for directories to look at all disks even if xattr not present on some

Link to comment

Yes it seems to point to either a fix or closer to a fix, or so it does not happen to anyone new going further. We'll see if there will be any talking points to it. I already opened the question on the RC13 thread.

 

Unfortunately, RC13 does not change anything. Issue persists.

 

Jun  4 21:32:38 Tower logger: mover started

Jun  4 21:32:38 Tower logger: moving TV Shows/

Jun  4 21:35:06 Tower logger: ./TV Shows/Modern Family/Season 02/Modern.Family.S02E21.mkv

Jun  4 21:35:06 Tower logger: rsync: get_xattr_data: lgetxattr(".","user.org.netatalk.supports-eas.1qmk2g",0) failed: Exec format error (8) (Errors)

Jun  4 21:35:07 Tower logger: .d..t...... ./

Jun  4 21:35:07 Tower logger: .d..t...... TV Shows/Modern Family/Season 02/

Jun  4 21:35:07 Tower logger: >f......... TV Shows/Modern Family/Season 02/Modern.Family.S02E21.mkv

Jun  4 21:35:56 Tower logger: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1042) [sender=3.0.7] (Errors)

Jun  4 21:35:56 Tower logger: skipping apps/

Jun  4 21:35:56 Tower logger: mover finished

Link to comment
  • 2 weeks later...

This is a pretty strange problem.  That extended attribute, user.org.netatalk.supports-eas.1qmk2g, is created by netatalk when it first starts up to determine if the underlying file system supports extended attributes.  netatalk immediately deletes it after successfully creating it.  The fact there are multiple ones not deleted suggests they got there because of a bug that was fixed in -rc13:

 

- shfs: fixed removexattr() for directories to look at all disks even if xattr not present on some

 

The fix keeps this from happening further but does not address the issue of these xattr's still there.

 

The files are not being deleted off the cache disk after being moved because for some reason, though the xattr's appear to show up in the list of xattr's, they cannot be accessed, hence the 'rsync' command in the mover script returns an error which causes the script to not delete the source file on the cache disk.

 

One way to fix this is look at each file/directory still on the cache disk that didn't get deleted and see if the correct copy of it is on some other disk (ie, a duplicate) - you can use the share browser of the webGui for this to display where the duplicates are for each of those files/directories.  If you are satisfied the duplicate is correct, then you can go and manually delete the corresponding file one the cache disk.  Make sense?

Link to comment

That does make sense. I have done this in the past. There have been duplicate files, and I have manually deleted the existing files on the cache drive.

 

If memory serves, the issue persisted after deleting the files.

 

I have one folder on the cache drive that I have set the mover to not move. It the folder containing all of my installed apps. Could the issue on the drive be tied to that folder somehow?

 

 

Link to comment

That does make sense. I have done this in the past. There have been duplicate files, and I have manually deleted the existing files on the cache drive.

 

If memory serves, the issue persisted after deleting the files.

If you can make it happen again, it would be most useful.

 

I have one folder on the cache drive that I have set the mover to not move. It the folder containing all of my installed apps. Could the issue on the drive be tied to that folder somehow?

I don't think so, but how are you setting that folder not to get moved?

Link to comment

netatalk immediately deletes it after successfully creating it.

 

Tom, can you please share where you got this information from.

 

Sure, by looking at the netatalk-2.2.3 source code, file etc/afpd/volume.c.  Here's the pertinent snippet:

 

        if ((sys_setxattr(vol->v_path, eaname, eacontent, 4, 0)) == 0) {
            sys_removexattr(vol->v_path, eaname);
            vol->v_vfs_ea = AFPVOL_EA_SYS;
        } else {
            LOG(log_warning, logtype_afpd, "volume \"%s\" does not support Extended Attributes, using ea:ad i$
                vol->v_localname);
            vol->v_vfs_ea = AFPVOL_EA_AD;
        }

Link to comment

I cleared out the cache drive entirely.

 

Deleted ALL files and folders.

 

Aside from my "apps" folder which exists only on the cache drive. It is enabled as "cache only" in the webGUI share settings section for that share.

 

I then created a folder that mirrors one on my data disks. Placed a test file in it. Attempted to run the mover. Same failure.

 

I then also tried to copy a test file directly to the general "Music" share folder. The item was successfully copied to the cache drive. Tried to run the mover scripts. Same failure.

 

SysLog attached. Running RC-13.

syslog-2013-06-13.txt

Link to comment