correct commads to copy to XFS

http://linux.die.net/man/1/rsync

I have some EXAMPLE's below. I'm trying to move files off a disk to provide better ones.

You will need to adjust for your environment.

 

I'm still clearing a disk to provide a full disk to disk example.

You'll have to be patient.

 

However here's how I moved a directory from one disk to another.

 

My goal was to move my TV folder to another disk.

This was my 2nd copy after the first completed.

root@unRAIDb:/mnt/disk6/Video# rsync -v -aPX TV /mnt/disk5/Video/
sending incremental file list

sent 75,097 bytes  received 408 bytes  151,010.00 bytes/sec
total size is 990,330,898,646  speedup is 13,116,096.93

 

After that completed I did a second compare using checksums.

This takes a long time as the file is read from source and destination again.

This time comparing actual data, not just size/mtime.

root@unRAIDb:/mnt/disk6/Video# rsync -v -rcPX TV /mnt/disk5/Video/ 
sending incremental file list

sent 126,332 bytes  received 416 bytes  11.58 bytes/sec
total size is 990,330,898,646  speedup is 7,813,384.82

 

Here I check the size of data still on the source disk

root@unRAIDb:/mnt/disk6/Video# du -hs TV
923G    TV

root@unRAIDb:/mnt/disk6/Video# find TV | wc -l 
3582

root@unRAIDb:/mnt/disk6/Video# find /mnt/disk5/Video/TV | wc -l  
3582

This shows how many files/directories are on source and destination (assuming I'm not merging)

When I was satisfied I did the same -rc with the --remove-source-files option

 

root@unRAIDb:/mnt/disk6/Video# rsync --remove-source-files -v -rcPX TV /mnt/disk5/Video/
sending incremental file list

sent 126,320 bytes  received 26,052 bytes  13.11 bytes/sec
total size is 990,330,898,646  speedup is 6,499,428.36

 

and then the final cleanup showing number of destination entries.

How many entries still existed on source (empty directories).

 

root@unRAIDb:/mnt/disk6/Video# find /mnt/disk5/Video/TV | wc -l 
3582
root@unRAIDb:/mnt/disk6/Video# find TV                  | wc -l 
376
root@unRAIDb:/mnt/disk6/Video# find TV -type f          | wc -l 
0
root@unRAIDb:/mnt/disk6/Video# find TV -type d -empty   | wc -l         
149
root@unRAIDb:/mnt/disk6/Video# find TV -depth -type d -empty -ls -delete 
(empty directoriy entries are deleted)

root@unRAIDb:/mnt/disk6# find TV | wc -l
find: `TV': No such file or directory
0

There is also a tool provided by someone else to assist. I've never used it.

 

diskmv -- A set of utilities to move files between disks

http://lime-technology.com/forum/index.php?topic=36201.msg337017#msg337017

 

No problem weebo. .time is one thing I have plenty of. I think I have followed these commands..my new disk certainly has data on it but when I look at the gui main page at source disk and destination disk there is a 20gb discrepancy between the two..This is where I am at...don't really want to go forward with moving the next disk onto the former source disk...I don't think I am understanding the output information when I performed the commands..sure enough it took a good couple of days to do..I was wondering if that the different file system formats would show space differently as this is where the discrepancy is..on space that is free.

 

Try making file lists and eyeballing them

replacing ? with disk number.

 

find /mnt/disk? -type f -printf "%P\r\n"  | sort > /mnt/disk?/filelist.txt

 

Some example commands I would do for reviewing.

root@unRAIDb:/mnt/disk6# find /mnt/disk5 -type f  | wc -l
7535

root@unRAIDb:/mnt/disk6# find /mnt/disk6 -type f  | wc -l
14133

root@unRAIDb:/mnt/disk6# du  -hs /mnt/disk5
1.8T    /mnt/disk5

root@unRAIDb:/mnt/disk6# du  -hs /mnt/disk6
4.5T    /mnt/disk6

root@unRAIDb:/mnt/disk6# find /mnt/disk5 -type f -printf "%P\r\n"  | sort > /mnt/disk5/filelist.txt
root@unRAIDb:/mnt/disk6# find /mnt/disk6 -type f -printf "%P\r\n"  | sort > /mnt/disk6/filelist.txt

root@unRAIDb:/mnt/disk6# wc -l /mnt/disk5/filelist.txt /mnt/disk6/filelist.txt 
   7535 /mnt/disk5/filelist.txt
  14133 /mnt/disk6/filelist.txt
  21668 total

root@unRAIDb:/mnt/disk6# diff -u /mnt/disk5/filelist.txt /mnt/disk6/filelist.txt | less 

I was wondering if that the different file system formats would show space differently as this is where the discrepancy is..on space that is free.

 

It's not the first time I've heard this. You cannot go by free space alone, use the filelist, du and wc commands to make sure.

i have had a try but unfortunateley i really do not know enough to understand what is going on..what i need to be doing for my setup..i really just need example code to move disk3 to disk5 and have them verified by checksum and then removed from the source disk, this is not something i would have ever done, my hand has been forced because of all the trouble i have with server crashing, my server is unusable and just completley crashes so i was hoping this might cure it..im close to giving up as i dont have the understanding, this just always worked for me..i was happy with it and now im at a loss.

I have some EXAMPLE's below. I'm trying to move files off a disk to provide better ones.

You will need to adjust for your environment.

 

I'm still clearing a disk to provide a full disk to disk example.

You'll have to be patient.

 

However here's how I moved a directory from one disk to another.

 

My goal was to move my TV folder to another disk.

This was my 2nd copy after the first completed.

root@unRAIDb:/mnt/disk6/Video# rsync -v -aPX TV /mnt/disk5/Video/
sending incremental file list

sent 75,097 bytes  received 408 bytes  151,010.00 bytes/sec
total size is 990,330,898,646  speedup is 13,116,096.93

 

After that completed I did a second compare using checksums.

This takes a long time as the file is read from source and destination again.

This time comparing actual data, not just size/mtime.

root@unRAIDb:/mnt/disk6/Video# rsync -v -rcPX TV /mnt/disk5/Video/ 
sending incremental file list

sent 126,332 bytes  received 416 bytes  11.58 bytes/sec
total size is 990,330,898,646  speedup is 7,813,384.82

 

Here I check the size of data still on the source disk

root@unRAIDb:/mnt/disk6/Video# du -hs TV
923G    TV

root@unRAIDb:/mnt/disk6/Video# find TV | wc -l 
3582

root@unRAIDb:/mnt/disk6/Video# find /mnt/disk5/Video/TV | wc -l  
3582

This shows how many files/directories are on source and destination (assuming I'm not merging)

When I was satisfied I did the same -rc with the --remove-source-files option

 

root@unRAIDb:/mnt/disk6/Video# rsync --remove-source-files -v -rcPX TV /mnt/disk5/Video/
sending incremental file list

sent 126,320 bytes  received 26,052 bytes  13.11 bytes/sec
total size is 990,330,898,646  speedup is 6,499,428.36

 

and then the final cleanup showing number of destination entries.

How many entries still existed on source (empty directories).

 

root@unRAIDb:/mnt/disk6/Video# find /mnt/disk5/Video/TV | wc -l 
3582
root@unRAIDb:/mnt/disk6/Video# find TV                  | wc -l 
376
root@unRAIDb:/mnt/disk6/Video# find TV -type f          | wc -l 
0
root@unRAIDb:/mnt/disk6/Video# find TV -type d -empty   | wc -l         
149
root@unRAIDb:/mnt/disk6/Video# find TV -depth -type d -empty -ls -delete 
(empty directoriy entries are deleted)

root@unRAIDb:/mnt/disk6# find TV | wc -l
find: `TV': No such file or directory
0

 

1. When you say this was your second copy..i assume you ran the same command twice, first one did the copy second double checked it ? so for me to start copying my disk3 to my new disk5 i would write..

/mnt/disk3# rsync -v -aPX /mnt/disk5/

this will copy everything including the parent folder to the empty disk5 ?..i am going to rerun on this again as the array has been briefly in use so not sure if new files would have been written to the disk3 or to the new disk5.

 

2. so now for comparison..the long way and more precise i should apply this command.. ?

/mnt/disk3# rsync -v -rcPX /mnt/disk5/

 

im afraid im lost from there. I know its a bit of a spoon feed, i am being given to many options here for what i need to do, which is just copy disk3 to disk5, verify it, then remove copied files from disk3 so i can then facilatate moving the next disk to disk3 after converting it to XFS...failing this ill have to try and hook up a laptop and do it via a windows enviroment and leave it on for umpteen days with some form of file copy/verification program

Anybody

Grab a file list for safe keeping and comparison, substituting your source disk (in my case I am copying from disk1 to disk3

This creates a filelist of the disk with RELATIVE paths so it can be used for verification to where ever the files are moved.

 

find /mnt/disk1 -type f -fprintf /mnt/disk1/disk1.filelist "%P\n"
wc -l /mnt/disk1/disk1.filelist
12 /mnt/disk1/disk1.filelist

 

OPTIONAL - Examine filelist

root@unRAIDb:/# less /mnt/disk1/disk1.filelist
BACKUPS/Documents/mydoc.txt
tmp/testfile.001
tmp/testfile.002
tmp/testfile.003
tmp/testfile.004
tmp/testfile.005
tmp/testfile.006
tmp/testfile.007
tmp/testfile.008
tmp/testfile.009
tmp/testfile.010
filelist.txt

 

OPTIONAL - Entirely up to the end user, grab a set of md5 hashes. This will take allot of time.

It could help provide confidence and peace of mind.

It's important that you cd to the right disk and use the correct file. The filelist will be  a list of files that are relative to the disk itself.

cd   /mnt/disk1
find . -type f -printf '%P\0' | xargs -0 -L1 md5sum | tee disk1.md5sums

root@unRAIDb:/mnt/disk1# wc -l disk1.filelist disk1.md5sums 
12 disk1.filelist
12 disk1.md5sums
24 total

 

 

OPTIONAL but recommended - Do the rsync in dry run mode from disk to disk with -n

Pay particular attention to the / at the end of each disk path and make sure the second path is where you really want it.

 

rsync -n -v -aPX /mnt/disk1/ /mnt/disk3/

root@unRAIDb:/# rsync -n -v -aPX /mnt/disk1/ /mnt/disk3/ 
sending incremental file list
./
disk1.filelist
disk1.md5sums
BACKUPS/
BACKUPS/Documents/
BACKUPS/Documents/mydoc.txt
tmp/
tmp/testfile.001
tmp/testfile.002
tmp/testfile.003
tmp/testfile.004
tmp/testfile.005
tmp/testfile.006
tmp/testfile.007
tmp/testfile.008
tmp/testfile.009
tmp/testfile.010

sent 518 bytes  received 70 bytes  1,176.00 bytes/sec
total size is 16,247,797,868  speedup is 27,632,309.30 (DRY RUN)

 

Issue the rsync to do the actual copy. (take out dry run -n flag).

root@unRAIDb:/# rsync    -v -aPX /mnt/disk1/ /mnt/disk3/ 
sending incremental file list
./
disk1.filelist
            213 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=15/17)
disk1.md5sums
            621 100%  606.45kB/s    0:00:00 (xfr#2, to-chk=14/17)
BACKUPS/
BACKUPS/Documents/
BACKUPS/Documents/mydoc.txt
            114 100%  111.33kB/s    0:00:00 (xfr#3, to-chk=10/17)
tmp/
tmp/testfile.001
  1,624,779,692 100%  473.28MB/s    0:00:03 (xfr#4, to-chk=9/17)
tmp/testfile.002
  1,624,779,692 100%  437.71MB/s    0:00:03 (xfr#5, to-chk=8/17)
tmp/testfile.003
  1,624,779,692 100%  390.70MB/s    0:00:03 (xfr#6, to-chk=7/17)
tmp/testfile.004
  1,624,779,692 100%  241.36MB/s    0:00:06 (xfr#7, to-chk=6/17)
tmp/testfile.005
  1,624,779,692 100%  168.39MB/s    0:00:09 (xfr#8, to-chk=5/17)
tmp/testfile.006
  1,624,779,692 100%  169.48MB/s    0:00:09 (xfr#9, to-chk=4/17)
tmp/testfile.007
  1,624,779,692 100%  141.46MB/s    0:00:10 (xfr#10, to-chk=3/17)
tmp/testfile.008
  1,624,779,692 100%  124.45MB/s    0:00:12 (xfr#11, to-chk=2/17)
tmp/testfile.009
  1,624,779,692 100%  130.62MB/s    0:00:11 (xfr#12, to-chk=1/17)
tmp/testfile.010
  1,624,779,692 100%  125.77MB/s    0:00:12 (xfr#13, to-chk=0/17)

sent 16,251,765,626 bytes  received 290 bytes  207,028,865.17 bytes/sec
total size is 16,247,797,868  speedup is 1.00

 

OPTIONAL - do it again, making sure no files have changed and if so, then redo it until no files change.

If they keep changing then step back and revisit what is updating your array. You may want to stop that process.

root@unRAIDb:/# rsync    -v -aPX /mnt/disk1/ /mnt/disk3/ 
sending incremental file list

sent 463 bytes  received 15 bytes  956.00 bytes/sec
total size is 16,247,797,868  speedup is 33,991,208.93

 

OPTIONAL - Do a recursive checksum copy to make sure all data compares 100%

This is in dry run mode with -n and using -rc instead of -a

rsync -n -v -rcPX /mnt/disk1/ /mnt/disk3/

root@unRAIDb:/# rsync -n -v -rcPX /mnt/disk1/ /mnt/disk3/ 
sending incremental file list

sent 654 bytes  received 15 bytes  5.13 bytes/sec
total size is 16,247,797,868  speedup is 24,286,693.38 (DRY RUN)

 

If you add more verbosity you can see the details of what is happening as in this example/

rsync -n -vv -rcPX /mnt/disk1/ /mnt/disk3/ 
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
disk1.filelist is uptodate
disk1.md5sums is uptodate
BACKUPS/Documents/mydoc.txt is uptodate
tmp/testfile.001 is uptodate
tmp/testfile.002 is uptodate
tmp/testfile.003 is uptodate
tmp/testfile.004 is uptodate
tmp/testfile.005 is uptodate
tmp/testfile.006 is uptodate
tmp/testfile.007 is uptodate
tmp/testfile.008 is uptodate
tmp/testfile.009 is uptodate
tmp/testfile.010 is uptodate
total: matches=0  hash_hits=0  false_alarms=0 data=0

sent 709 bytes  received 572 bytes  12.03 bytes/sec
total size is 16,247,797,868  speedup is 12,683,682.96 (DRY RUN)

 

OPTIONAL - To double check the hash values using the md5sums from earlier.

Remember we are now cd'ing to disk3 using disk1's md5sums.  This will take a long time.

root@unRAIDb: # cd /mnt/disk3

root@unRAIDb:/mnt/disk3# md5sum -c disk1.md5sums  2>&1 | tee disk1.md5sums.log  
BACKUPS/Documents/mydoc.txt: OK
tmp/testfile.001: OK
tmp/testfile.002: OK
tmp/testfile.003: OK
tmp/testfile.004: OK
tmp/testfile.005: OK
tmp/testfile.006: OK
tmp/testfile.007: OK
tmp/testfile.008: OK
tmp/testfile.009: OK
tmp/testfile.010: OK
disk1.filelist: OK


root@unRAIDb:/mnt/disk3# grep -v OK disk1.md5sums.log 
(if anything is printed here, something went wrong)

root@unRAIDb:/mnt/disk3# grep -c -v OK disk1.md5sums.log 
0

You should have 0 here. 

 

OPTIONAL - you can grab a filelist on the new disk for some extra sanity checking and comparisons.

root@unRAIDb:/mnt/disk3# cd /
root@unRAIDb:/# find /mnt/disk3 -type f -fprintf /mnt/disk3/disk3.filelist "%P\n"

root@unRAIDb:/# wc -l /mnt/disk3/disk1.filelist /mnt/disk3/disk3.filelist 
12 /mnt/disk3/disk1.filelist
15 /mnt/disk3/disk3.filelist
27 total

(There are extra files for a reason)

root@unRAIDb:/# diff -u /mnt/disk3/disk1.filelist /mnt/disk3/disk3.filelist 
--- /mnt/disk3/disk1.filelist   2015-03-15 12:19:22.100529205 -0400
+++ /mnt/disk3/disk3.filelist   2015-03-15 12:40:12.096904379 -0400
@@ -10,3 +10,6 @@
tmp/testfile.009
tmp/testfile.010
disk1.filelist
+disk1.md5sums
+disk1.md5sums.log
+disk3.filelist

 

The final removal of source files by matched checksum.

rsync --remove-source-files -vv -rcPX /mnt/disk1/ /mnt/disk3/

root@unRAIDb:/# rsync --remove-source-files -vv -rcPX /mnt/disk1/ /mnt/disk3/ 
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
disk1.filelist is uptodate
sender removed disk1.filelist
disk1.md5sums is uptodate
sender removed disk1.md5sums
BACKUPS/Documents/mydoc.txt is uptodate
sender removed BACKUPS/Documents/mydoc.txt
tmp/testfile.001 is uptodate
sender removed tmp/testfile.001
tmp/testfile.002 is uptodate
sender removed tmp/testfile.002
tmp/testfile.003 is uptodate
sender removed tmp/testfile.003
tmp/testfile.004 is uptodate
sender removed tmp/testfile.004
tmp/testfile.005 is uptodate
sender removed tmp/testfile.005
tmp/testfile.006 is uptodate
sender removed tmp/testfile.006
tmp/testfile.007 is uptodate
sender removed tmp/testfile.007
tmp/testfile.008 is uptodate
sender removed tmp/testfile.008
tmp/testfile.009 is uptodate
sender removed tmp/testfile.009
tmp/testfile.010 is uptodate
sender removed tmp/testfile.010
total: matches=0  hash_hits=0  false_alarms=0 data=0

sent 709 bytes  received 676 bytes  8.47 bytes/sec
total size is 16,247,797,868  speedup is 11,731,261.00

 

OPTIONAL - double check all files were removed

root@unRAIDb:/# find /mnt/disk1 -depth -type f | wc -l 
0

 

OPTIONAL - Remove empty lingering directories.

root@unRAIDb:/# find /mnt/disk1 -depth -empty -ls
4294967424    0 drwxrwxrwx   2 root     root            6 Mar 15 12:44 /mnt/disk1/BACKUPS/Documents
4294967426    0 drwxrwxrwx   2 nobody   users           6 Mar 15 12:44 /mnt/disk1/tmp

root@unRAIDb:/# find /mnt/disk1/ -depth -empty -ls -delete    
   128    0 drwxrwxrwx   2 nobody   users           6 Mar 15 12:45 /mnt/disk1/
find: cannot delete `/mnt/disk1/': Device or resource busy

 

The final removal of source files by matched checksum.

rsync --remove-source-files -vv -rcPX /mnt/disk1/ /mnt/disk3/

root@unRAIDb:/# rsync --remove-source-files -vv -rcPX /mnt/disk1/ /mnt/disk3/ 
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
disk1.filelist is uptodate
sender removed disk1.filelist
disk1.md5sums is uptodate
sender removed disk1.md5sums
BACKUPS/Documents/mydoc.txt is uptodate
sender removed BACKUPS/Documents/mydoc.txt
tmp/testfile.001 is uptodate
sender removed tmp/testfile.001
tmp/testfile.002 is uptodate
sender removed tmp/testfile.002
tmp/testfile.003 is uptodate
sender removed tmp/testfile.003
tmp/testfile.004 is uptodate
sender removed tmp/testfile.004
tmp/testfile.005 is uptodate
sender removed tmp/testfile.005
tmp/testfile.006 is uptodate
sender removed tmp/testfile.006
tmp/testfile.007 is uptodate
sender removed tmp/testfile.007
tmp/testfile.008 is uptodate
sender removed tmp/testfile.008
tmp/testfile.009 is uptodate
sender removed tmp/testfile.009
tmp/testfile.010 is uptodate
sender removed tmp/testfile.010
total: matches=0  hash_hits=0  false_alarms=0 data=0

sent 709 bytes  received 676 bytes  8.47 bytes/sec
total size is 16,247,797,868  speedup is 11,731,261.00

 

OPTIONAL - double check all files were removed

root@unRAIDb:/# find /mnt/disk1 -depth -type f | wc -l 
0

 

OPTIONAL - Remove empty lingering directories.

root@unRAIDb:/# find /mnt/disk1 -depth -empty -ls
4294967424    0 drwxrwxrwx   2 root     root            6 Mar 15 12:44 /mnt/disk1/BACKUPS/Documents
4294967426    0 drwxrwxrwx   2 nobody   users           6 Mar 15 12:44 /mnt/disk1/tmp

root@unRAIDb:/# find /mnt/disk1/ -depth -empty -ls -delete    
   128    0 drwxrwxrwx   2 nobody   users           6 Mar 15 12:45 /mnt/disk1/
find: cannot delete `/mnt/disk1/': Device or resource busy

 

This is an extremely good write up. I do question if we ever need to run --remove-source-files becuase the plan is to format the old disk to XFS and deleting the files from the old disk first seems like an unnecessary step. Still very good and thorough approach.

This is an extremely good write up. I do question if we ever need to run --remove-source-files because the plan is to format the old disk to XFS and deleting the files from the old disk first seems like an unnecessary step. Still very good and thorough approach.

 

It's optional.  I do it to be sure all the files are moved.

By doing the find  -type f | wc -l you can see if anything is left over.

Eventually there may be a way to remove a disk from the array in an automatic fashion, however it will require the disk to be empty.

 

Also keep in mind this procedure works equally well for sub directories within the disk.

If a person wanted to move /mnt/disk1/TV/ to /mnt/disk2/TV/  then you would want --remove-source-files.

It also works well when doing server to server.

I do it this way pretty often.

im basically just copying your code into a screen session and changing your disk1 to disk3 for my needs...some of the lines are coming back with errors...not sure if there is something i shuld be ommiting..

 

md5sum: It: No such file or directory
md5sum: on: No such file or directory
md5sum: the: No such file or directory
md5sum: Alcohol-thumb.jpg: No such file or directory
md5sum: tv/Glee/Season: No such file or directory
md5sum: 3/Glee: No such file or directory
md5sum: S03E08: No such file or directory
md5sum: Hold: No such file or directory
md5sum: On: No such file or directory
md5sum: To: No such file or directory
md5sum: Sixteen-thumb.jpg: No such file or directory
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
md5sum: tv/Glee/Season: No such file or directory
md5sum: 3/Glee: No such file or directory
md5sum: S03E02: No such file or directory
md5sum: I: No such file or directory
md5sum: am: No such file or directory
md5sum: Unicorn-thumb.jpg: No such file or directory
root@warptower:/mnt/disk3# /mnt/disk3# wc -l disk3.filelist disk3.md5sums
bash: /mnt/disk3#: No such file or directory
root@warptower:/mnt/disk3# /mnt/disk3 wc -l disk3.filelist disk3.md5sums
bash: /mnt/disk3: Is a directory
root@warptower:/mnt/disk3#

Try something like this to make the md5sums file.

It's probably the spaces or quoting characters

 

cd  /mnt/disk3

find . -type f -printf '%P\0' | xargs -0 -L1 md5sum | tee disk3.md5sums

ok it seems to be doing the md5sums as i can see the file has been created on disk3, i am following this to the letter and doing it within a screen session, just for my sanity im getting an error entering this command to examine file list..

 

OPTIONAL - Examine filelist

 

Code: [select]

root@unRAIDb:/# less /mnt/disk1/filelist.txt

BACKUPS/Documents/mydoc.txt

tmp/testfile.001

tmp/testfile.002

tmp/testfile.003

tmp/testfile.004

tmp/testfile.005

tmp/testfile.006

tmp/testfile.007

tmp/testfile.008

tmp/testfile.009

tmp/testfile.010

filelist.txt

 

it is complaining about /# not being a directory, i have tried ommiting it to no avail, im aware its not a crucuial part but i would like to see it happen none the less...i will for now disconnect the screen session and look back tomorrow morning to see progress so i can then carry on with the next set of instruction.

 

Thanks for your time weebo it is very much appreciated.

root@unRAIDb:/# is my prompt, never enter that

Enter what's after the # character

make sure you change the disk reference.

 

Ignore my prompt on any screen capture I've left.

less /mnt/disk1/filelist.txt

My brutally frank assessment is that you should read up more about the Linux command line before proceeding.

Or use a windows Gui tool like puresync or teracopy if uncomfortable.

 

I've been doing this so long that things are obvious and second nature to me.

I might not be the right one in telling you how to do this.

 

before you delete anything, make sure all the files you expect, exist on the destination disk.

I was wondering if that the different file system formats would show space differently as this is where the discrepancy is..on space that is free.

 

It's not the first time I've heard this. You cannot go by free space alone, use the filelist, du and wc commands to make sure.

 

I've converted 3 disks from ReiserFS to XFS so far. On all 3 disks, I have gained a little bit of free space.

 

3TB RFS drive to 3TB XFS drive - Went from 49.5GB to 51.1GB Free

3TB RFS drive to 3TB XFS drive - Went from 10.7GB to 12.3GB Free

2TB RFS drive to 2TB XFS drive - Went from 53.6GB to 54.7GB Free

 

These are media drives and contain mostly larger files. Source and destination drives between copies are of the same model number.

 

I'll post some results from other drives that have smaller files when I get to them.

 

Gary

root@unRAIDb:/# is my prompt, never enter that

Enter what's after the # character

make sure you change the disk reference.

 

Ignore my prompt on any screen capture I've left.

less /mnt/disk1/filelist.txt

My brutally frank assessment is that you should read up more about the Linux command line before proceeding.

Or use a windows Gui tool like puresync or teracopy if uncomfortable.

 

I've been doing this so long that things are obvious and second nature to me.

I might not be the right one in telling you how to do this.

 

before you delete anything, make sure all the files you expect, exist on the destination disk.

 

i sort of knew that and tried ommiting it..i kept on looking then realised that for me i needed to type it without the .txt extension, its on disk3 as disk3.filelist for some reason..clearly the way i must have typed it but i understand what i am seeing now and getting expected result so far, once i have done it once i will be fine with it.

 

I have this readout..

 215548 disk3.filelist
       5 disk3.md5sums
  215553 total

 

baring in mind that previous to you explaining this in more detail i had performed a copy from disk3 to disk5 already but i am doing it again for completeness so is that saying that its basically copied and that theres just a few bits left over

Loady, what I see happening here is potential for significant data loss. The only time I have lost data with unRaid was by user error. The things you are doing now are best done on a test server where the worst that can happen is that you lose some test data.  It's so easy to type a command slightly wrong.

 

Be very careful.  Moving data is one of the easiest ways to lose data due to user's error.

 

i kept on looking then realised that for me i needed to type it without the .txt extension, its on disk3 as disk3.filelist for some reason..clearly the way i must have typed it but i understand what i am seeing now and getting expected result so far, once i have done it once i will be fine with it.

 

OK that's sort of my fault as I did not update the procedure correctly.

 

I have this readout..

 215548 disk3.filelist
       5 disk3.md5sums
  215553 total

 

It's possible that the cd was not done first.

The number of entries in disk3.md5sums should be within 1 file of disk3.filelist

cd  /mnt/disk3

pwd

rm -vi disk3.md5sums

 

find . -type f -printf '%P\0' | xargs -0 -L1 md5sum | tee disk3.md5sums

wc -l /mnt/disk3/disk3.filelist /mnt/disk3/disk3.md5sums

now i have this after those commands...

 215548 /mnt/disk3/disk3.filelist
  215549 /mnt/disk3/disk3.md5sums
  431097 total

i need to replace that command with the one further back in the instructions?. seems to be correct now and should carry on with the next set?

Looks good.

 

The rsync with the -n is just something that's going to print allot of stuff to your screen.

when you take off the -n it will do the actual work.

 

be careful how you type it.

 

after you copy the files with rsync you can do another filelist and wc to compare the number of files on the destination.

 

example:

 

find  /mnt/disk3 -type f -fprintf /mnt/disk3/disk3.filelist "%P\n"

wc -l /mnt/disk1/disk1.filelist  /mnt/disk3/disk3.filelist 

 

These numbers should match if the destination disk *(disk3 in this example)* was empty before the rsync.

wc -l /mnt/disk5/disk3.filelist /mnt/disk5/disk5.filelist
  215548 /mnt/disk5/disk3.filelist
  215552 /mnt/disk5/disk5.filelist

 

the filelist that was of disk3 that was on disk3 has been moved to disk5 now...6 file descrepency ?..i can live with that

trying to set the now empty drive...empty except for 45mb of nothing i want to XFS, when i restart the array it is saying unmountable..the drive doesnt need to be precleared again to change FS ?

trying to set the now empty drive...empty except for 45mb of nothing i want to XFS, when i restart the array it is saying unmountable..the drive doesnt need to be precleared again to change FS ?

At this point it would be unmountable as the file system no longer conforms to the one you have requested.  When you change the setting in the GUI you have merely said what you want - nothing has yet been done to the disk.  You should be offered the option to format that disk.  Once that has been done the disk will mount correctly.