best way to migrate data across UNRAID servers?

I built a new UNRAID Server and I want to migrate all files from my previous two servers. One server has all SATA drives and one has IDE drives. Transferring over the network is agonizingly slow. The new server has no IDE drive inputs. Is There a faster way to migrate data onto the new server than over the network? If I pull the SATA drives from my old server and plug them into my new server, what will happen?

Thanks,

Joe M.

Pulling drives and transferring files locally (on the same machine) is fine.

I would strongly recommend using rsync

using something like this:

rsync --stats --progress  -avh    /source/    /destination/

But slow as it may be, rsync over the network might be your most hassle free way to do the IDE drives.

So the Rsync command would be something like ?

rsync --stats --progress  -avh    /TOWER1/Movies    /Tower2/Movies

Or is the something else that should be in there? Tower1 would the network name for the 1st Unraid server and Movies is the share name on that server.

... Transferring over the network is agonizingly slow.

Assuming you have a Gb network, it's not likely that the network is the bottleneck -- it's most likely the write speed to a parity-protected array.    Note that even attaching a drive from your other servers directly to an internal SATA port and then RSync'ing it to the protected array you will still have this limitation.

The fastest way is to copy the data BEFORE you assign a parity drive ... but personally I'd rather just let the copies be somewhat slower and have all of the data fault tolerant as soon as it's written to the array.

The "human time" involved is the same either way -- a few minutes to start a copy.  It's only the "computer time" that's longer ... and that requires NONE of your time.

Note that if you have enough available drive slots and SATA ports, you COULD simply add the drives from your other SATA-based server to your new array ... no copying required.    You would, however, have to do this as part of the initial configuration (or do a "New Config"), as once  you've enabled parity that can't be done.

Data drives crated by unRAID can be added at any time using New Config.

Data drives crated by unRAID can be added at any time using New Config.

Exactly what I said 

... You would, however, have to do this as part of the initial configuration (or do a "New Config"), as once  you've enabled parity that can't be done.

can you add a drive using new config after parity has been enabled?

can you add a drive using new config after parity has been enabled?

New Config wipes the current configuration so the fact that parity was enabled is irrelevant.  After doing a New Config drives with existing data can be added keeping their contents intact.    However parity then needs to be recalculated to get back to a protected state.

If you are asking whether a new drive can be added to a parity protected array without losing its contents and keeping parity intact then the answer is no.    Any drive that is added to a parity protected array is always pre-cleared when you add it (if you have not already done it before trying to add it to reduce array downtime).

So the Rsync command would be something like ?

 

rsync --stats --progress  -avh    /TOWER1/Movies    /Tower2/Movies

 

Or is the something else that should be in there? Tower1 would the network name for the 1st Unraid server and Movies is the share name on that server.

I would share the shares using NFS and then the command

rsync -aruv Tower1:/mnt/user/Movies/ /mnt/user/Movies

would be the syntax if you were running it on the new server.

You can add the n option so it will do a dry run so you can see if it's correct first.

As the others have noted, you will not get a faster speed with the drives being local if you keep the parity on the new server.

I WOULD NOT recommend you put the old SATA drives into the new server as part of the parity protected array. Copying from those old drives to the new drives inside a single parity protected array will run at about 1/2 the speed of a network copy.

If anything, I would assign the old SATA drives 1 x 1 as cache drives in the new array and then run the mover. Make sure the directory structure on the old SATA drives is the same as what you want on the new server if you're doing this. Otherwise, mount the old drives outside of the array and copy or rsync the data to the new array.

At the end of the day, the safest way is to just keep your old parity protected arrays intact and copy over the network.

You guys are very helpful. Thank you.

I just finished copying 18tb to a new server. It was pretty painless using a windows machine to copy between the two unRAID servers. The only thing I wonder about Rsync vs. copying the way I did it was, does rysnc do any type of file verification? I had a handful of files which I could not copy from the original server to the new one due to some weird errors which I presume was due to some issues I had when adding drives to that server. Would using rsync check the hash on the files and verify the copy was successful?

For my go forward auto sync solution I am using "Allway Sync" which runs on windows and allows me to automatically sync the new server with the old one.