From 4.6 to 4.7b1 or 5.0b2 with WD Green drives (no jumper)

I´m fully confused how to move forward with my Advanced Format drives as I have just used most of them with an unRAID 4.x format not jumpered. Here are the details of my disk inventory:

 

WD15EARS-00S:
Disk /dev/sda: 1500.3 GB, 1500301910016 bytes
1 heads, 63 sectors/track, 46512336 cylinders, total 2930277168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1              63  2930277167  1465138552+  83  Linux
Partition 1 does not end on cylinder boundary.

WD15EARS-00S:
Disk /dev/sdb: 1500.3 GB, 1500301910016 bytes
1 heads, 63 sectors/track, 46512336 cylinders, total 2930277168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1              63  2930277167  1465138552+  83  Linux
Partition 1 does not end on cylinder boundary.

WD15EARS-00S:
Disk /dev/sdc: 1500.3 GB, 1500301910016 bytes
1 heads, 63 sectors/track, 46512336 cylinders, total 2930277168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1              63  2930277167  1465138552+  83  Linux
Partition 1 does not end on cylinder boundary.

WD10EAVS-00D:
Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes
1 heads, 63 sectors/track, 31008336 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1              63  1953525167   976762552+  83  Linux
Partition 1 does not end on cylinder boundary.

WD2500JS-40T:
Disk /dev/sde: 250.0 GB, 250059350016 bytes
1 heads, 63 sectors/track, 7752336 cylinders, total 488397168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1              63   488397167   244198552+  83  Linux
Partition 1 does not end on cylinder boundary.

 

What is the advise how to move forward? I´m Mac user so would really like to jump immediately to 5.x which supports AFP....some day. Btw having read all the related topics I know now why my unRAID is that slow 

Thanks in advance.

Have you loaded data on these drives?

Yes sure - 58% overall used capacity (only SDE is empty: WD2500JS-40T)

There are a number of options - but this is what I would do.

 

1.  Buy one new 2T drive.

 

2.  Once YOU DECIDE 4.7 is ready for production use (look at the testing thread and other reports on 4.7 before you decide), you can begin this process.

 

3.  Preclear your new 2T drive with the new preclear script using "-A" option (capital A!)  At the end make sure drive is partitioned to start at sector 64.

 

4.  Stop array, unassign your first 2T drive from the devices page, and assign the new 2T drive to the same slot.

 

5.  Go to the main page - should offer to rebuild the disk.

 

6.  Start the array - let the disk rebuild.

 

7.  When complete, ensure you are happy with the disk rebuild.  You can mount the old disk outside the array and run spot comparision tests.  Make sure you are 100% (ok, 99.9999%) confident that the rebuild worked well.

 

8.  Next either preclear the old disk, or run the dd command to zero out the first 8 sectors on the disk.  This is important because otherwise unRAID will not repartition the disk to sector 64 alignment.

 

9.  Stop array, unassign 2nd 2T drive, and assign the disk you just the the "dd" command on in its place.  GO TO THE DRIVE SETTING PAGE AND SET THE OPTION TO USE THE 4K ALIGNMENT PARTITIONS.

 

10.  When you start the array, the second disk contents will rebuild onto the first disk.  Make sure partition is aligned at sector 64.

 

11.  Go back to step 7, rebuilding and then taking the old disk to replace the next disk

 

12.  When you are done you will have one extra 2T drive, which you can preclear (with "-A" option) and add to the array.

 

Note that you can do this whole procedure by overwriting each disk in succession.  I just like to have the backup in case something goes wrong.  Each person's risk tolerance is different though, so that would be your call.

 

These steps assume that the jumper is not installed on any of the disks.

Well, bjp999 that seems that I have something to do now 

 

Thanks a lot for your comprehensive guidance; that will help me a big deal.

@bjp999: would it also be possible to pre-clear the new drive with the -A option so that it has the new format, stop the array and simply copy the data from the old-format-drive to the new-format-drive, shutdown the server, swap the drives?

 

In that case I would be more secure with my data and would go one-by-one....

 

I guess the only drive where I can´t do this is the parity which I need to rebuild....

@bjp999: would it also be possible to pre-clear the new drive with the -A option so that it has the new format, stop the array and simply copy the data from the old-format-drive to the new-format-drive, shutdown the server, swap the drives?

 

In that case I would be more secure with my data and would go one-by-one....

 

I guess the only drive where I can´t do this is the parity which I need to rebuild....

Not "easily" possible.  When you swap the drives the new would have a different model/serial number than the old, and not pre-cleared, and it would be cleared by unRAID first before being re-constructed with the old's contents.

 

Best bet is to go one-by-one, swap in a newly pre-cleared (with -A option) drive for an old.  (The old is your backup)

Let the new get re-constructed by parity and the other disks.  Let the new run for a few days before continuing on to the next.

 

 

Problem is you can't repartition a disk while it is in the array. So to do what you are asking you'd have to drop parity protection for the entire operation. I like the way I laid out better, but that would work. After the copy you would need to remove the disk (that you have already copied data off of) from the array and run the "dd" command to clear the MBR, then add it back to the array, set the setting in unRaid to use the 4k aligned partition, start the array, and then format the disk. Without a parity disk, unRaid will complete the format quickly and then allow you to copy the contents of the next disk to that disk.

 

When all the copying is done you would have to prep the parity disk (dd and set the 4k setting) and assign it to the parity slot.

I agree that using an extra drive and rebuilding each disk in turn is the easiest solution. Don't do any writes to the array during each disk rebuild. That way, you can use the old disk and the others to recover from a failure if you just happen to get unlucky.

 

You can't just copy the data from disk to disk and swap disks using a copy command.

 

Each bit stored on the old and new drive has to match. You could possibly do a low level sector to sector+1 copy and then a "trust my parity" after to swap the 2 disks. Hopefully someone else with more experience in these commands can comment. There would be some very big advantages - Each drive rebuild would not require working all the other drives at the same time and it would be much quicker copying from an array disk to a disk outside the array than rebuilding a drive (likely 3-4 times as fast).

 

I believe you would have to format the new drive with the proper unRAID partition and then the command is along the lines of this;

 

dd if=/dev/sda1 of=/dev/sdb1 bs=4096 conv=noerror

 

to copy the partition from the old drive partition #1 "sda1" to the new drive partition #1 "sdb1".

 

Besides the developer, I'd saye Joe is the expert on what's required to create a valid unRAID partition.

 

Peter

 

"Easiest" way to format a drive as unRAID would with a 4k-aligned sector, is to let unRAID do it for you.

 

Choose the new "MBR 4k-aligned" option in settings.

zero out the existing MBR on the drive with

dd if=/dev/zero count=1 of=/dev/sdX

On the "Devices" page, sssign the new disk as the "cache drive"

 

When you start the array, it should get properly partitioned.  No need to even format it, as if you use "dd" to copy as described previously, it will copy the file-system-formatting with it.

 

Then, perform a full parity check.  There should be no errors. 

Then, stop the array, un-assign the drive you temporarily made the cache drive.

 

It is best if you leave the array stopped for the next step (This example is using /dev/sda as the old disk to be replaced, and /dev/sdb as the new disk you had made the cache drive for creating the 4k-aligned partition)

 

Verify the partition is starting on sector 64 with

fdisk -l -u /dev/sdb

 

use:

dd if=/dev/sda1 of=/dev/sdb1 bs=4096 conv=noerror

 

This will copy the first partition of /dev/sda to the first partition of /dev/sdb.

 

Then, when the "dd" completes... (It may take 8 hours or more, be patient) 

leave the array stopped while the "dd" command is being run.  There can be NO writes to the source disk at all while it is being copied.

 

Assign the new disk in place of the old in the array

Type "initconfig" to set a new initial disk configuration, but before starting the array...

Use the "Trust my Parity" procedure as described in the wiki, THEN start the array.

let the parity check complete.  It should have no errors.

 

Joe L.

Joe - good idea using the cache disk location to format the drive.

 

I hope this is making sense. Let us know which way you proceeded. I'd be interested in knowing how fast this works compared to just rebuilding onto a new drive.

 

One note, be VERY careful with the drive designations (sda1, sdb1 etc) if you attempt this method. The dd command can easily wipe the data on a drive if you enter the wrong one.

 

After the dd command I would also recommend you also check both disks and ensure they appear the same. I think you could leave the new drive assigned as the cache drive and then start the array so you can easily access the data disk and the cache disk and compare them. That way, if you find a mistakenly overwritten data disk you can recover by simply rebuilding the overwritten disk. Once you start the array and check the disks copied correctly then stop the array and do the drive re-assignment, initconfig and trust my parity steps.

 

Peter

 

Thanks a lot guy´s - I´m overwhelmed by the level of help on my question.

 

However, I do believe that I will go the "secure" way with unRAID 4.7 and let unRAID rebuild each disk.

 

I ordered a new 2TB disk which will be my new parity drive (instead of the old 1.5TB drive). So I do hope that there is nothing against starting with the parity drive. This is the way I understood (...including some more questions):

 

1. Preclear your new 2T drive with the preclear script using "-A" option. At the end make sure drive is partitioned to start at sector 64 (fdisk -l -u /dev/sdx)

2. Stop array, unassign your first 2T drive from the devices page, and assign the new 2T drive to the same slot (...I can do that with the server being on power?)

3. Go to the main page - should offer to rebuild the disk.

4. Start the array - let the disk rebuild.

5. When complete, ensure you are happy with the disk rebuild.  You can mount the old disk outside the array and run spot comparision tests.  Make sure you are 100% (ok, 99.9999%) confident that the rebuild worked well (How do I do this? With the disk in an external case?)

6. Next either preclear the old disk, or run the dd command to zero out the first 8 sectors on the disk.  This is important because otherwise unRAID will not repartition the disk to sector 64 alignment (I think I will use the preclear way with the -A option)

7. Stop array, unassign 2nd 2T drive, and assign the disk you just precleared on in its place.  GO TO THE DRIVE SETTING PAGE AND SET THE OPTION TO USE THE 4K ALIGNMENT PARTITIONS (I´m not familiar with this setting as not existing in 4.6 - do I have to do this between step 2 and 3 as well?

8. When you start the array, the second disk contents will rebuild onto the first disk.  Make sure partition is aligned at sector 64.

9. Go back to step 7, rebuilding and then taking the old disk to replace the next disk

10. When you are done you will have one extra 2T drive, which you can preclear (with "-A" option) and add to the array.

One more piece of advice. Run a parity chech on the array before you begin and double check all of the smart reports for signs of disks failing. Always smart to do this before array upgrades or maintenance.

 

unRaid only writes a MBR and creates the partition if it does not already exist. And the new unRaid setting of 4k sector alignment only comes into play when unRaid is creating the partition. So technically, since preclear -A is creating your partitions, the unRaid settings won't matter. But I still suggest setting them correctly.

Thanks a lot guy´s - I´m overwhelmed by the level of help on my question.

 

However, I do believe that I will go the "secure" way with unRAID 4.7 and let unRAID rebuild each disk.

 

I ordered a new 2TB disk which will be my new parity drive (instead of the old 1.5TB drive). So I do hope that there is nothing against starting with the parity drive. This is the way I understood (...including some more questions):

 

1. Preclear your new 2T drive with the preclear script using "-A" option. At the end make sure drive is partitioned to start at sector 64 (fdisk -l -u /dev/sdx)

2. Stop array, unassign your first 2T drive from the devices page, and assign the new 2T drive to the same slot (...I can do that with the server being on power?)

I would power down to swap the drives.  unRAID is not hot-swap aware.

3. Go to the main page - should offer to rebuild the disk.

4. Start the array - let the disk rebuild.

5. When complete, ensure you are happy with the disk rebuild.  You can mount the old disk outside the array and run spot comparision tests.  Make sure you are 100% (ok, 99.9999%) confident that the rebuild worked well (How do I do this? With the disk in an external case?)

Use either unMENU or SNAP to mount the drive not assigned to the array.  You can use a USB drive enclosure, or a spare SATA port on your motherboard/disk-controller.

 

6. Next either preclear the old disk, or run the dd command to zero out the first 8 sectors on the disk.  This is important because otherwise unRAID will not repartition the disk to sector 64 alignment (I think I will use the preclear way with the -A option)

7. Stop array, unassign 2nd 2T drive, and assign the disk you just precleared on in its place.  GO TO THE DRIVE SETTING PAGE AND SET THE OPTION TO USE THE 4K ALIGNMENT PARTITIONS (I´m not familiar with this setting as not existing in 4.6 - do I have to do this between step 2 and 3 as well?

You can do it at any time after booting unRAID 4.7.

8. When you start the array, the second disk contents will rebuild onto the first disk.  Make sure partition is aligned at sector 64.

9. Go back to step 7, rebuilding and then taking the old disk to replace the next disk

10. When you are done you will have one extra 2T drive, which you can preclear (with "-A" option) and add to the array.

Sounds good.

Thanks a lot gents - will go with that. I feel good about this now with your advise.

I would power down to swap the drives.  unRAID is not hot-swap aware.

 

The reference was to changing the drives on the devices page. This does not require a power down. In fact, the server needs to be powered down to connect the extra drive. After that, all the work can be done without any reboots or power down. This is assuming there is a free SATA port available.

 

Peter

 

Preclear run all night long - but at 19% it stopped more or less and the drive light was on all the time. This is the log:

 

Jan 22 08:58:49 Tower kernel: ata6.00: error: { UNC } (Errors)
Jan 22 08:58:54 Tower kernel: ata6.00: qc timeout (cmd 0xec) (Drive related)
Jan 22 08:58:54 Tower kernel: ata6.00: failed to IDENTIFY (I/O error, err_mask=0x5) (Errors)
Jan 22 08:58:54 Tower kernel: ata6.00: revalidation failed (errno=-5) (Minor Issues)
Jan 22 08:58:54 Tower kernel: ata6: hard resetting link (Minor Issues)
Jan 22 08:58:55 Tower kernel: ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 310) (Drive related)
Jan 22 08:58:55 Tower kernel: ata6.00: configured for UDMA/33 (Drive related)
Jan 22 08:58:55 Tower kernel: ata6: EH complete (Drive related)
Jan 22 08:58:58 Tower kernel: ata6.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 (Errors)
Jan 22 08:58:58 Tower kernel: ata6.00: irq_stat 0x40000001 (Drive related)
Jan 22 08:58:58 Tower kernel: ata6.00: failed command: READ DMA EXT (Minor Issues)
Jan 22 08:58:58 Tower kernel: ata6.00: cmd 25/00:08:28:00:5c/00:00:2e:00:00/e0 tag 0 dma 4096 in (Drive related)
Jan 22 08:58:58 Tower kernel: res 51/40:08:28:00:5c/00:00:2e:00:00/e0 Emask 0x9 (media error) (Errors)
Jan 22 08:58:58 Tower kernel: ata6.00: status: { DRDY ERR } (Drive related)
Jan 22 08:58:58 Tower kernel: ata6.00: error: { UNC } (Errors)
Jan 22 08:59:03 Tower kernel: ata6.00: qc timeout (cmd 0xec) (Drive related)
Jan 22 08:59:03 Tower kernel: ata6.00: failed to IDENTIFY (I/O error, err_mask=0x5) (Errors)
Jan 22 08:59:03 Tower kernel: ata6.00: revalidation failed (errno=-5) (Minor Issues)
Jan 22 08:59:03 Tower kernel: ata6: hard resetting link (Minor Issues)
Jan 22 08:59:03 Tower afpd[7405]: logout oliver
Jan 22 08:59:03 Tower afpd[7405]: 63818.17KB read, 49355.21KB written
Jan 22 08:59:03 Tower afpd[9459]: server_child[1] 7405 done
Jan 22 08:59:03 Tower kernel: ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 310) (Drive related)
Jan 22 08:59:03 Tower kernel: ata6.00: configured for UDMA/33 (Drive related)
Jan 22 08:59:03 Tower kernel: ata6: EH complete (Drive related)
Jan 22 08:59:07 Tower kernel: ata6.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0 (Errors)
Jan 22 08:59:07 Tower kernel: ata6.00: irq_stat 0x40000001 (Drive related)
Jan 22 08:59:07 Tower kernel: ata6.00: failed command: READ DMA EXT (Minor Issues)
Jan 22 08:59:07 Tower kernel: ata6.00: cmd 25/00:08:28:00:5c/00:00:2e:00:00/e0 tag 0 dma 4096 in (Drive related)
Jan 22 08:59:07 Tower kernel: res 51/40:08:28:00:5c/00:00:2e:00:00/e0 Emask 0x9 (media error) (Errors)
Jan 22 08:59:07 Tower kernel: ata6.00: status: { DRDY ERR } (Drive related)
Jan 22 08:59:07 Tower kernel: ata6.00: error: { UNC } (Errors)
Jan 22 08:59:12 Tower kernel: ata6.00: qc timeout (cmd 0xec) (Drive related)
Jan 22 08:59:12 Tower kernel: ata6.00: failed to IDENTIFY (I/O error, err_mask=0x5) (Errors)
Jan 22 08:59:12 Tower kernel: ata6.00: revalidation failed (errno=-5) (Minor Issues)
Jan 22 08:59:12 Tower kernel: ata6: hard resetting link (Minor Issues)
Jan 22 08:59:12 Tower kernel: ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 310) (Drive related)
Jan 22 08:59:12 Tower kernel: ata6.00: configured for UDMA/33 (Drive related)
Jan 22 08:59:12 Tower kernel: sd 6:0:0:0: [sdg] Unhandled sense code (Drive related)
Jan 22 08:59:12 Tower kernel: sd 6:0:0:0: [sdg] Result: hostbyte=0x00 driverbyte=0x08 (System)
Jan 22 08:59:12 Tower kernel: sd 6:0:0:0: [sdg] Sense Key : 0x3 [current] [descriptor] (Drive related)
Jan 22 08:59:12 Tower kernel: Descriptor sense data with sense descriptors (in hex):
Jan 22 08:59:12 Tower kernel: 72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00
Jan 22 08:59:12 Tower kernel: 2e 5c 00 28
Jan 22 08:59:12 Tower kernel: sd 6:0:0:0: [sdg] ASC=0x11 ASCQ=0x4 (Drive related)
Jan 22 08:59:12 Tower kernel: sd 6:0:0:0: [sdg] CDB: cdb[0]=0x28: 28 00 2e 5c 00 28 00 00 08 00 (Drive related)
Jan 22 08:59:12 Tower kernel: end_request: I/O error, dev sdg, sector 777781288 (Errors)
Jan 22 08:59:12 Tower kernel: Buffer I/O error on device sdg, logical block 97222661 (Errors)
Jan 22 08:59:12 Tower kernel: ata6: EH complete (Drive related)

Try a new SATA cable. Check the power connection too. It appears you started having trouble with the SATA port connecting to the drive.

 

Peter

The drive was DOA - after a swap all running well.

 

I'm now on my way changing one drive after the other with the Advanced formatting.

 

I'm afraid that I had write activities on the server during the rebuild, hence 2 parity errors now after a check. Should I re-run the parity check correcting the errors?

You should be able to write to the array during a drive rebuild without any issues. I would run another parity check or 2 and make sure they run cleanly. You want to run at least 2 checks without any errors (well, I would want to see at least 2 checks without errors).

 

Peter

 

Peter, quick feedback: the parity check was running with 0 errors the first time. Accordingly to your advise I will restart the check another 2 times. Let's see if the 0 error status remains.

 

Other than that: only 2 drives more to upgrade to the Advanced format 

 

Thanks a lot.