Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add

[Biggy2872]

Note from Joe L:

I reached the character limit for a single post in my original preclear_disk thread and wanted more room to add more release notes!  I split the threads so that I could add new stuff without getting rid of all of these following posts by users of the utility..  I would love to merge the threads back into a single one again, but it will only merge the threads in date order.   That would put my new release notes near the end of a very long set of pages.  For that reason, the original thread with only the release notes is locked, and this one split from it.

The original preclear_disk.sh thread with the release notes is here:
http://lime-technology.com/forum/index.php?topic=2817.0


What is the possibility of including this in the unmenu awk interface? Maybe in a future version.

My thoughts (provided it is possible) would be to include a button beside each disk on the "Disk Mgmt" page in the "Drive Partitions - Not In Protected Array" along with radio button for the options of "no read/write", "1 cycle", "5 cycles", "20 cycles" and "test"...

Cheers, and keep up the good work!

Matt

[Joe L.]

What is the possibility of including this in the unmenu awk interface? Maybe in a future version.

My thoughts (provided it is possible) would be to include a button beside each disk on the "Disk Mgmt" page in the "Drive Partitions - Not In Protected Array" along with radio button for the options of "no read/write", "1 cycle", "5 cycles", "20 cycles" and "test"...

Cheers, and keep up the good work!

Matt

That is a long-term goal, although I would put it on its own plug-in page.   The biggest issue is the display of progress as it performs the clear.  I am taking advantage of a feature of the "dd" command to get its status when writing to the drive.  I would need to re-write that section.

Then, I would need a way to start, and stop the process from the web-browser, and a way to get the periodic update of status as it progresses.  I know I don't want to submit a task and wait 200+ hours for the browser to return...   

First I just want to make sure it is doing the correct "pre-cleared" signature, and not creating a black-hole somewhere in the universe.   

Before I tackle anything else, I need to get the next version of unmenu.awk out published.   bjp999 recently added a bit of logic and figured out how to get it to POST to a plug-in as well as GET.  I've added that to the next version, along with a few fixes.
(non-geek translation... unmenu.awk can now handle more complicated data entry forms)

Glad you like unmenu.awk.  It certainly is turning out to be interesting. 

Joe L.

[WeeboTech]

Then, I would need a way to start, and stop the process from the web-browser, and a way to get the periodic update of status as it progresses.  I know I don't want to submit a task and wait 200+ hours for the browser to return...

Perhaps submit it to the background via batch.
The process can write it's pid in /var/run and log file in /var/log.
The browser interface section can refresh on the log and if need be use the pid file to send a kill to the process and it's children.

[Joe L.]

Then, I would need a way to start, and stop the process from the web-browser, and a way to get the periodic update of status as it progresses.  I know I don't want to submit a task and wait 200+ hours for the browser to return...

Perhaps submit it to the background via batch.
The process can write it's pid in /var/run and log file in /var/log.
The browser interface section can refresh on the log and if need be use the pid file to send a kill to the process and it's children.

Sounds like it could work...  I'll put it on the list if somebody does not get to it first...

I need to get my own array hardware working properly first.  I've determined that one drive tray slot locks up the server when it has a disk installed.  Not sure if it is the cable, or the controller, or the drive tray connector in the case... Time will tell.   (I can't experiment in the evenings as much, as we are using the server to watch movies...)

Joe L.

[bjp999]

This looks very impressive!  Can't wait to try it when I get another disk!

[Joe L.]

I re-seated my existing disk controller cables and interface card in an attempt to diagnose the DMA errors I've been experiencing since attempting to expand my array into the last 4 empty slots.  As I stated earlier, I was getting a DMA error that just locked up the server.

Since these 4 slots are not yet assigned to anything in my array, my data is safe... I do need to test these slots by reading and writing to disks in them, and this preclear_disk.sh script is perfect for this.  I can keep a disk far more active than otherwise, and at no risk to the overall parity protection of my array.

Last night I re-ran a pre-clear cycle of my tiny small 8Gig test drive  It is at the end connector of the first cable of the disk controller.  It ran successfully in about 25 minutes.  I then tried a pre-clear of a much larger 750Gig drive. It is on the end of the second cable off of the same Promise IDE controller.

As you might have guessed, the 750Gig drive took quite a bit longer to pre-read/clear/post-read than my 8Gig drive.  It took just under 10 hours for 1 cycle.  It also experienced some changes to the SMART data. 

The preclear_disk.sh script is designed to take a SMART status report when it starts, and another at its end, and to show you any differences between them if they exist.  In my example screen-shot, the Raw_Read_Error_Rate and See_Error_Rate are un-changed, but their "raw value" changed. (last value on the line)  These are not likely to be problems.  The Airflow_Temperature_Cel changed... also not likely to be a problem.  There was an increase in the Hardware_ECC_Recovered counter.   I'll need to keep an eye on that.  It indicated the hardware in the disk corrected an error it detected in reading the disk.  The unRAID OS never even knew anything as the error-correction-code in the drive's firmware handled the error.

Makes you kind of wonder if all this is also happening on disks in our Windows PCs, and we are not notified by it until it fails to boot...

Here is a screen shot of how it looked when it was done:


I'm going to run this 750Gig drive through a few more pre-read/clear/post-read cycles to see if it changes any more, or if I get any more DMA errors.   First, I'm going to save a copy of my syslog, as the SMART reports are logged there.  That way, if I do have another DMA error lockup, the SMART report in the saved syslog will be available next time for comparison.

Note: the SMART difference output is in "diff" format.  The lines with a leading "<" are from the before SMART report, the lines with a leading ">" are from the after SMART report.   Lines that are unchanged are not shown at all.

Joe L.

[Joe L.]

I just loaded the new 4.4 version of unRAID.  Apparently, Tom has not included the "ncurses" package.

This will break the display of the preclear_disk.sh script.

To fix it, you can either install the "ncurses" package, or, a lot easier, change these few lines in the script

from:
clearscreen=`tput clear`
goto_top=`tput cup 0 1`
screen_line_three=`tput cup 3 1`
bold=`tput smso`
norm=`tput rmso`
ul=`tput smul`
noul=`tput rmul`


to:
if [ -x /usr/bin/tput ]
then
  clearscreen=`tput clear`
  goto_top=`tput cup 0 1`
  screen_line_three=`tput cup 3 1`
  bold=`tput smso`
  norm=`tput rmso`
  ul=`tput smul`
  noul=`tput rmul`
else
  clearscreen=`echo -n -e "\033[H\033[2J"`
  goto_top=`echo -n -e "\033[1;2H"`
  screen_line_three=`echo -n -e "\033[4;2H"`
  bold=`echo -n -e "\033[7m"`
  norm=`echo -n -e "\033[27m"`
  ul=`echo -n -e "\033[4m"`
  noul=`echo -n -e "\033[24m"`
fi


I'll post a new version of the preclear script shortly with these changes.

Edit: updated version now attached to first post in this thread.

Joe L.

[j4ck4l]

This is an excellent utility. I'm using it now to clear my new 1,5TB disk. This is gonna take a while

[Joe L.]

This is an excellent utility. I'm using it now to clear my new 1,5TB disk. This is gonna take a while

I'll bet it will take a while... 
Please let us know how long it does take.... (I hope you only did one cycle, at least at first)

I figure your disk is twice the capacity as mine, but probably twice as fast, so 10 hours or so for one cycle is my guess....

Joe L.

[Joe L.]

I noticed that the 4.4final and 4.5beta releases do not have a working "smartctl" command.  This will not prevent the preclear script from running, but you will be unable to learn if the disk SMART attributes change during the preclear process.

To fix the smartctl program all you need to do is install the missing library it needs.  It can be downloaded from:
 http://slackware.cs.utah.edu/pub/slackware/slackware-12.0/slackware/a/cxxlibs-6.0.8-i486-4.tgz

Then, using file-explorer on your windows PC you can open up
\\tower\flash
and create a packages folder.   You can copy or move the downloaded file there.   From windows it will be at
\\tower\flash\packages\cxxlibs-6.0.8-i486-4.tgz

If you log in via the system console, or via telnet, the flash drive is mounted at /boot.  The new directory you created is therefore at
/boot/packages   Your file will then be at /boot/packages/cxxlibs-6.0.8-i486-4.tgz

Once downloaded, and saved as cxxlibs-6.0.8-i486-4.tgz you can install it by changing to the directory where you downloaded it
(I have all my downloaded packages in the /boot/packages directory, so after logging in on the system console or via telnet
To change directory I type
cd /boot/packages

and install it by typing  :
installpkg cxxlibs-6.0.8-i486-4.tgz

As an alternative, if I did not want to change directory to where I put the file, I could just give the full path to the downloaded file like this:
installpkg /boot/packages/cxxlibs-6.0.8-i486-4.tgz

Once it is installed, the smartctl program will work until you reboot, at which time you will need to re-install it once more.

Joe L.

[j4ck4l]

This is an excellent utility. I'm using it now to clear my new 1,5TB disk. This is gonna take a while

I'll bet it will take a while... 
Please let us know how long it does take.... (I hope you only did one cycle, at least at first)

I figure your disk is twice the capacity as mine, but probably twice as fast, so 10 hours or so for one cycle is my guess....

Joe L.

Running for 13:20 hours now. Post-read @ 30%
approx. 3 minutes per percent so i guess still 3,5 hours to go.

[Joe L.]

This is an excellent utility. I'm using it now to clear my new 1,5TB disk. This is gonna take a while

I'll bet it will take a while... 
Please let us know how long it does take.... (I hope you only did one cycle, at least at first)

I figure your disk is twice the capacity as mine, but probably twice as fast, so 10 hours or so for one cycle is my guess....

Joe L.

Running for 13:20 hours now. Post-read @ 30%
approx. 3 minutes per percent so i guess still 3,5 hours to go.

I'm running a preclear_disk cycle on a 750Gig SATA drive I have plugged into a new 2-port PCI-Bus SATA controller.  It is a very inexpensive controller card, and only rated at SATA 1.0 speeds, but I figure I am very limited by the PCI bus, so it really does not matter.

I'm  9 hours, 20 minutes into the process, and 85% complete in the post-read process. 

Joe L.

[Joe L.]

It took 10 hours, 2 minutes to pre-read/clear/post-read my 750Gig SATA drive.     This is interesting in that it is almost exactly the same time it took for an IDE based drive of the same size.    Read speeds averaged in the 75-80MB/s range.   Write speeds averaged in the mid 60MB/s through mid 70MB/s.  It shows the PCI bus can keep up with a single SATA drive and we are still mostly limited by the drive itself.

I stopped my unRAID array, assigned the newly cleared 750 Gig SATA drive on the devices page to an empty slot, then went back to the main page. I was presented with a display showing a "blue" indicator for the new drive.

The text alongside the "Start" button indicated it would be cleared if it was not already pre-cleared when the array was started. 
I checked the "checkbox" under the "Start" button (to enable it) and then clicked on the "Start" button to start the array.

The screen indicated that it was starting.  After I refreshed it, it showed the new disk as "Unformatted" and the array was up and running.  My array was off-line for perhaps a minute as I assigned the new drive.
 
Only a minute of down-time is a HUGE accomplishment, as in the past it took about 4 hours of down-time to add a 750 Gig drive while it was being cleared.   In addition, I had some confidence in the drive as any marginal sectors would have been identified.

I clicked on the "Format" button, and in a minute or two more I saw the new disk was available for data.

Joe L.

[WeeboTech]

I clicked on the "Format" button, and in a minute or two more I saw the new disk was available for data.

Does it pay to have a FORMAT option in the pre-clear script?

[Joe L.]

I clicked on the "Format" button, and in a minute or two more I saw the new disk was available for data.

Does it pay to have a FORMAT option in the pre-clear script?

No, it would not pay at all to have a FORMAT option. 

If the drive was formatted it would have a file-system type of "83" set in the 16 bytes that define the first partition in the MBR. It would then not have a valid pre-clear signature, and the unRAID software would go about clearing it when you assign it to the array.   After it cleared the drive, you would still have to format it.  For a large drive, you are facing 4 or more hours of down-time as the drive is cleared.

The only other way to add a "formatted" drive to the array would be to use the button labeled "Restore", the array would be on line quickly, but you would lose parity protection for many hours as it would then start a full parity calculation.   On my array, it takes over 12 hours... I'd rather keep the array protected, so this is not the best way to add a new drive for me.

Joe L.