jj_uk

New disk stress testing (burn in)

15 posts in this topic Last Reply

Recommended Posts

Does UnRaid provide a way to stress test a new drive? I'd like to test each disk for, say, 24 hrs hard labor before using it for data. - I found 'preclear', buts it seems to be un-supported.

Note, I've not yet installed UnRaid, i'm waiting for some disks to arrive in the post....

Share this post


Link to post

Most on here now say it's no longer required.. I'm not so sure about it. Others suggested tools from vendors but I liked the pre-clear script it was easy and universal. Can't really do much as the dev is no longer working on it. 

Share this post


Link to post
27 minutes ago, witalit said:

the dev is no longer working on it.

 

He is, sort of. Like everyone else apart from the small team at LT, he's a volunteer and real life sometimes gets in the way. The trouble with the Preclear plugin is that the last time he was called away it was left in a non-working state due to updates made to unRAID and the GUI - in fact, it interfered quite badly with the basic functioning of the system and many people lost faith in it and uninstalled it.

 

Preclearing disks has a long and controversial history amongst unRAID users. It started life as a bash script at a time when adding an un-prepared disk to an existing array took the array off-line for several hours while the disk was cleared. A cleared disk is one that is filled entirely with zeros and such a disk can be added to an existing array without invalidating parity - hence the idea of pre-clearing a disk in advance in order to allow it to be added without interrupting the array's availability. The script evolved, as useful things do, into a tool that not only cleared a disk but read back all those zeros to make sure they had been written correctly. In order better to detect bad sectors a optional full read pass was also added before the clearing pass, so read-write-read became the norm. Many people applied the procedure several times over and a routine for comparing SMART parameters before and after each pass was added. The script had evolved into a disk testing and stressing tool. More recently, in keeping with the modern GUI, a plugin was written that removed the need for typing in a command terminal and potentially making a mistake and zeroing the wrong disk. More recently one of the core functions of unRAID itself was overhauled and adding a new disk to the array became much easier. The disk still needs to be cleared before it can be added but the process takes place without affecting the array's availability. At that point the original purpose of Preclear, both the script and the plugin, became redundant, though their potential use as disk testing tools remained.

 

Over the years, as new Linux kernels and tool-sets have been used both the original script and the plugin have needed patches and maintenance in order to keep them working. If that doesn't happen problems arise and earlier this year the general advice was that preclear was dead - use another tool. The trouble is that other tools fall into two categories and neither is ideal. On the one hand there are the tools that hard disk manufacturers or third parties supply that run on other operating systems, such as Windows. On the other there are Linux tools that run from the command line. I use the badblocks command that is included in the unRAID distribution and find it both thorough and effective. The downside is that it involves typing and the consequent possibility of making a mistake and over-writing the wrong disk. I thought badblocks was an ideal candidate for wrapping into a simple plugin but I chose a bad time to make the suggestion. The broken state of the Preclear plugin was causing a lot of support issues for another plugin and emotions were running high on that day and I received a mauling for my "stupid" suggestion. The Preclear plugin itself was marked as incompatible. It has since been updated but I don't know its current state.

 

I test all disks before I use them and I use badblocks from the command line in destructive read/write mode:

badblocks -wv /dev/sdX

This writes and then reads different bit patterns in four passes and leaves the disk full of zeros. This is almost the same state as a precleared disk, except that it lacks the "signature" necessary to allow it to be introduced into the array without being cleared again. If I had the skill needed to write a plugin I'd write one based on badblocks with options for destructive read/write mode that automatically adds the precleared signature, for non-destructive read/write mode and for read-only mode. That would make it easy to use and safer. But I wouldn't dare suggest that someone else might like to rise to the challenge again.

Share this post


Link to post

I'm using preclear as i write on 6.5.1 and it's just fine. No problems at all. Onto my 3rd 8TB drive, each one of 3 cycles, no issues with the plugin (or disks) so far ...

Sent from my LG-D855 using Tapatalk

Share this post


Link to post
1 hour ago, superloopy1 said:

No problems at all.

 

Assuming you're referring to the plugin, I no longer have it installed but looking at the syslogs of people who clearly do shows a lot of spamming by a component called rc.diskinfo that makes the logs difficult to read. Have you noticed that yourself?

Share this post


Link to post
 
Assuming you're referring to the plugin, I no longer have it installed but looking at the syslogs of people who clearly do shows a lot of spamming by a component called rc.diskinfo that makes the logs difficult to read. Have you noticed that yourself?
Cant say i have but thereagain i dont take much notice of the syslog entries either. Unless of course it fills due to the aforementioned 'spamming' as you call it which i've not experienced since reinstalling the plugin. It serves my purpose, is working aok so it stays for the present.

Sent from my LG-D855 using Tapatalk

Share this post


Link to post

I use unraid since 8 years now and never did a preclear on any disk...

But what i "always" do is to zero the first 100GB of each disk i connect to unraid - thats all.

Share this post


Link to post
1 hour ago, Zonediver said:

I use unraid since 8 years now and never did a preclear on any disk...

But what i "always" do is to zero the first 100GB of each disk i connect to unraid - thats all.

That is a perfectly acceptable approach, but in such a case you need to wait until unRAID itself clears the disk before it is ready to be used.   This is less of an issue than it used to be as you can now continue to use the array while the clear is happening whereas before unRAID v6 the array was offline during the clear.

Share this post


Link to post

I will be using it for stress-testing the drives. I'd rather if failed on the pre-clear testing than in the array.

 

Share this post


Link to post
Posted (edited)
2 hours ago, itimpi said:

That is a perfectly acceptable approach, but in such a case you need to wait until unRAID itself clears the disk before it is ready to be used.   This is less of an issue than it used to be as you can now continue to use the array while the clear is happening whereas before unRAID v6 the array was offline during the clear. 

 

That is not completely correct.

Unraid already recognizes the disk as empty and inserts it into the array without deleting it. The disc is just formatted and available immediately.

 

 

 

 

Edited by Zonediver

Share this post


Link to post
5 hours ago, Zonediver said:

I use unraid since 8 years now and never did a preclear on any disk...

But what i "always" do is to zero the first 100GB of each disk i connect to unraid - thats all.

Zeroing the first 100GB doesn't guarantee a clear disk.

 

1 hour ago, Zonediver said:

 

That is not completely correct.

Unraid already recognizes the disk as empty and inserts it into the array without deleting it. The disc is just formatted and available immediately.

Unraid only skips the clearing process when adding a drive to an existing parity protected array if it detects a valid preclear signature.

 

"Empty" is not a well defined term. To some people, it means a drive with no partitions defined, to others, it means a valid partition table, with formatted partitions containing no file references. In no case is "empty" a valid term to describe  a precleared drive.

 

4 hours ago, itimpi said:

That is a perfectly acceptable approach, but in such a case you need to wait until unRAID itself clears the disk before it is ready to be used.   This is less of an issue than it used to be as you can now continue to use the array while the clear is happening whereas before unRAID v6 the array was offline during the clear.

 

This is completely accurate. Unraid will never skip the clearing process without a valid preclear signature on the drive. "Empty" drives do not have the preclear signature, and are not guaranteed to be full of zeroes.

 

 

 

Share this post


Link to post

Lots here that has gone way over my head . I'm sticking with the preclear plugin until such time as it fails me. So far, every disk that hits my array has had a 3 cycle 'burn in' and, touch wood, no failures to date in over 4 years That'll do for me ... each to their own.

 

Sent from my LG-D855 using Tapatalk

 

 

 

 

Share this post


Link to post
On 5/15/2018 at 11:55 PM, John_M said:

 

He is, sort of. Like everyone else apart from the small team at LT, he's a volunteer and real life sometimes gets in the way. The trouble with the Preclear plugin is that the last time he was called away it was left in a non-working state due to updates made to unRAID and the GUI - in fact, it interfered quite badly with the basic functioning of the system and many people lost faith in it and uninstalled it.

 

Preclearing disks has a long and controversial history amongst unRAID users. It started life as a bash script at a time when adding an un-prepared disk to an existing array took the array off-line for several hours while the disk was cleared. A cleared disk is one that is filled entirely with zeros and such a disk can be added to an existing array without invalidating parity - hence the idea of pre-clearing a disk in advance in order to allow it to be added without interrupting the array's availability. The script evolved, as useful things do, into a tool that not only cleared a disk but read back all those zeros to make sure they had been written correctly. In order better to detect bad sectors a optional full read pass was also added before the clearing pass, so read-write-read became the norm. Many people applied the procedure several times over and a routine for comparing SMART parameters before and after each pass was added. The script had evolved into a disk testing and stressing tool. More recently, in keeping with the modern GUI, a plugin was written that removed the need for typing in a command terminal and potentially making a mistake and zeroing the wrong disk. More recently one of the core functions of unRAID itself was overhauled and adding a new disk to the array became much easier. The disk still needs to be cleared before it can be added but the process takes place without affecting the array's availability. At that point the original purpose of Preclear, both the script and the plugin, became redundant, though their potential use as disk testing tools remained.

 

Over the years, as new Linux kernels and tool-sets have been used both the original script and the plugin have needed patches and maintenance in order to keep them working. If that doesn't happen problems arise and earlier this year the general advice was that preclear was dead - use another tool. The trouble is that other tools fall into two categories and neither is ideal. On the one hand there are the tools that hard disk manufacturers or third parties supply that run on other operating systems, such as Windows. On the other there are Linux tools that run from the command line. I use the badblocks command that is included in the unRAID distribution and find it both thorough and effective. The downside is that it involves typing and the consequent possibility of making a mistake and over-writing the wrong disk. I thought badblocks was an ideal candidate for wrapping into a simple plugin but I chose a bad time to make the suggestion. The broken state of the Preclear plugin was causing a lot of support issues for another plugin and emotions were running high on that day and I received a mauling for my "stupid" suggestion. The Preclear plugin itself was marked as incompatible. It has since been updated but I don't know its current state.

 

I test all disks before I use them and I use badblocks from the command line in destructive read/write mode:


badblocks -wv /dev/sdX

This writes and then reads different bit patterns in four passes and leaves the disk full of zeros. This is almost the same state as a precleared disk, except that it lacks the "signature" necessary to allow it to be introduced into the array without being cleared again. If I had the skill needed to write a plugin I'd write one based on badblocks with options for destructive read/write mode that automatically adds the precleared signature, for non-destructive read/write mode and for read-only mode. That would make it easy to use and safer. But I wouldn't dare suggest that someone else might like to rise to the challenge again.

 

 

Interesting read thanks for that. Yes of course I don't blame the dev, I thank him for the great script he created.. but its just a shame no one else can pick it up :-).

Share this post


Link to post
3 hours ago, witalit said:

but its just a shame no one else can pick it up

 

You'll see from the tone of the discussion in the support thread (it starts around page 74) that many seasoned users and potential co-developers consider it toxic, which is a shame. I don't comment in that thread any more.

Share this post


Link to post
1 hour ago, John_M said:

 

You'll see from the tone of the discussion in the support thread (it starts around page 74) that many seasoned users and potential co-developers consider it toxic, which is a shame. I don't comment in that thread any more.

 

Ouch - I had to break reading after a while. Felt like a bit of a witch hunt there.

 

When debating what is best processor, best way to test a disk, best ... there are quite often multiple views. And quite often the different views have merit.

 

It isn't as simple as claiming that there is no use or need testing a disk  before starting to use it. The clear functionality in unRAID just makes the drive content compatible with use in the unRAID array - it represents a quite limited test of the drive. And a significant percentage of drives dies during the first days or weeks.

 

Not sure if unRAID is a hard environment to write plugins for. The basic tests done by the plugin are quite simple to implement. Most work would be code to find which drives aren't already claimed by the array, cache or UD and in a safe way make sure the user only starts data-destructive tests on the intended drives, and fixing an interface that can display progress and final result.

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


Copyright © 2005-2018 Lime Technology, Inc.
unRAID® is a registered trademark of Lime Technology, Inc.