Multiple problems now fixed; can I trust my data?

After I added two more drives to my unraid 5 rc box, for a total of 15 2&3 TB drives, I had problems with preclears and parity checks failing to complete, followed by errors on four disks and a red dot on one (DISK_INVALID) which I rebuilt when I'd established I had 30A worth of drive on one of four 18A rails; i.e. all my problems related to power.

Now that everything appears fixed, and my parity has been tested and passed, can I be 100% about the integrity of the data that's on there? I suppose my question is quite general really, in so far as I'm unsure of what's happening under the hood. How does unraid decide that what's on disk is correct, and if it says everything's okay, can I believe it? Thank you all!

... can I be 100% about the integrity of the data that's on there?

Quick answer:  No

More details:  Since you've had errors reported on multiple disks, it's possible you wrote the wrong data to a disk ... so even if you've always had good parity, that parity could simply be protecting bad data.    If all of your errors (except the disabled disk) were read errors; and if you always verify your writes; then it's likely everything's fine.  But the only way to be 100% certain of that is to compare your data against your backups.

It's a good idea to maintain checksums of your files (MD5s, SHA1's, etc.) ... then it's much easier to verify whether or not any files have been corrupted by simply verifying the checksums.    But in the absence of that, a compare against backups is the only surefire way to confirm nothing has been corrupted.

Thank you Gary - if the errors occurred during a parity check, whilst no shares were set to be exported - would it be safe to say that the errors likely were all read errors? Ie the parity process never writes back to data drives? I suspect the read errors all occurred at the time the PSU let the drives down - they spontaneously showed as missing - does this sound plausible?

Good thinking as regards checksums - shall make a habit of this going forwards.

If a drive reports a read error, UnRAID corrects it and rewrites the correct data ... and if there's no write error, then it's probably all okay.    If a drive reports a write error, UnRAID disables it (that's what happened to one of your drives).

So ... the bottom line is that your assumption is PROBABLY good ... and all of your data is PROBABLY just fine.

But 100% is a very strong criteria 

As for checksums ... there are several packages that do this from within UnRAID.  What I use is a Windows utility -- I just "point" to a drive on UnRAID (e.g. \\Tower\disk1), right-click, and select "Create checksums" to update them (the first time will take several hours as it computes them all;  but after that it only updates those that have new data in their folders).    Verification is just as easy -- same process, except you select "Verify checksums"    This is the utility I use:  http://corz.org/windows/software/checksum/

If a drive reports a read error, UnRAID corrects it and rewrites the correct data ... and if there's no write error, then it's probably all okay.    If a drive reports a write error, UnRAID disables it (that's what happened to one of your drives).

 

So ... the bottom line is that your assumption is PROBABLY good ... and all of your data is PROBABLY just fine.

 

But 100% is a very strong criteria 

 

As for checksums ... there are several packages that do this from within UnRAID.  What I use is a Windows utility -- I just "point" to a drive on UnRAID (e.g. \\Tower\disk1), right-click, and select "Create checksums" to update them (the first time will take several hours as it computes them all;  but after that it only updates those that have new data in their folders).    Verification is just as easy -- same process, except you select "Verify checksums"    This is the utility I use:  http://corz.org/windows/software/checksum/

Thank you for sharing this, very helpful.

Thank you for sharing this, very helpful.

Glad you found it useful ... I looked a LOT -- and tried quite a few alternatives -- before settling on that package.  It works very well; is faster than most; provides a choice of MD5 or SHA1 (or both); and is very simple to use from your Windows client instead of requiring a login to the UnRAID box and Linux command line controls (which I can do, but would prefer to avoid).

Just another thank you gary as I come back to mark the question as solved.

I have calculated hashes for everything on my drive and given the nature of the data was able to compare it to known values; its now a force of habit of mine to routinely check my CRCs just as I check parity every week.

Thank you again for answering my wooly + non-specific question!