myMain Questions

I just installed UnMENU/myMain to get more information on my unRAID server, and after setting this up I am getting some messages in unMain that I wanted clarity on.

I recently precleared several 4TB drives, some using JoeL's script and some using BJP's updated script. 2 of my 4TB drives report a potential HBA issue (as shown in the picture). I dont' know if these tie into the ones I cleared a specific way (i.e. BJP's revised post-clear) or not. I am sure the disk10 was done with the updated script, but not sure about disk8.

I am also seeing messages about GPT Warning. It is suggesting using GNU Parted. Is this something I need to worry about? Do I need to empty these disks to fix this issue?

I don't know if it's worth posting a syslog or not. I've started with just the myMain screen shot but can add the syslog if recommended.

I just installed UnMENU/myMain to get more information on my unRAID server, and after setting this up I am getting some messages in unMain that I wanted clarity on.

 

I recently precleared several 4TB drives, some using JoeL's script and some using BJP's updated script. 2 of my 4TB drives report a potential HBA issue (as shown in the picture). I dont' know if these tie into the ones I cleared a specific way (i.e. BJP's revised post-clear) or not. I am sure the disk10 was done with the updated script, but not sure about disk8.

 

I am also seeing messages about GPT Warning. It is suggesting using GNU Parted. Is this something I need to worry about? Do I need to empty these disks to fix this issue?

 

I don't know if it's worth posting a syslog or not. I've started with just the myMain screen shot but can add the syslog if recommended.

myMain has a list of common sizes of normal commercial disks. It compares the size of the disk with that size with one of those values. If it doesn't find an exact match it suspects that the motherboard may have installed an HPA on the disk. HPAs are not the end of the world, but for a number of reasons motherboards that add HPAs to disks can be dangerous to your array.

As disks get bigger we have had to add entries to the list of normal sizes. The most current new entry is for 4T drives, and the value is 3,907,018,532.

Just above the drive table, to the right side, there is a prompt that says "Select View". Click on the view called "Detail." Find the 4T drive in the list that that is reporting an possible HPA issue, and look at the value in the "Size (k)" column. Tell me what number you see there. It should match the value above.

I have also noticed the "GPT warning" message. It is not causing any harm. Need to investigate that further.

I've attached the detail screenshot, but the two disks in question are showing 3,906,985,768.

Should I be re-formatting these drives?

I've attached the detail screenshot, but the two disks in question are showing 3,906,985,768.

 

Should I be re-formatting these drives?

Interesting.  Re-formatting won't change the size of the disk if you have an HPA issue.  You'll need to use a tool like HDAT2 (works very well) or follow the procedure that's been outlined elsewhere in the forum using Linux tools to fix it.

Note that there's NO issue just using the drives are they are, except you lose a tiny bit of space (32,764 bytes to be precise) that you likely wouldn't have ever used anyway (How many 4TB drives are filled to within the last 32Kbytes ??).    The only negative thing is you could not re-assign either of these drives as the parity drive.

I've attached the detail screenshot, but the two disks in question are showing 3,906,985,768.

 

Should I be re-formatting these drives?

I am a little out of my area of expertise here. May need Joe L. But notice that disks 8 and 10 exhibit this issue, but disk 9 (same size disk) does not.

I would suggest you run the "fdisk -l /dev/sdX" where /dev/sdX is you disk's device designator (e.g., /dev/sdz) on each of the three drives. You can find the device info on the "default view" of myMain OR by doing the configuration change below to the detail view. Once you've run these 3 command, post the results.

Now to add the device column to the detail view

1. Click where is says "Detail view" in a large font to the left of the refresh icon near the top of the screen. You might not think it is clickable but it is.

2. A new window or tab will appear. Click the radio button on the field called "dev"

3. Click the little white block between the "disk" and the "share" blocks on the "Fields" row near the top of the screen.

4. You should now see the "dev" field show up as a bluish box between "disk" and "share".

5. Click save.

Go back to myMain and refresh. You should see the three letter device designator (e.g., sdz) shown for each drive on the details view.

There are many fields and you can customize them as you like on each view. I don't think anyone uses this feature, except me, but this is the "my" part of "myMain" - seeing the data you want to see on the view you want to see it on, ordered the way you want it ordered. If ever you want to go back to the stock version of a view, you can always hit the "Default" button and poof - your customized version is gone and you can start again.

I fear my instructions previously published made this seem harder than it is. Hope this helps!

I've attached the detail screenshot, but the two disks in question are showing 3,906,985,768.

 

Should I be re-formatting these drives?

 

Interesting.  Re-formatting won't change the size of the disk if you have an HPA issue.  You'll need to use a tool like HDAT2 (works very well) or follow the procedure that's been outlined elsewhere in the forum using Linux tools to fix it.

 

Note that there's NO issue just using the drives are they are, except you lose a tiny bit of space (32,764 bytes to be precise) that you likely wouldn't have ever used anyway (How many 4TB drives are filled to within the last 32Kbytes ??).    The only negative thing is you could not re-assign either of these drives as the parity drive.

Gary - I'm thinking this has nothing to do with a motherboard added HPA, but that this is a byproduct or remnant of the preclear issue faced earlier.

... An afterthought:  Have either of these drives ever been installed in a different PC ??  If these are new drives that you've simply added to UnRAID on this PC it seems you MAY have found a bug in the pre-clear revisions bjp made.  Did you pre-clear these with the 32-bit or 64-bit implementation?

I've attached the detail screenshot, but the two disks in question are showing 3,906,985,768.

 

Should I be re-formatting these drives?

 

Interesting.  Re-formatting won't change the size of the disk if you have an HPA issue.  You'll need to use a tool like HDAT2 (works very well) or follow the procedure that's been outlined elsewhere in the forum using Linux tools to fix it.

 

Note that there's NO issue just using the drives are they are, except you lose a tiny bit of space (32,764 bytes to be precise) that you likely wouldn't have ever used anyway (How many 4TB drives are filled to within the last 32Kbytes ??).    The only negative thing is you could not re-assign either of these drives as the parity drive.

I think I can live without the missing space, but am curious on the cause of this. One of the drive is basically empty so I can fix it, move the data from the other 4TB and then fix that one, but since I use the same process on each drive I don't understand how this issue arose.

I am assuming this process will wipe out the partition, which is fine, but will I need to re-clear the disk before it's reused? Or will UnRAID want to clear it itself again (basically hanging the GUI/Array for 10 hours)?

I've attached the detail screenshot, but the two disks in question are showing 3,906,985,768.

 

Should I be re-formatting these drives?

 

I am a little out of my area of expertise here. May need Joe L. But notice that disks 8 and 10 exhibit this issue, but disk 9 (same size disk) does not.

 

I would suggest you run the "fdisk -l /dev/sdX" where /dev/sdX is you disk's device designator (e.g., /dev/sdz) on each of the three drives. You can find the device info on the "default view" of myMain OR by doing the configuration change below to the detail view. Once you've run these 3 command, post the results.

 

Now to add the device column to the detail view

 

1. Click where is says "Detail view" in a large font to the left of the refresh icon near the top of the screen. You might not think it is clickable but it is.

2. A new window or tab will appear. Click the radio button on the field called "dev"

3. Click the little white block between the "disk" and the "share" blocks on the "Fields" row near the top of the screen.

4. You should now see the "dev" field show up as a bluish box between "disk" and "share".

5. Click save.

 

Go back to myMain and refresh. You should see the three letter device designator (e.g., sdz) shown for each drive on the details view.

 

There are many fields and you can customize them as you like on each view. I don't think anyone uses this feature, except me, but this is the "my" part of "myMain" - seeing the data you want to see on the view you want to see it on, ordered the way you want it ordered. If ever you want to go back to the stock version of a view, you can always hit the "Default" button and poof - your customized version is gone and you can start again.

 

I fear my instructions previously published made this seem harder than it is. Hope this helps!

Thanks, I will give this a try and post the results. It will likely be later tonight when I get the chance.

... An afterthought:  Have either of these drives ever been installed in a different PC ??  If these are new drives that you've simply added to UnRAID on this PC it seems you MAY have found a bug in the pre-clear revisions bjp made.  Did you pre-clear these with the 32-bit or 64-bit implementation?

No, I've never used them outside of unRAID, and the pre-clear would have all been done on the 64-bit version.

I just went back through the preclear reports and one was done using BJP's script, and one was using v13 of JoeL's script.

What is interesting though is the two disks in question were both taken from external WD My Book enclosures. They are the only ones I've had like this (the other were purchased as internal disks). Not sure if that matters in the least, but is an interesting coincidence at a minimum.

I am assuming this process will wipe out the partition, which is fine, but will I need to re-clear the disk before it's reused? Or will UnRAID want to clear it itself again (basically hanging the GUI/Array for 10 hours)?

Actually, if you use HDAT2 to do it -- and I assume the same thing is true with the Linux commands, but I'm not a "Linux guy" so I can't say for sure -- it will simply change the reported size of the disk to the correct value.  The current partition structure won't change ... you'll just have some unallocated space after it.    What impact this will have in Linux I don't know.  If it was a Windows drive, nothing would change at all ... there'd just be some unallocated space after the partition unless you subsequently used a partition tool to expand the partition.

I'm curious as to whether this is actually an HPA issue, or if the revised pre-clear has simply not used all of the drive's space for the partition it created.    Since bjp is both the author of MyMain and also made the pre-clear revisions, he can answer that 

In any case, if the drive is subsequently reported with a different size, I suspect UnRAID will indeed treat it as a new/different drive.    I don't know for certain, but I suspect the end result will be that it wants to rebuild the drive ... which should work fine PROVIDED you only change ONE of the drives at a time -- and then wait for the rebuild to complete.    From UnRAID's perspective, it'll be as if you changed to a larger drive ... but if you change both at once it'll think there are TWO new drives, which, of course, is an error UnRAID can't protect against, so it won't be able to rebuild either one.

What is interesting though is the two disks in question were both taken from external WD My Book enclosures. They are the only ones I've had like this (the other were purchased as internal disks). Not sure if that matters in the least, but is an interesting coincidence at a minimum.

BINGO !!  Certainly NOT a "coincidence."    I think you've just identified the issue ... they almost certainly have an HPA (probably has something to do with WD's SES system on these drives ... but the reason is irrelevant).

I think you've just "cleared" the changes to pre-clear as being just fine 

So ... up to you whether you want to "fix" the issue, or just leave them as they are.    The ONLY "problem" this creates is the warning in MyMain for those drives.

The ONLY "problem" this creates is the warning in MyMain for those drives.

... and bjp's customization options would, I believe, let you set these drives to ignore that warning -- so you wouldn't see it anymore either unless you revert to the "Default" view.  He outlined this in his last post ... I'm sure if you need detailed help in doing it he'll be glad to oblige if you ask 

I'm curious as to whether this is actually an HPA issue, or if the revised pre-clear has simply not used all of the drive's space for the partition it created.    Since bjp is both the author of MyMain and also made the pre-clear revisions, he can answer that 

Although I made a few changes to the preclear script, it was only to add some logging so myMain could see the status and speed up the post read verify. I made no functional changes to how the partitions are created or how the preclear signature if applied. I am as ignorant as you about how this part of preclear works.

What is interesting though is the two disks in question were both taken from external WD My Book enclosures. They are the only ones I've had like this (the other were purchased as internal disks). Not sure if that matters in the least, but is an interesting coincidence at a minimum.

 

BINGO !!  Certainly NOT a "coincidence."    I think you've just identified the issue ... they almost certainly have an HPA (probably has something to do with WD's SES system on these drives ... but the reason is irrelevant).

 

I think you've just "cleared" the changes to pre-clear as being just fine 

 

So ... up to you whether you want to "fix" the issue, or just leave them as they are.    The ONLY "problem" this creates is the warning in MyMain for those drives.

myMain has an easy way to suppress the HPA warning.

If you set the "hpa_ok" attribute to 1 on the drive settings page, you can turn off that HPA warning.  Just click on the drive's ID in the ID column (last few letters of the SN).  The drive settings dialog will open.  Find the "Other Notes" field. Below it should be a number of blank rows where you can enter custom attributes. (There probably aren't any defined yet, but you might have some if you had override values for any smart issues). Find the fist blank row and enter hpa_ok in the first entry field, and 1 in the value field (see below for a screenshot). Click Save (very important). The HPA warning will go away for that drive.

Obviously these disks would be poor candidates for parity disks, because the size would be just shy of the size needed to protect a 4T drive without the HPA. I don't think there are any other issues, but not 100% sure if the alignment would have been altered. I doubt it.

...  I'm sure if you need detailed help in doing it he'll be glad to oblige if you ask 

... or even if you don't  :)

Clearly I'd suggest you add those fields in MyMain so you don't see the warning.    Other than that, I wouldn't bother to change anything.  As I noted before, the ONLY issue with those drives is that they can't be assigned as your parity drive ... but by the time your current 4TB parity drive needs replaced, you'll likely be doing it with a 5TB or 6TB drive anyway 

... the HPA has no bearing on the alignment of the partition you're using.

bjp ==> By the way, the customization options in MyMain are slick.  I'd never realized they were even there.

Is there a list of all the options posted somewhere?  [Or a button in MyMain that shows them?]

bjp ==> By the way, the customization options in MyMain are slick.  I'd never realized they were even there.

 

Is there a list of all the options posted somewhere?  [Or a button in MyMain that shows them?]

If you click on the stylized myMain in my sig, you will get a screen with lots of links to info about myMain. The main instruction pages (screenshots with bubble help) is the closest thing to what you are requesting. There are screenshots of all of the configuration screens and configuration options available. Some of the bubble help may be unclear or confusing (I can only guess based on the fact that no one seems to understand any of the things it says) - feel free to point out anything confusing and I will explain it in a post and link the question and answer to help future users.

Thanks!!

Thank you guys for the responses and helping me clear this up. As gary mentioned I will likely switch to a 5TB or 6TB parity drive when I can so this will become moot.

It's something to note though I guess when you buy a cheap external drive - it may not work out exactly as you thought off the bat.

I am just glad I didn't try and use one of those external drives as parity as who know where I'd be once I added a normal 4TB data disk into the mix.

I am just glad I didn't try and use one of those external drives as parity as who know where I'd be once I added a normal 4TB data disk into the mix.

Actually that wouldn't have been a problem -- UnRAID wouldn't have let you add the drive 

... of course, that would been a real PITA -- but you could have then simply made the new drive your parity drive, and then added the slightly smaller unit as another data drive.    But clearly it's best that you didn't encounter that !!

... and if you ever need an extra 64K of storage in your array, you could always "fix" your 2 drives and get back that wasted 32k in each of them  8)

... and if you ever need an extra 64K of storage in your array, you could always "fix" your 2 drives and get back that wasted 32k in each of them  8)

Woohoo. 

Let's hope it never comes to that!

Clearly I wasn't serious about doing that 

If your multi-TB array gets down to where you need 32KB more to add a file, it's safe to say it's TOO FULL 

No kidding. It's like getting a drink and trying to make sure you fill it right up to the rim. I'd rather just get a second glass 

Actually, I try very hard to fill the drives on my media server as much as possible ... to the point where the last few DVDs I've added in each case were "hand selected" to maximize the total amount of used space (I've even moved a few around to do that).    And NONE of those drives come even close to only the last 32KB of space -- the smallest amount of free space on any of those drives is 22MB ... about 687 times the amount of space you're "wasting" due to the HPA's 

By the way, I'm not sure what the threshold is for MyMain to call a drive "Full" -- but over half of my drives are shown as "Full" ... and they actually have between 22M and 446M of free space on them (I'd agree they're "Full" -- but 446M is more than 17 times the size of the first hard drive I bought !!).  A drive with 19G of free space is shown as 100% used (but not "Full").

By the way, I'm not sure what the threshold is for MyMain to call a drive "Full" -- but over half of my drives are shown as "Full" ... and they actually have between 22M and 446M of free space on them (I'd agree they're "Full" -- but 446M is more than 17 times the size of the first hard drive I bought !!).  A drive with 19G of free space is shown as 100% used (but not "Full").

I've heard it suggested to not fill a drive beyond about 90%. Don't remember the reasoning. But I, like you, tend to fill my drives one at a time. I don't remember the threshold for full, but being under a gig, much less 500meg, is good enough for getting your money's worth in my book.