Drive rebalancing


Recommended Posts

I have my user shares set for "High-Water" with a minimum size of 15000000, but I have some drives with way less free space than others.

 

Device 	Identification                                             	Temp. 	Size   	Used 	Free 	Reads       	Writes      	Errors 	View
Parity	Hitachi_HDS5C3030ALA630_MJ1323YNG1SUPC (sdd) 2930266532 	99 °F	3 TB  	-   -       	7,154,775	6,107,378	0	
Disk 1	Hitachi_HDS5C3030ALA630_MJ1323YNG1UB8C (sde) 2930266532 	108 °F	3 TB  	2.94 TB	64.7 GB	10,241,322	581,597    	0	
Disk 2	Hitachi_HDS5C3030ALA630_MJ1323YNG1U3PC (sdb) 2930266532 	109 °F	3 TB  	2.95 TB	53.6 GB	2,670,633	987,541    	0	
Disk 3	Hitachi_HDS5C3030ALA630_MJ1323YNG1TLWC (sdc) 2930266532        	104 °F	3 TB  	2.99 TB	11.4 GB	3,106,694	1,081,903	0	
Disk 4	SAMSUNG_HD102UJ_S1ZUJDWS308586 (sdi) 976762552               	90 °F	1 TB  	912 GB	87.9 GB	3,504,580	1,082,739	0	
Disk 5	SAMSUNG_HD103UJ_S13PJDWS346374 (sdh) 976762552              	100 °F	1 TB  	921 GB	79.5 GB	2,701,855	264,421    	0	
Disk 6	SAMSUNG_HD102UJ_S1ZUJDWS308596 (sdj) 976762552               	104 °F	1 TB  	820 GB	181 GB	1,034,065	386,396   	0	
Disk 7	Maxtor_6L200P0_L40CF2NH (hda) 199148512                      	109 °F	204 GB	161 GB	43.0 GB	7369        	69            	0	
Disk 8	WL3000GSA6472C_WOL240243956 (sdf) 2930266532                   	95 °F	3 TB  	2.97 TB	33.8 GB	5,831,322	3183        	0	
Disk 9	WDC_WD20EARS-00MVWB0_WD-WCAZA2130725 (sdk) 1953514552   	91 °F	2 TB  	1.86 TB	138 GB	3,484,990	1,583,902	0	 

 

First, I'm not sure why some have so little free space compared to others of the same size.

Second, I'm going to have to replace a 1TB drive with a 3TB drive (or 2), and then I want to automatically re-balance the drives to they all have similar free space (so that none are over 90% filled).  I'm unaware of any way to automatically re-balance the drives, so I guess this is a suggestion/request.

 

Any ideas on why they are so unbalanced, or how to re-balance?

Link to comment

Why:  well the two things that I know impact that are split levels (too restrictive means everything goes to one drive vice spread around) and 2) allocations strategy (high-water vs. most free). 

 

As you can imagine most free will ensure your data is evenly spread amongst the drives available for a given share.  But first you also need to ensure the user share split level isn't set such that you are restricting the allowable disks.

 

How: 1) fix split level if too restrictive,  2) set allocation to most-free, 3) CHOOSE a) use Midnight Commander to bulk move data between drives b) keep using server as normal and watch as new data is placed onto the drives with the most space.

Link to comment

thanks for the fast response.

 

I had split level set to 0 for the video share, and it was blank for all the other shares.  I have since deleted the 0 from the video share also, so they are all now blank.

 

I have also changed from high-water to most-free for all shares.  I'm 90% sure I had most-free set long ago, but had a similar problem, so I changed to high-water.  the descriptions of the options aren't quite as clear as one might like :)

 

Hopefully this will let them balance out as I copy a few more movies, but it sounds like I'm going to have to manually re-balance once I get the new drive installed.

 

thanks again.

Link to comment

Not sure leaving split-level blank will do what you want.  Heck I'm not even sure what that equates to, 999 or 0.  You should read this http://lime-technology.com/wiki/index.php/Un-Official_UnRAID_Manual#Split_level and choose an appropriate split level.  You might even need to rethink your organization structure first and create new shares to accommodate the new plan.

 

As for allocation method, I do recall that for it to take effect (as well as new split levels) you need to reboot.

Link to comment

The potential problem with having no split level set in conjunction with the Most Free allocation option is that it can mean that related files are likely to get scattered across the disks.  As an example if you had a folder containing the files making up a DVD or a TV series then they would all tend to go to different disks.  If this does not matter to you then go ahead.

Link to comment

User shares are still a bit too confusing for me.  I have a deeper structure for photos than movies which is deeper than music, so there is no one setting that would work 'the same' for all my media, and I have no desire/ability to change the directory structure of movies and music to match that of photos, so there's not really one setting that is 'best'.

 

With that said, for movies, I pretty much keep everything as one mkv file, and a jpg and an xml for each movie, all under the Movie folder.  For example...

 

video  ->

              Movie  ->

                              300  ->

                                            300.jpg

                                            300.mkv

                                            300_mkv_JRSidecar.xml

 

I don't really care if those 3 files are kept together, but it would probably be better if they were.  However, with that said, what would you recommend for split level?

Link to comment

Based on your video share description I'd say set split level at 2.  The danger there, if music is more shallow, is that you might find yourself spinning up the whole array while listening to music.  Not great for power, and not great if your drive idle time is short enough such that one disk spins down while listening to a song on another disk and then you have to wait for the sleeping drive to be spun back up for the next track.

 

It isn't ideal but you can artificially make music "deeper" by just adding another layer like say "Audio" such that you end up with an individual Artist on only a single disc and not spread across the array.

 

  L1      L2

Video-Movie-[movie title]-*.mkv

Music-Artist-Album-*.mp3

 

  L1      L2

Video-Movie-[movie title]-*.mkv

Audio-Music-Artist-Album-*.mp3

 

I can't speak to your photo share as I have no idea how you have it set up and how you use it.  But actually that megs the question, when you are talking about depth with respect to split level, you realize they don't really equate?!  That is to say, it doesn't matter how deep the structure goes, what matters is where in that structure would want to stop allowing the system to sprinkle files all over your array.  For Movies, at the least you'd want to prevent at least an individual movie from spreading.  For music at leat an individual album, if not even the artist as I mention above.  For photos, well if you are feeding a live slideshow you'd sure want to avoid spreading the pics over too many drives such that the slideshow keeps all your drives spinning / constantly hits sleeping drives.  But otherwise, if it is just a photo archive then perhaps you don't care so much.

Link to comment

"Most free" will NOT "... ensure your data is evenly spread amongst the drives available."  What it will do is ensure the free space is uniform among all your drives -- but when all the drives aren't the same size the result is that the larger drives get all the writes until they have less free space than the smaller drives.

 

i.e. It does exactly what it says -- writes to the drive with the most free space.  Your drives are all fairly full, so what would happen if you change to "most free" is all the writes will go to the drive with the most free space (disk6) until its free space drops below the next drive (disk9);  then they'll toggle between those two until they drop below the free space on disk4;  then they'll toggle between those 3 drives until the space hits that of disk5;  etc.

 

It WILL, however, uniformly fill up the remaining space on your disks.

 

Note that if you do as you indicated and replace one of the smaller disks with a new 3TB disk, the "most free" allocation would mean ALL of the writes would go to that disk until it was about 94% full (i.e. until it had less space than disk6 -- currently 181GB).

 

By the way, there's NO reason to "balance" your drives.  With drives as close to full as yours are, I'd be tempted to change the allocation method to "fill-up" so UnRAID would simply go ahead and fill the remaining space on each of the drives (one-at-a-time).    You have to do this for each of the shares, by the way (as I assume you know).

 

The key thing your Web GUI pictures shows is NOT that you need to "rebalance" your drives => it's that you need to add some space to your array !!  Replace those three 1TB drives with 3TB drives ... or, better yet, replace the parity drive with a 4TB drive; then use the old parity drive to replace one of the 1TB drives;  and then replace the other two 1TB drives with 4TB drives.  Altogether, that will add 8TB of storage capacity to your array.

 

 

Link to comment

[nodding] ... I guess my perspective, narrow as it is, was coming from an array with all equal array drives so "most space" really does keep all my drives at about equal levels.  I suppose that is what high-water is really meant to deal with.

 

The one thing though I would quibble with Gary is that I don't think he should go beyond that 94% full on any of his drives.  Not from experience, but from enough threads on this forum, I understand there can be real speed penalties with pushing ReiserFS much further and even problems accessing data as the drive approaches 100%.

Link to comment

I don't think he should go beyond that 94% full on any of his drives.  Not from experience, but from enough threads on this forum, I understand there can be real speed penalties with pushing ReiserFS much further and even problems accessing data as the drive approaches 100%.

 

There's NO problem READING from drives that are VERY full => several of my drives are 99.9+% full (a couple are more than 99.99% full).  WRITES are indeed very slow for the last few GB => but if this is a media server, it's very likely that the data is essentially static.    [Actually, the writes themselves aren't slow ... but UnRAID "thinks" for quite a few seconds before actually starting the write process on a very full drive =>  I suspect there are some calculations being done to determine exactly where it can write the data]

 

For the last 50GB or so on each of my disks, I manually wrote to the disks themselves instead of to the shares.  My "fullest" disk is a 1.5TB drive with 21MB free ... that's almost 99.999% full  :)    And there is no problem copying data from it; streaming several movies at a time from it (I've tested up to 5-at-a-time and there are no glitches); etc.

 

Link to comment

thanks for all the great information.  I've pulled the trigger and purchased a new 4TB drive.  i plan to replace the parity with this new drive, then use the old parity to replace one of the smaller drives.  But, I'm not sure how to accomplish all this.  i have 6 SATA ports on the motherboard, and a 4 port card installed, and I have 10 drives connected currently, meaning I don't have room to install the new drive to pre-clear it, unless I disconnect one of the other drives.  I suppose I could disconnect the cache drive, and use that space for the pre-clear, but is there another/better way?

 

Then, once I have pre-cleared the new drive, then swapped the parity, how/were will I pre-clear the old parity to get it ready to use in the array?  is that even necessary to do (pre-clear) with the old parity drive?

 

thanks again for all the help!

Link to comment

Do you have another system you could boot the free version of UnRAID on and do your pre-clears?

 

If not, you could do the following:

 

Before you begin, run a parity check to confirm all is well -- if you have any sync errors, fix them and run another parity check => you do NOT want to make these changes until you know your array is good, since you'll be running "at risk" while the new parity sync is done on the 4TB drive.

 

Then Stop the array and shut down.

 

Replace the parity drive with the new 4TB drive -- but do NOT boot.

 

Prepare a DIFFERENT flash drive with the free version of UnRAID, and include a copy of the pre-clear script on it.

 

Boot to the new flash drive -- but do NOT assign any drives to it.    Just run the pre-clear script on your new 4TB drive (be CERTAIN you've selected the correct drive !!).    This will take a LONG time (likely close to 2 days just for one pass)

 

Assuming the pre-clear completes successfully, shut down;  put the "real" (licensed) flash drive back on the system, and boot to that.    It will show a missing parity drive -- Stop the array; assign the new drive as parity; and Start the array back up and let it do a parity sync.    When it's done, do a parity check to confirm all went well.

 

Now shut down; replace one of your 1TB drives with the old parity drive;  then reboot.  The array will show a "missing" drive -- Stop it and assign the 3TB (old parity) drive in its place ... then Start the array again and it will rebuild the drive on to the 3TB unit.

 

 

attach the new 4TB to one of the SATA ports;  boot a NEW flash drive prepared with the free version of UnRAID (with the pre-clear script on it);  and then do the pre-clear.  Obviously do NOT assign any drives to that version;  and be CERTAIN that you're pre-clearing the correct drive.

 

Then shut down; and boot to the correct flash drive; Stop the array;  Assign your new 4TB as parity;

Link to comment
  • 2 weeks later...

Do you have another system you could boot the free version of UnRAID on and do your pre-clears?

 

If not, you could do the following:

 

Before you begin, run a parity check to confirm all is well -- if you have any sync errors, fix them and run another parity check => you do NOT want to make these changes until you know your array is good, since you'll be running "at risk" while the new parity sync is done on the 4TB drive.

 

Then Stop the array and shut down.

 

Replace the parity drive with the new 4TB drive -- but do NOT boot.

 

Prepare a DIFFERENT flash drive with the free version of UnRAID, and include a copy of the pre-clear script on it.

 

Boot to the new flash drive -- but do NOT assign any drives to it.    Just run the pre-clear script on your new 4TB drive (be CERTAIN you've selected the correct drive !!).    This will take a LONG time (likely close to 2 days just for one pass)

 

Assuming the pre-clear completes successfully, shut down;  put the "real" (licensed) flash drive back on the system, and boot to that.    It will show a missing parity drive -- Stop the array; assign the new drive as parity; and Start the array back up and let it do a parity sync.    When it's done, do a parity check to confirm all went well.

 

Now shut down; replace one of your 1TB drives with the old parity drive;  then reboot.  The array will show a "missing" drive -- Stop it and assign the 3TB (old parity) drive in its place ... then Start the array again and it will rebuild the drive on to the 3TB unit.

 

 

attach the new 4TB to one of the SATA ports;  boot a NEW flash drive prepared with the free version of UnRAID (with the pre-clear script on it);  and then do the pre-clear.  Obviously do NOT assign any drives to that version;  and be CERTAIN that you're pre-clearing the correct drive.

 

Then shut down; and boot to the correct flash drive; Stop the array;  Assign your new 4TB as parity;

 

This is basically what I did, but slightly different.

 

I put the new 4TB drive into a second computer, started a fresh/free version of unRAID and precleared the drive with that machine, leaving my array in tact the whole time.

 

Then, I swapped my 3TB parity disk for the newly precleared 4TB disk, and that is currently re-snycing parity.  So far, so good.

 

I have the old parity sitting on my desk now, and am wondering if I need to/should preclear that disk in the second machine, while waiting for the sync to finish, or if, as it sounds from the above, I can just swap a 1TB drive for the old 3TB Parity drive, as it is, without deleting/preclearing/formatting the old Parity drive.  It seems like it needs to be precleared, but that will take an extra day vs just waiting for the sync to finish.

 

Thoughts?

Link to comment

I have the old parity sitting on my desk now, and am wondering if I need to/should preclear that disk in the second machine, while waiting for the sync to finish, or if, as it sounds from the above, I can just swap a 1TB drive for the old 3TB Parity drive, as it is, without deleting/preclearing/formatting the old Parity drive.  It seems like it needs to be precleared, but that will take an extra day vs just waiting for the sync to finish.

 

Thoughts?

 

Preclear it.

 

I've never added an old parity drive back to a server but I have added old data drives. Unraid always wanted them to be zero'd then reformatted. That means the server is offline while it does so.

 

I preclear the old data drives on a different machine first then add them back. Unraid will still reformat it but that only takes a few mins. It doesn't need to be zero'd since it's precleared which means your server is only offline for however long it takes to put the drive back in the box!.

 

Link to comment

I strongly considered that, but I'm moving out of the country in less than a week, and I'm not sure I'll have time to finish the parity sync, then check, then preclear a 3TB drive, then one more drive, then rebuild the old 1TB drive onto the new 3TB drive; all in time to get it all safely packed before I need to be out of here, so I took the (hopefully very) low risk.

 

I've never had any issues with the array, or any of the disks, so I'm just gonna keep my fingers crossed, and hope for the best!!! :)

Link to comment

I cancelled the pre-clear on the old 3TB parity drive.  it was about 80% finished pre-reading.  i tried running preclear_disk.sh /dev/sdc -n, but got a long screen that I can only see the end of, but it looks like a list of valid parameters and their descriptions, and i can't find any way of scrolling up that list to see anything but about the last parameter, so it does me no good.

 

so, so far, one person says I do need to pre-clear, one person seems to say that I don't, and one person says that I should pre-clear with the -n parameter, but I don't really know how to do that.

 

At this point, nothing is happening, and I'd like to get this drive ready for the array when the parity sync (then parity check) finishes.

 

I intend to put this 3TB drive into the place where a 1TB drive currently resides, and that 1TB drive does have data I want to keep, so I want to make sure that replacing it with this 3TB drive will result in that 1TB of data being rebuilt onto the new (old Parity) 3TB drive as quickly as possible (NOT have the array be unavailable while the 3TB is cleared).

 

advice?

 

Also, I have one more 2TB external drive that I eventually want to pre-clear and replace one more 1TB drive with, once this is all done.  I don't believe I can replace 2 drives with larger drives at one time.  I have to replace one, wait for the date to rebuild, (then check parity???), then replace the next 1TB drive; correct?

 

thanks again!

Link to comment

I figured out the -n option, and am preclearing the 3TB (old parity) drive with this option.  it should finish about the same time as the parity sync/parity check, so I'm hoping that will be good enough.

 

Once this drive swapping is all done, I'll be back to wishing/hoping for a way to 'automatically' rebalance all the drives, to give me 20% or more free space on all drives.  i guess I'll just need to copy/paste to do it manually, if I decide it's worth having them more 'balanced'.

 

I know garycase said he's had no problems reading up to 5 movies simultaneously from a VERY full drive...

 

I don't think he should go beyond that 94% full on any of his drives.  Not from experience, but from enough threads on this forum, I understand there can be real speed penalties with pushing ReiserFS much further and even problems accessing data as the drive approaches 100%.

 

There's NO problem READING from drives that are VERY full => several of my drives are 99.9+% full (a couple are more than 99.99% full).  WRITES are indeed very slow for the last few GB => but if this is a media server, it's very likely that the data is essentially static.    [Actually, the writes themselves aren't slow ... but UnRAID "thinks" for quite a few seconds before actually starting the write process on a very full drive =>  I suspect there are some calculations being done to determine exactly where it can write the data]

 

For the last 50GB or so on each of my disks, I manually wrote to the disks themselves instead of to the shares.  My "fullest" disk is a 1.5TB drive with 21MB free ... that's almost 99.999% full  :)    And there is no problem copying data from it; streaming several movies at a time from it (I've tested up to 5-at-a-time and there are no glitches); etc.

 

 

...but I've been getting glitches, and LONG pauses in playback of only one movie in the last few weeks, so I suspect either I've got something in my network jacked up, or having full, different size drives is causing some issues.  i do use a 500 GB cache drive, so I would expect any writing and/or unpacking of movies to NOT affect my playback, but something's going wrong :(

 

I'll report back in a couple of days on my (hopeful) success, or sooner with questions if necessary.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.