Setting up Your File Structure and User Shares on unRAID®


Recommended Posts

An additional comment (which you are free to ignore!):

 

Some time ago, someone commented that it seemed surprising that so many of the oldest veterans, including many moderators, were NOT using User Shares.  I have to include myself, as one who has never been interested in turning them on, apart from possibly better familiarizing myself with them for the sake of helping others.  I can't speak for others, but I think they are a great UnRAID feature, just not for me.  I like more control over my data, knowing exactly where everything is.  I place my various categories of data on specific disks, and as yet haven't found further data abstraction[1] to be helpful.  I keep a separate index to all of my files, 'Bones' is all on Disk 2, 'Masterpiece' and similar PBS is on Disk 4, movies and history-related are on Disk 8, backup copies of photos and music and other collections are on Disk 10 with an even older backup on Disk 1, etc etc.  In addition to preferring complete control, I don't like adding additional and unnecessary software layers (such as Fuse), that may add additional bugs and vulnerabilities and performance hits.

 

I'm in no way wanting to knock User Shares, or even suggesting that you need to accommodate users like me (and others).  But if you think it useful, you might consider adding a comment or two that User Shares aren't the only way, that some prefer to store directly.

 

1. I'm not against all data abstractions.  I would hate to go back to direct sector addressing, and messing with drive geometries!

Link to comment

An additional comment (which you are free to ignore!):

 

Some time ago, someone commented that it seemed surprising that so many of the oldest veterans, including many moderators, were NOT using User Shares.  I have to include myself, as one who has never been interested in turning them on, apart from possibly better familiarizing myself with them for the sake of helping others.  I can't speak for others, but I think they are a great UnRAID feature, just not for me.  I like more control over my data, knowing exactly where everything is.  I place my various categories of data on specific disks, and as yet haven't found further data abstraction[1] to be helpful.  I keep a separate index to all of my files, 'Bones' is all on Disk 2, 'Masterpiece' and similar PBS is on Disk 4, movies and history-related are on Disk 8, backup copies of photos and music and other collections are on Disk 10 with an even older backup on Disk 1, etc etc.  In addition to preferring complete control, I don't like adding additional and unnecessary software layers (such as Fuse), that may add additional bugs and vulnerabilities and performance hits.

 

I'm in no way wanting to knock User Shares, or even suggesting that you need to accommodate users like me (and others).  But if you think it useful, you might consider adding a comment or two that User Shares aren't the only way, that some prefer to store directly.

 

1. I'm not against all data abstractions.  I would hate to go back to direct sector addressing, and messing with drive geometries!

Good point RobJ.  I added a note at the beginning.

 

Thanks,

TomH

Link to comment
  • 4 months later...

Quick question. I've read through your post a few times and through the forum, but in actual use i seem to be totally misunderstanding something.

 

I have 2 unraid servers and am trying to move content from one to the other. I want to have the following structure with user shares on the new unraid. New unraid is currently 3 3TB drives, and all user shares set to high water.

 

Movies->Movie_Name->Movie Files  (with all the movie files remaining on the same drive and the Movie_Name folders residing on any drive)

With this structure and the blog post I was using split level 1, but after starting the copy disk1 when well beyond 50% usage and didn't start placing on disk2 until I changed split level to 2. It then copied new movies to disk2, switched split level back to 1 and it again started copying files only to disk 1. What should the setting be and how am I getting this so wrong?

 

My next share will be:

TV_Shows->Show_Title->Season_#->Season Files (again I want to split on Season_# (keeping all files for season together), so I thought this would be split level 2. Is this correct?

 

Is there a tool/website that will determine the proper split level based on custom inputs? Could there be?

 

Thanks!

Link to comment

Quick question. I've read through your post a few times and through the forum, but in actual use i seem to be totally misunderstanding something.

 

I have 2 unraid servers and am trying to move content from one to the other. I want to have the following structure with user shares on the new unraid. New unraid is currently 3 3TB drives, and all user shares set to high water.

 

Movies->Movie_Name->Movie Files  (with all the movie files remaining on the same drive and the Movie_Name folders residing on any drive)

With this structure and the blog post I was using split level 1, but after starting the copy disk1 when well beyond 50% usage and didn't start placing on disk2 until I changed split level to 2. It then copied new movies to disk2, switched split level back to 1 and it again started copying files only to disk 1. What should the setting be and how am I getting this so wrong?

 

My next share will be:

TV_Shows->Show_Title->Season_#->Season Files (again I want to split on Season_# (keeping all files for season together), so I thought this would be split level 2. Is this correct?

 

Is there a tool/website that will determine the proper split level based on custom inputs? Could there be?

 

Thanks!

 

From what you shared above, split-level 1 is correct.  With the share name being 'movies' and the next levels being 'movie_name' and 'movie files', this will ensure that the movie files stay on the same disk as the movie name. 

 

A couple of questions for you:

1. How big is each movie directory?

2. During the copy process, when did you decide it was doing the wrong thing?  (i.e., can you quantify "well beyond 50%"?)

 

If your first disk was filled to 1.449TB, it would have copied the next file before switching to the next disk.  In other words, the high-water mark (in this case 1.5TB) doesn't trigger until it adds a file that goes above it.  That next movie directory file could have been a large one (e.g. 20GB).

 

If you don't think this is what happened, please let me know and we can look into this further.

 

On your TV show hierarchy, split-level 2 is correct.

 

Thanks,

TomH

Link to comment
  • 3 months later...
  • 2 weeks later...
  • 3 months later...

Nice post and comments. I would like to share my opinion too. I my case, I come here trying to escape from several USB HDDs that I was using to store movies, shows, ISO files and backups. When I read the first time about shares, I thought it was a dream made true. And it is! I have been some time thinking about different approaches before deciding unRAID was my way to go. And the biggest reasons where two:

 

- Simplicity

- Long time support: unRAID has been around several years already, so it's quite mature, and the future looks promising too.

 

And talking a bit more about simplicity, I decided to think about my unRAID server as "one really big network disk". Only for Time Machine backups, because it's recommended, it's stored on a single disk, but all the other stuff, is spread across all the disks and I don't even care or want to know where they are, because I'm thinking of it as a whole, as one single really big disk. I disabled the indivual disks network access  and I only see the shares. There's some risk? Of course, but that's why we have a parity disk and there's also a risk being more precise about each single disk storage, so for me it doesn't worth the effort. It would also make it much more difficult for my wife to use it. Now it's really simple to understand: My shares are called: Media, Backups, ISOs, Software, Work. And two more private shares: one with my name, and another with my wife name, so each can store there personal stuff as each of us want. That's it. For me, this is the way to go, and the most comfortable way to use it. And I'm really, really, really happy with my setup :).

Link to comment

... I decided to think about my unRAID server as "one really big network disk".

 

That's indeed the right approach.  As a NAS, UnRAID provides two major features:  (1) the unified view that User Shares provides, so the entire storage pool "looks" like a single disk; and (2) fault tolerance.

 

Don't forget, however, that fault-tolerance does NOT mean you don't need to backup your important data.  You might want to read through this thread:  http://lime-technology.com/forum/index.php?topic=31020.0

 

 

 

Link to comment

... I decided to think about my unRAID server as "one really big network disk".

 

That's indeed the right approach.  As a NAS, UnRAID provides two major features:  (1) the unified view that User Shares provides, so the entire storage pool "looks" like a single disk; and (2) fault tolerance.

 

Don't forget, however, that fault-tolerance does NOT mean you don't need to backup your important data.  You might want to read through this thread:  http://lime-technology.com/forum/index.php?topic=31020.0

 

Thanks, because after reading this thread I though I was doing something weird, because they said that some long time users where sharing individual disks to have more control about storage and I don't really see the point of it. I bought unRAID and built my server specifically trying to avoid that ;).

 

About the data, yes, I know, but thank you! I store things that I can retrieve again from the Internet, backups, and in the Work section, I store old things that I know are not important, but I keep just in case, but is not a drama if they disappear, or backups from my websites. There's nothing stored ONLY on my server that is really critical to me ;). For those things I have a hosting provider where I have Owncloud, and files are stored on my Mac, on the unRAID server Backup share, and on the hosting provider server (3 copies, one far away from home).

Link to comment

Thanks, because after reading this thread I though I was doing something weird, because they said that some long time users where sharing individual disks to have more control about storage and I don't really see the point of it. I bought unRAID and built my server specifically trying to avoid that ;).

 

Some folks think it's "more organized" to force certain material to be on specific disks.    But the beauty of UnRAID is you don't have to do that -- just organize it by shares, and forget about the underlying disk structure.    I consider my UnRAID servers "BIG disks" ... in my case 15TB, 24TB, and 40TB.    There IS one specific case where you need to use the disk shares instead of the user shares ==>  If you decided, for some reason, that you wanted to move files from one disk to another (not likely in your case), it's important that you do this via the direct disk references instead of the user shares, because of a "user share copy bug" -- this can result in lost data.    I won't go into more detail, since it's not something you'll likely need to worry about ... just remember NOT to try to move data between disks by copying to the share name.

 

 

... 3 copies, one far away from home ...

 

My kinda backup guy  :)  [EVERYTHING I have is stored in at least 3 places, in some cases as many as 5]

 

Link to comment

Take TV shows for example. If you keep each series on a single disk and you do suffer an unrecoverable loss of a disk then you know what you have lost. Every whole series that is missing. If the episodes are spread all over the place then you have to go series by series and replace what is lost. The beauty of unRAID is that you just create the shares properly and then it will automatically keep the series together while you treat each share as one big network drive.

 

So, saying that it's harder to keep things organized for that "just in case" possibility is rather short sited.

Link to comment

It's not "harder" -- it's just not necessary.    If you lose a disk, you should, of course, immediately replace it and let UnRAID do a rebuild.  If you lose multiple disks (and thus actually lose a disk or more worth of data), then just replace the failed disks and restore the data from your backups.    If you use a sync utility to restore your backups (I use SyncBack) it's trivial to only restore what's missing.    I could, for example, run a backup profile in Restore mode to restore my entire "TV Series" folder from my backup server, and NOTHING would be copied unless it was missing, but any missing episodes would be copied no matter which disk they had been on.

 

Treating the entire array as a single large disk also eliminates issues like having a full disk (i.e. if you keep specific series all on one single disk, and try to copy more episodes and run out of room);  and also makes it trivial to make your "disk" larger ... by simply adding another drive.

 

One thing I DO recommend is to use an appropriate split level so if you have media that isn't stored in a unified container (e.g. DVDs in their original .VOB format) the individual parts aren't on different disks ... which can cause "freezing" during playback while another disk spins up.

 

Link to comment

I'm not dismissing your choice to manually structure your file distribution -- I'm simply noting that one of the nice features of UnRAID is that it doesn't require you to do that.  If you choose to do so, that's fine.    Doing it like that can indeed make it simpler to restore the data, especially since you don't maintain backups and would have to re-rip everything.    But even then, you have to know what was on the disk to know what you need to re-rip ... so to be prepared for that you need to maintain directories of the disks.    Given that you have those directories, it would seem you could easily re-rip what you needed regardless of which titles were on the disk  [it is, I'll grant, probably simpler to grab a bunch of disks for the same series than to find a more random collection of titles].

 

Whether original disks are "backups" is debatable ... I certainly don't consider mine backups of my server.  The process of ripping, encoding, cataloging, re-rendering, etc. represents a LOT of hours that I would definitely not want to repeat for my 4,000+ DVDs (not to mention the thousands of TV episodes I've recorded, cataloged, etc.).    The cost of my backup server is easily worth it as insurance against ever having to redo that work !!

 

Link to comment

One size doesn't fit all and dismissing good reasons other people have for doing something because you don't require it is rather sad.

 

I neither wanted to dismiss other users reasons to do whatever they want!! I was just curious about what reasons they had to do it, having a easier feature to share things. Just that! ;)

 

The process of ripping, encoding, cataloging, re-rendering, etc. represents a LOT of hours that I would definitely not want to repeat for my 4,000+ DVDs (not to mention the thousands of TV episodes I've recorded, cataloged, etc.).    The cost of my backup server is easily worth it as insurance against ever having to redo that work !!

 

WOW!! That's a lot of DVDs!! :)

Link to comment
  • 2 years later...

I understand the advantage/difference in User Shares vs Disk Shares.  I'm not sure that I understand the advantage of one large User Share vs multiple User Shares.  Since I'm setting up Unraid as a Media Server, I will have Movies, TV, Books, Audiobooks, Music, Videos (Personal), Photos (Personal), Graphics, etc.  Is there a downside to creating each of these as individual User Shares as opposed to having them all under one large Media User Share?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.