Newbie questions (mainly filesystem choice)

[karateo]

Hi! My first thread and second post!

 

Finally I got the opportunity to build my home server. I have tried a Synology but didn't like the price/performance and some software limitations.

Tried Proxmox with OMV but didn't really need all that options and the complexity.

Now I am using it as a remote backup at parents home using rsync.

 

Now I am using trial UnRaid with only two disks to understand how things work and when I feel comfortable I will make the switch.

 

My hardware is

• Xeon E3-1230v3
  
• 4Gb ECC - going to upgrade in the future
  
• 2x 1TB WD Green
  
• 2x 500Gb WD Blue (one is 2.5" from an old laptop
  

 

My usage

• File server
  
• Plex/Serviio transcoding to 1 TV
  
• Rsync to remote Synology (or any other way to remote backup (for example BTSync)
  
• Backup storage for NVR
  
• Torrents
  
• Windows OS for some apps (Autocad, Video editing/encoding...)
  
• Nice portal for sharing photos/video with family (for example owncloud)
  
• maybe more but still I don't need high disk performance
  

 

My budget for the next 2-3 months is pretty limited to a licence or one 2-3TB drive.

 

What are my thoughts/questions.

1) XFS or BTRFS

I have seen corrupted images and private photos/videos is super valuable to me. I have a remote backup, plus an external drive with outdated copy.

But I would like to keep my source as secure as possible. One drive parity is more than enough. I can use my remote backup in the case of 2-drive failure.

I feel that bit rots are the biggest threat to my files.

 

From https://btrfs.wiki.kernel.org/index.php/Main_Page

Stability status

The filesystem disk format is no longer unstable, and it's not expected to change unless there are strong reasons to do so.

So I assume no longer unstable = almost stable!

 

Are there enough tools in UnRaid to manage a btrfs system? I see btrfs scrub is in the webgui. Snapshots are not. So I will need to use terminal for that.

 

How does btrfs correct errors? Does it use snapshots?

Also, I wonder how this page http://www.snapraid.it/compare.html will look like if we add the UnRaid+BTRFS

 

 

---

 

2) Just to clarify some things about licencing.

I have read some pages from this

https://lime-technology.com/forum/index.php?topic=38192.0

and

http://lime-technology.com/registrationkeys/

 

Other than unRAID Server Basic, registration keys work on all available versions of unRAID Server OS.

i)  Are the keys valid for future updates?

If not, UnRaid 6 roughly for how many years will be supported?

 

ii) Also, according to my current needs the basic licence is more than enough. Right?

 

---

 

One week ago I didn't know anything about zfs, xfs, btrfs, proxmox, omv, unraid, snapraid etc.

So please forgive any mistakes I make!

[CHBMB]

I think most of us use XFS rather than BTRFS probably because at time of release XFS was more stable.  Maybe someone else can offer some useful opinions on the current state of play.

 

Historically your license key has been valid for major upgrades.  Whether that remains is anybodys guess, although I suspect V7 is a long long way away...

 

Yes a basic license would be more than enough for you currently.

[karateo]

Ok about the licence.

Now about the filesystem.

 

 

First of all I did a small test and got confused!

I created an array with diskparity as  parity and disk1 as array disk 1.

Then run NEW CONFIG and readded disk1 as array disk1

and diskparity as arraydisk2

Started the array and my test files where present in both disks. Like RAID1.

I thought parity drive had no usable files!

 

<hr>

 

There is so little feedback or documentation about Unraid+btrfs.

I have been reading in https://btrfs.wiki.kernel.org a lot today.

 

I need to clear in my mind the use of Unraid AND btrfs simultaneously!

Unraid on it's own is a type or raid.

For example, for 3 disks (2 array 1 parity)

A file is written in disk1 and some bits (or a copy of the file, see above) in parity

Another file is written in disk2 and some bits in parity

 

Btrfs in unraid is used in every disk on its own. No subvolumes or btrfs raid https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices

 

 

So if I create a share and include more than one disk, I can't make use of snapshots.

Or I should snapshot the same share folder in all disks that it exists.

 

Am I in the right track?

 

So, in order to benefit from some of the btrfs features, I should choose a share-folder to be located only on one disk.

Or maybe create manually subvolumes, but I need to think about it more!

 

So much new information to process! Looking for some feedback/comments. Anything will be helpful!

[itimpi]

First of all I did a small test and got confused!

I created an array with diskparity as  parity and disk1 as array disk 1.

Then run NEW CONFIG and readded disk1 as array disk1

and diskparity as arraydisk2

Started the array and my test files where present in both disks. Like RAID1.

I thought parity drive had no usable files!

It does not in the sense you are thinking off.  It is just that with the special case of 1 data disk and 1 parity disk they will be identical at the individual sector level as a by-product of the way parity is calculated.  As soon as a second data disk is added this will no longer be true.

[CHBMB]

First of all I did a small test and got confused!

I created an array with diskparity as  parity and disk1 as array disk 1.

Then run NEW CONFIG and readded disk1 as array disk1

and diskparity as arraydisk2

Started the array and my test files where present in both disks. Like RAID1.

I thought parity drive had no usable files!

It does not in the sense you are thinking off.  It is just that with the special case of 1 data disk and 1 parity disk they will be identical at the individual sector level as a by-product of the way parity is calculated.  As soon as a second data disk is added this will no longer be true.

That makes sense, but I'd never have made that leap of logic myself...

[karateo]

It does not in the sense you are thinking off.  It is just that with the special case of 1 data disk and 1 parity disk they will be identical at the individual sector level as a by-product of the way parity is calculated.  As soon as a second data disk is added this will no longer be true.

 

Ok then! Only have two spare drives to make tests so I got into that special case!

 

So to get started, using btrfs at least provides protection against bit rots which was my main concern.

So, from this table

http://www.snapraid.it/compare.html

using unraid+btrfs we get all the advantages from both columns!

[lionelhutz]

You do not get data corruption repair ability when using BTRFS as a single on a stand-alone filesystem on a single drive. Scrub will just tell you there are errors but it can't fix them.

 

I tried BTRFS on a single cache disk and the filesystem kept having corruption issues and I'm not the only one who has has that happen. I would test it heavily as the file system on the data disks before relying on it.

[karateo]

You do not get data corruption repair ability when using BTRFS as a single on a stand-alone filesystem on a single drive. Scrub will just tell you there are errors but it can't fix them.

 

I tried BTRFS on a single cache disk and the filesystem kept having corruption issues and I'm not the only one who has has that happen. I would test it heavily as the file system on the data disks before relying on it.

 

This is what I was looking for.

So when does BTRFS can repair errors?

I was searching yesterday the btrfs wiki but I didn't find anything.

 

[karateo]

Getting ideas from the other topic

https://lime-technology.com/forum/index.php?topic=44400

 

Just played around and created btrfs raid1 in unraid using terminal. I also mounted the array and could create Shared folder. Doesn't look or work perfectly but it's possible (why shouldn't right?).

 

So, I will format my drives with xfs for now to be super stable. I will also use par2 and when the time comes with a future update, I will change to btrfs.

 

I could try btrfs but the are no advantages at the moment I think. Right?

 

 

[bonienl]

For what it is worth...

 

I have set up all my data disks using XFS and the cache pool using BTFRS. My system is super stable and I have yet to encounter any bit rot.

 

This last one I verify on a monthly basis using a SHA2 comparison (bunker utility) and have been doing this for the last year.

 

[Noob]

Hey there!

 

I found your link over in my other btrfs thread and thought I would come here to clarify your question. If you want to know A LOT about btrfs in unRAID, you should read that whole thread, but it is kind of long.

 

btrfs (at the file system level) is designed to allow for detection of silent data corruption (bit-rot) on any disk. If you run a scrub command on a single btrfs disk it will show you that your file has changed and no longer matches the checksum. It will not be able to repair the corruption, though, because it doesn't have a good copy of the file to rebuild from.

 

btrfs in any RAID configuration (1,5,6,etc) will have some data redundancy available to it, so when you run a scrub and corruption is detected on a particular file btrfs will look to the second copy of the file and rebuild your corrupted data. It is able to know which one is the "good" copy and which is the "bad" copy because one still matches its original checksum.

 

In unRAID, since the goal is to toss out traditional RAID arrangements, each disk that is using btrfs is considered a standalone disk. unRAID does provide parity to allow a failed disk to rebuild, but it does this using XOR data on a separate parity disk, rather than using the btrfs RAID functionality. Because of this, btrfs is not able to rebuild your corrupted file (because it doesn't realize that there is a second copy on the XOR parity drive).

 

So, for the moment, unRAID is very limited in this respect. I'm still using the trial myself because I have to have bit-rot protection. unRAID gives me bit-rot detection if I select btrfs as my file system, but not being able to fix that bit-rot is a deal breaker for me.

 

Hope that helps!

[karateo]

I read all your topic from the moment I found it.

As I understand it, we can't have everything.

 

Raid with one drive failure protection

+

Being able to read data when >1 drives fail

+

Bit rot protection (which need some kind of redundancy)

+

Least storage "wasted" for redundancy

 

So, I will format my drives with xfs for now to be super stable. I will also use par2 and when the time comes with a future update, I will change to btrfs.

 

This way, I will have bitrot protection for the super important staff plus the advantages of UnRaid!

 

Now I need to read about par2 and how much redundancy do I need to select!

[Noob]

Yup, that will give you the features that you want for the time being.

 

I've decided against that route because Par2 leaves little files all over the place. They are necessary in order to be able to rebuild, but they really clutter up my file/folder structure. You can hide the Par2 files in your browser/file explorer, but just knowing they are there really bugs me.

 

I'm going to wait for some kind of btrfs implementation myself.

[karateo]

I have been preclearing disks to create the array and I got the following.

 

What do you suggest to do with this drive?

Parity? Array? Throw away (but I am running low on budget for new disk at the moment)?

Maybe more preclear cycles?

 

Disk Temperature: 32C, Elapsed Time:  15:14:40

========================================================================1.15

== WDCWD10EADS-00M2B0  WD-WCAV

== Disk /dev/sdc has been successfully precleared

== with a starting sector of 64

============================================================================

** Changed attributes in files: /tmp/smart_start_sdc  /tmp/smart_finish_sdc

                ATTRIBUTE  NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS      RAW_VALUE

      Temperature_Celsius =  115    119            0        ok          32

No SMART attributes are FAILING_NOW

 

18 sectors were pending re-allocation before the start of the preclear.

35 sectors were pending re-allocation after pre-read in cycle 1 of 1.

0 sectors were pending re-allocation after zero of disk in cycle 1 of 1.

0 sectors are pending re-allocation at the end of the preclear,

    a change of -18 in the number of sectors pending re-allocation.

0 sectors had been re-allocated before the start of the preclear.

0 sectors are re-allocated at the end of the preclear,

    the number of sectors re-allocated did not change.

[itimpi]

Since the values for reallocated sectors and pending sectors both went to 0 at the end of the run the drive is probably fine for any type of use.

 

I would suggest putting it through an additional pre-clear cycle to check they stay zero, and also post the full set of SMART attributes at the end.

[karateo]

Since the values for reallocated sectors and pending sectors both went to 0 at the end of the run the drive is probably fine for any type of use.

 

I would suggest putting it through an additional pre-clear cycle to check they stay zero, and also post the full set of SMART attributes at the end.

 

Thanks!

I will need to wait another 15 hours to start my array

 

 

[karateo]

I think that I am a little bit unlucky today...

 

After 31days with no power loss (i don't have a ups yet)

When it was 99% and it was about 1 minute to finish after 15hours....

 

A short power cut happened!!!

 

I already added the drive as parity and running parity check. I see no pending sector allocates in smart so the preclear didn't find any errors I think.

I tried to find logs but as it didn't complete it didn't write the report.

 

[karateo]

A weird problem.

New install with a few dockers and a few plugins (temperature, system stats which I uninstalled), community applications, dynamix active streams).

 

See pictures of problem. Webgui becomes unstable and eventually nothing works. Tried diagnostics command from putty but got the following.

Powerdown command didn't do anything.

 

Also, it's the second time it happens.

Ony thing I found is that the first time it got unstable, I uninstalled system stats and i got a minutes of stable webgui. But after a while the problem appeared again.

 

https://www.dropbox.com/sh/fh87sin35j1t6nc/AADtqd5LzZVK_gDmNPEtS4jha?dl=0

 

# diagnostics

Starting diagnostics collection...

Warning: file_put_contents(): Only 0 of 15 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 36

 

Warning: file_put_contents(): Only 0 of 12 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 36

 

Warning: syntax error, unexpected $end, expecting TC_DOLLAR_CURLY or TC_QUOTED_STRING or '"' in /var/local/emhttp/disks.ini on line 628

in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 36

 

Warning: file_put_contents(): Only 0 of 5429 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 36

 

Warning: file_put_contents(): Only 0 of 4125 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 36

 

Warning: file_put_contents(): Only 0 of 3269 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 36

 

Warning: file_put_contents(): Only 0 of 14 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 36

 

Warning: file_put_contents(): Only 0 of 3801 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 36

 

Warning: file_put_contents(): Only 0 of 2 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 50

 

Warning: file_put_contents(): Only 0 of 34 bytes written, possibly out of free disk space in /usr/local/emhttp/plugins/dynamix/scripts/diagnostics on line 52

done.

ZIP file '/boot/logs/tower-diagnostics-20151228-1032.zip' created.

 

After I stopped array this is my disk space.

 

root@TOWER:/boot/logs# df -h

df: ‘/mnt/disk1’: No such file or directory

df: ‘/mnt/disk2’: No such file or directory

df: ‘/mnt/user’: No such file or directory

Filesystem      Size  Used Avail Use% Mounted on

tmpfs          128M  2.6M  126M  3% /var/log

/dev/sda1      7.5G  107M  7.4G  2% /boot

/dev/loop0      1.8G  1.8G    0 100% /var/lib/docker

 

After a restart I found the diagnostics zip and I attached it.

tower-diagnostics-20151228-1032.zip