btrfs cache pool


limetech

Recommended Posts

This feature expands on the cache disk feature.  You can define a set of devices which will be formatted as btrfs cache pool using "btrfs raid1" for both data and metadata.  Note that "btrfs raid1" is a bit different than traditional "raid1" in that:

 

- you can have an odd number of devices in the pool

- even if more than 2 devices, each piece of data will exist on only two devices refer to https://wiki.archlinux.org/index.php/Btrfs#RAID_features

Link to comment
  • 4 weeks later...
  • 4 weeks later...
  • 3 weeks later...

Can we have the option of simply pooling drives together as one mount point? (i.e. /mnt/cache) so effectively.. RAID0 - No redudancy whatsoever, just pooling of drives.

That's a possibility for a future release, along with also the other raid levels btrfs supports.

 

Just wanted to reiterate support for RAID 0 support for cache pools.  I have a number of older/smaller SSDs that I would love to repurpose for my unRAID cache pool.

Link to comment
  • 1 month later...

I posted this in the Support forum and never got a reply.  I figured maybe this is a better forum to discuss since it is a roadmap item.

 

v6b12

 

I have 3x 128GB SSDs in my BTRFS cache pool.  I have only 3 slots assigned for the 3 disks.

 

4J0oAsn.png

 

Why is my total space on 256GB and what is going on with Cache 2 (see below):

 

Label: none  uuid: 313257bb-cada-4750-9df3-4ac8b895270d
Total devices 3 FS bytes used 93.46GiB
devid    1 size 119.24GiB used 85.02GiB path /dev/sdb1
devid    2 size 3.97MiB used 0.00B path /dev/sdc1
devid    3 size 119.24GiB used 85.00GiB path /dev/sdd1

Btrfs v3.17.2

Link to comment
  • 2 months later...

Can we have the option of simply pooling drives together as one mount point? (i.e. /mnt/cache) so effectively.. RAID0 - No redudancy whatsoever, just pooling of drives.

I would like to upvote this also.

 

I am in the process of building myself a new server based on updated hardware and am very much wanting to to a RAID0 of 2 240GB SSD drives for my cache drive.

Link to comment

Can we have the option of simply pooling drives together as one mount point? (i.e. /mnt/cache) so effectively.. RAID0 - No redudancy whatsoever, just pooling of drives.

I would like to upvote this also.

 

I am in the process of building myself a new server based on updated hardware and am very much wanting to to a RAID0 of 2 240GB SSD drives for my cache drive.

 

I'd actually think that even better than a RAID0 would be a btrfs "single" where it's basically a JBOD (yes, even though it's called "single", it does support multiple devices).  So it's like unRAID but without a parity device.  So still no protection from device failure (just like RAID0) but at least if you lose 1 disk, you don't lose everything in your pool.

 

Thoughts?  I would think this is better than RAID0 especially for SSDs.  I realize that RAID0 has write-performance benefits, but I don't think those benefits are very strong when A) dealing with SSDs and B) dealing with a NAS.

Link to comment

Can we have the option of simply pooling drives together as one mount point? (i.e. /mnt/cache) so effectively.. RAID0 - No redudancy whatsoever, just pooling of drives.

I would like to upvote this also.

 

I am in the process of building myself a new server based on updated hardware and am very much wanting to to a RAID0 of 2 240GB SSD drives for my cache drive.

 

I'd actually think that even better than a RAID0 would be a btrfs "single" where it's basically a JBOD (yes, even though it's called "single", it does support multiple devices).  So it's like unRAID but without a parity device.  So still no protection from device failure (just like RAID0) but at least if you lose 1 disk, you don't lose everything in your pool.

 

Thoughts?  I would think this is better than RAID0 especially for SSDs.  I realize that RAID0 has write-performance benefits, but I don't think those benefits are very strong when A) dealing with SSDs and B) dealing with a NAS.

I personally would rather have the RAID0 for maximum speed on my cache drive.  I don't run cache pool right now but I want maximum speed on my cache drive.  I plan on backing up the docker image to the parity protected array just in case so I don't care about "loosing" one of the drives in the RAID0 cache.

Link to comment

Can we have the option of simply pooling drives together as one mount point? (i.e. /mnt/cache) so effectively.. RAID0 - No redudancy whatsoever, just pooling of drives.

I would like to upvote this also.

 

I am in the process of building myself a new server based on updated hardware and am very much wanting to to a RAID0 of 2 240GB SSD drives for my cache drive.

 

I'd actually think that even better than a RAID0 would be a btrfs "single" where it's basically a JBOD (yes, even though it's called "single", it does support multiple devices).  So it's like unRAID but without a parity device.  So still no protection from device failure (just like RAID0) but at least if you lose 1 disk, you don't lose everything in your pool.

 

Thoughts?  I would think this is better than RAID0 especially for SSDs.  I realize that RAID0 has write-performance benefits, but I don't think those benefits are very strong when A) dealing with SSDs and B) dealing with a NAS.

I personally would rather have the RAID0 for maximum speed on my cache drive.  I don't run cache pool right now but I want maximum speed on my cache drive.  I plan on backing up the docker image to the parity protected array just in case so I don't care about "loosing" one of the drives in the RAID0 cache.

 

Hmm, I get the desire for maximum speed, but are we talking purely about writing data to the array over the network?  If so, you will gain nothing by using RAID0 for cache.  You also wont' gain anything by using an SSD over an HDD for the cache in that scenario.  Where SSDs really win is in 3 key areas:

 

1)  No spin up delays and less latency.

2)  Docker applications performing IO intensive tasks will perform better (this is subjective).

3)  Virtual Machines (especially localized ones) will see a massive performance increase.

 

#3 is the only one that I'd expect to see a performance gain with RAID0 on (potentially), but even that's arguable.  If you're not using containers or VMs (just using the cache for it's network write-speed increase), the bottleneck will be your network connection which on a 1gbps link is capped at 125MB/s (within performance capabilities of HDDs).

 

Just trying to understand the desire / use case for RAID0.

Link to comment

Just trying to understand the desire / use case for RAID0.

 

More storage.

No storage amount difference using BTRFS jbod capability vs raid 0

 

I don't have a horse in this race, but I figured the reason they'd maybe want RAID-0 over RAID-1 would be storage.

 

I was unaware they already had a means of doing jbod pooling. So now I'm curious why RAID-0 over JBOD too. The only possible difference would be RAID-0 giving even higher insane level of IOPS over JBOD? I don't even know if that would be a potential benefit as even a single SSD drives already provide insane level of IOPS.

 

*shrug*

Link to comment

Just trying to understand the desire / use case for RAID0.

 

More storage.

No storage amount difference using BTRFS jbod capability vs raid 0

 

I don't have a horse in this race, but I figured the reason they'd maybe want RAID-0 over RAID-1 would be storage.

 

I was unaware they already had a means of doing jbod pooling. So now I'm curious why RAID-0 over JBOD too. The only possible difference would be RAID-0 giving even higher insane level of IOPS over JBOD? I don't even know if that would be a potential benefit as even a single SSD drives already provide insane level of IOPS.

 

*shrug*

 

We're on the same page now.  I agree that having an alternative to RAID1 (regardless of SSD usage) would be great.  I just don't see the reason it should be RAID0 vs. JBOD.  I guess RAID0 with HDDs for the cache could be better for VMs than JBOD, but with SSDs, I doubt it.  I also would think that JBOD could be better for isolating IOPS (if we could find ways to direct IO to specific disks).  I haven't spent much time reading through the options with JBOD mode in BTRFS yet, and this isn't a feature we'd target for 6.0.

Link to comment

 

Hmm, I get the desire for maximum speed, but are we talking purely about writing data to the array over the network?  If so, you will gain nothing by using RAID0 for cache.  You also wont' gain anything by using an SSD over an HDD for the cache in that scenario.  Where SSDs really win is in 3 key areas:

 

1)  No spin up delays and less latency.

2)  Docker applications performing IO intensive tasks will perform better (this is subjective).

3)  Virtual Machines (especially localized ones) will see a massive performance increase.

 

#3 is the only one that I'd expect to see a performance gain with RAID0 on (potentially), but even that's arguable.  If you're not using containers or VMs (just using the cache for it's network write-speed increase), the bottleneck will be your network connection which on a 1gbps link is capped at 125MB/s (within performance capabilities of HDDs).

 

Just trying to understand the desire / use case for RAID0.

#2 and #3 along with more storage

Link to comment

 

Hmm, I get the desire for maximum speed, but are we talking purely about writing data to the array over the network?  If so, you will gain nothing by using RAID0 for cache.  You also wont' gain anything by using an SSD over an HDD for the cache in that scenario.  Where SSDs really win is in 3 key areas:

 

1)  No spin up delays and less latency.

2)  Docker applications performing IO intensive tasks will perform better (this is subjective).

3)  Virtual Machines (especially localized ones) will see a massive performance increase.

 

#3 is the only one that I'd expect to see a performance gain with RAID0 on (potentially), but even that's arguable.  If you're not using containers or VMs (just using the cache for it's network write-speed increase), the bottleneck will be your network connection which on a 1gbps link is capped at 125MB/s (within performance capabilities of HDDs).

 

Just trying to understand the desire / use case for RAID0.

#2 and #3 along with more storage

Capacity is the same regardless of raid 0 or jbod style btrfs pool. 

 

On #2 and 3 I would wager raid0 wouldn't yield much performance improvement that you would notice. That is unless you are running some crazy database apps with large quantities of users both reading to and writing from it concurrently.

 

I'm actually curious enough that I'll probably do a manual test with raid 0 and single mode to see what happens with my localized VM.

Link to comment

I would like to simply pool multiple SSDs onto a single mount point. (i.e. /mnt/cache) Not sure if/how share allocation would work in this case? (High-Water / Free Space etc..)

While those setting appear on the share settings page for a cache-only share I don't think they have any meaning.
Link to comment

I would like to simply pool multiple SSDs onto a single mount point. (i.e. /mnt/cache) Not sure if/how share allocation would work in this case? (High-Water / Free Space etc..)

While those setting appear on the share settings page for a cache-only share I don't think they have any meaning.

 

The cache pool currently uses a BTRFS RAID 1 method to group the devices together, and therefore there is no option for controlling which files get written to which disk.  Using btrfs single, however, there may be methods to direct write operations to a specific disk as opposed to letting BTRFS "auto-decide".  I need to research that more when we're looking closer at this feature.

 

- Jon

Link to comment

I would like to simply pool multiple SSDs onto a single mount point. (i.e. /mnt/cache) Not sure if/how share allocation would work in this case? (High-Water / Free Space etc..)

While those setting appear on the share settings page for a cache-only share I don't think they have any meaning.

 

The cache pool currently uses a BTRFS RAID 1 method to group the devices together, and therefore there is no option for controlling which files get written to which disk.  Using btrfs single, however, there may be methods to direct write operations to a specific disk as opposed to letting BTRFS "auto-decide".  I need to research that more when we're looking closer at this feature.

 

- Jon

 

I thought Raid 1 was a mirror raid and the same data was written to all discs

Link to comment

I would like to simply pool multiple SSDs onto a single mount point. (i.e. /mnt/cache) Not sure if/how share allocation would work in this case? (High-Water / Free Space etc..)

While those setting appear on the share settings page for a cache-only share I don't think they have any meaning.

 

The cache pool currently uses a BTRFS RAID 1 method to group the devices together, and therefore there is no option for controlling which files get written to which disk.  Using btrfs single, however, there may be methods to direct write operations to a specific disk as opposed to letting BTRFS "auto-decide".  I need to research that more when we're looking closer at this feature.

 

- Jon

 

I thought Raid 1 was a mirror raid and the same data was written to all discs

 

It is, but btrfs RAID 1 is slightly different.  Traditional RAID 1 solutions only support two physical disk devices (no more, no less).  BTRFS RAID 1 is different because it will allow for more than 2 disks and even support odd numbered configurations.  BTRFS RAID 1 just wants to make sure that for each write operation, two devices get it.  That's all.  So in a 3 disk configuration, the writes will spread across all 3.  Keep in mind though from a capacity standpoint, you could end up sacrificing some space if you don't use disks that conform to this idea. 

 

http://carfax.org.uk/btrfs-usage/

 

That calculator will let you figure out usage based on various raid settings with btrfs and various disk devices of various quantities.

Link to comment
  • 2 weeks later...

So jonp, I have not had a chance to mess with this but I was wondering if I could perhaps do this via the command line manually? Kind of like what was done initially for cache pool for unRAID v6 beta 6.

 

I probably wont be playing with this for another week or two but figured I would ask the question now.

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.