• Content count

  • Joined

  • Last visited

Community Reputation

2 Neutral

About JTok

  • Rank
  • Birthday August 25


  • Gender
  • URL
  • Location
  1. Sanity check - 24 drive bays

    There are some limitations associated with Plex and hardware acceleration that you may want to familiarize yourself with before spending a lot of money on hardware. For one, Nvidia limits encoding to 2 streams on at least some (if not all) of their GPUs. There are also potential issues with dockers, if that's how you are running it. I would read through this page as well as the Plex forums to get a feel for the best purchase for what you want to do. It may be in your best interest to re-ask this question there as well. https://support.plex.tv/articles/115002178853-using-hardware-accelerated-streaming/ Sent from my iPhone using Tapatalk
  2. I have made a few updates to the script again. Added the option to disable delta syncs. Added option to only use rsync. Dry-runs can be used with the rsync_only option. Dates and times are now included in log messages. Log files can be generated (enabled by default) Error logs can be managed separately from regular logs. Option to choose the number of logs to keep. Log messages can be sent through unRAID notification system. As with the last update, I've tested as best I can, but I can't guarantee everything. The script can still be found here: https://github.com/JTok/unraid-vmbackup -JTok
  3. A few other words of advice: Avoid using the DMZ on your router. Only forward the ports needed to the device that needs it. Make sure that whatever username/password you use for accessing your shares isn't the same username/password you use to manage your game servers. (Honestly, I highly recommend using a unique username/password for accessing shares. Depending on your level of paranoia you can tell your desktop to remember it or not.) I don't know your level of technical expertise with networking, but you could consider VLANs (potentially complicated)... or if you will have two NICs on your server you could set up two physical networks (potentially less complicated). Either option could be set up in such a way that it isolates your game servers from the rest of your network to mitigate some of the risks. Both options could carry some additional costs if you don't already have the equipment available.
  4. [Plug-In] unBALANCE

    Hello, I just happened to notice that after using this that it seems to de-sparsify (un-sparsify?) files. Rsync has a --sparse option, but I'm not sure what effect that would have on non-sparse files. Specifically my vdisk backups were affected. If I had moved docker.img, or libvirt.img, they would presumably have been affected as well. I also have some dockers that create sparse files. Disclaimer: Unfortunately, I forgot to double-check the files it de-sparsed (de-sparsified?) before I removed them, so maybe I'm wrong about this. It would be worth double-checking. You could run a check like this to return a list of sparse files. The command searches sub-directories. find /directory/to/be/copied -type f -printf "%S\t%p\n" | gawk '$1 < 1.0 {print}' Hopefully this information is useful to someone. Thanks, JTok Edit: I forgot to include a link to the code source: https://www.thegeekdiary.com/how-to-find-all-the-sparse-file-in-linux/
  5. I have updated the script on github to include the following changes: Moved variables close to the top to make them easier to access. Added the ability to skip specific vdisks. Restoring the VMs to their previous state can be overridden. Backing up xml and/or vdisks can be skipped. Number of days to keep backups can be set. Changed to use mmin instead of mtime because of issues with accuracy. see: https://unix.stackexchange.com/a/92351 Number of backups to keep can be set. Always checks for img and qcow2 files by default. Timestamps are always used except when the number of backups being kept is set to 1. Toggle for compressing backups. Option to compare backups to original file and retry once. A few other minor things. Currently I've tested everything as well as I can, but I can't guarantee everything will work, so please feel free to let me know if you find any issues. I've tried to make it easy and customizable as possible. You can find the script here: https://github.com/JTok/unraid-vmbackup -JTok
  6. I have created a variation on the original script that uses cp for the initial copy and then uses rsync for subsequent copies. I did a whole lot of testing on my own system and I found that while dd was slightly faster than cp, it did not create sparse files very well. Rsync was so much slower that it was nearly pointless to run. It did sparse files about as well as dd though. All of my tests were from my SSD cache array to my storage array. I also found that using rsync to delta sync was about as fast as re-copying the whole file using cp/dd when using --inplace. However, since the inplace argument creates some risk, and I did want to keep one rolling backup, I have the script make a copy first. Effectively the script does this: First run: uses cp to create a sparse copy of your vdisk. It also makes a copy of your xml. Second run: BEFORE shutting down the VM it makes a copy of the backup from step one using cp. Then it shuts down the VM and uses rsync to perform a delta sync with the inplace argument. Subsequent runs: BEFORE shutting down the VM it uses rsync to perform a delta sync of the latest backup to the second copy with the inplace argument. Then it shuts down the VM and uses rsync to perform a delta sync with the inplace argument. I also changed the script to make a note of the state of the VM when it starts the backup so that it can restore the VM to that state. I have some VMs that I don't keep running, so I don't want them started after a backup. Paused VMs will just be left in the stopped state. I *think* the timestamps will still work with this, but it will just use the cp option for everything. I'm also pretty sure it will keep making them forever without clearing them out. I am working on fixing that. You can find the script here: https://github.com/JTok/unraid-vmbackup Please be sure to read the Change Log and To-Do List as they contain important notes about the state of the script currently. -JTok Edit 1: forgot to mention: the original script does not make sparse copies. Additional explanation: When you create a VM you choose a disk size. Let's say we pick 40GB. Then we install an OS. The OS is taking up 10GB of that 40GB, so the vdisk will say it is 40GB, but only take up 10GB of actual space on the storage disk. This is highly desirable with lots of VMs. However, the original script didn't use the --sparse argument with rsync so it would create a 40GB vdisk that took up 40GB of actual space on the storage disk even though 30GB of that was actually just zeros. So sparse files are basically files that the file system knows are smaller than they appear to be, because they don't have all the unnecessary zeros in them. Edit 2: I have tested restoring the files using cp from the shell to put them back and they do work. However, I have not tested restoring them via the shares directly. You have been warned.
  7. [Plugin] Ransomware Protection

    I have been reading about how this plugin works, and I have an alternative method that might work as well (rumor has it that @Squid likes feature creep). Hopefully I didn't miss a post in this thread where someone already talked about this method; I tired to read through the whole thing before posting. I have some experience detecting ransomware in Windows environments, and we use a slightly different method to detect it on file shares that could be appealing to some here. Instead of using bait files we just monitor files against a list of known extensions and file-names used by ransomware, and use a white-list to override it for application specific extensions that we might have in our environment. The pros: - You know when I file gets modified without relying on the ransomware to pick your bait files/shares first - You don't have bait files/shares The cons: - One of your actual files has to be encrypted before you know it's happening (though if a bait file is not picked first, you may still be in the same boat or worse) - You are relying on a list to be kept up-to-date I would love to now offer you some code and a more concrete explanation of how I think this could be implemented, but it has been some time since I had to do anything beyond the most basic scripting in Linux. I'm also assuming that this is possible, and not a burden on the system; I just don't have enough experience to guess at at this point. Widows reference materials (the first link maintains a list of known bad files/extensions here): https://fsrm.experiant.ca/ https://community.spiceworks.com/how_to/100368-cryptolocker-canary-detect-it-early Another alternative is some kind of behavior monitoring (i.e. watching for encryption), but I suspect that is far more difficult to implement. (We use this as well, but pay handsomely for the feature.) Regardless, thank you for the plugin (and all the others). -JTok Quick Edit: Forgot to mention that I've seen fail2ban be used for something similar by reviewing samba logs. The original is in German, so here is the google translate link and the original. Not sure it makes sense to use in this case, but hey, I'm told that knowledge is power. Translated: https://translate.google.com/translate?sl=auto&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=http%3A%2F%2Fwww.heise.de%2Fsecurity%2Fartikel%2FErpressungs-Trojaner-wie-Locky-aussperren-3120956.html&edit-text Original: https://www.heise.de/security/artikel/Erpressungs-Trojaner-wie-Locky-aussperren-3120956.html
  8. Follow-up: I apparently spoke too soon. The issue has returned. I'm still reasonably certain that this isn't a hardware problem, but I have spare components I can swap in (despite the current hardware having passed every test I've run). It will be about a week or two before I'm able to get everything swapped and tested, but I will report back once done in case it matters.
  9. This seemed to fix the issue for me as well. For the sake of complete records: I am running Turnkey Linux Core (Debian based); https://www.turnkeylinux.org.

Copyright © 2005-2017 Lime Technology, Inc. unRAIDĀ® is a registered trademark of Lime Technology, Inc.