We have a central instance of smbd that we allow users to have home directories on, as well as project specific shares. It’s a beefy ZFS on Linux instance that we call “tank” in reference to Jeff Bonwick’s discussion of the humble ZFS beginnings within Sun. We run a backup strategy between it and a couple other trailing-mirrored instances that we’ve positioned around the facility. We have a desire to be able to review everything a user did in terms of interaction with tank. This can be accomplished via something called a “Stackable VFS Module”.
We’ve recently gotten some significantly larger storage systems and after running some 50T pools with basically all the defaults it felt like time to dig into what common options are used to chase performance. The intended use for these systems is ultimately CIFS/NFS targets for scientists who are running simulations that generate small (1M) to large (100G) files. I’m not being rigorous and offering any benchmarks, just digging into documented performance parameters and explaining the rational.
I stumbled across a discussion about using the ssd parameter as a mount option with BTRFS and realized that I was very likely afflicted by what was being discussed in the mailing list. I'd not anticipated any maintenance operations that would be necessary when starting to use BTRFS as a daily driver, but IRC and the community was incredibly helpful.
As our digital legacies continue to grow with advanced capture systems, its important to consider taking some control over assuring how that legacy is perpetuated. There are many enterprise grade technologies that are within easy reach of the individual who wants to start building out their home lab. Rather than send monolithic emails to people who inquire about this topic it seemed more appropriate to write out my opinions openly, and maintain them as more experience is gained. This is written from the approach of accomplishing this task 'properly' as I personally see it. People have different initial objectives when considering this project. I'll focus exclusively on storage and backup while ignoring things like streaming/trans-coding, or acquisition.