Self Hosters of Privacy Guides, tell us about your setups

I’m curious to learn about what other self-hosters are running and doing. The questions below are just to prompts to get the conversation started, you can answer some or all of the, or answer however you want.

  1. Own hardware or VPS? Hardware Specs?
  2. What is your base OS?
  3. Do you use containers (docker/podman/etc)?
  4. What are some of the services you self-host?
  5. What are some of the steps you’ve taken to secure your server against network and physical threats?
  6. How do you handle backups?
  7. What would you do differently if you were to start from scratch?
4 Likes

To kick things off:

Hardware?

In the past, I used a VPS, recently purchased a used low power enterprise mini-pc. It sips power, is smaller than an average router, and is quite powerful for its size/power consumption.

Base OS

Proxmox (Debian based hypervisor),

On top of Proxmox there will be one main VM guest, probably MicroOS, which will act as a container server.

What are some of the services you self-host

Not entirely sure yet what all I will self-host. I do know I will be self hosting:

  1. Nextcloud
  2. Adguard Home or Pi-hole
  3. Immich or another photo hosting service.

Questions 5, 6, 7

TBD

For physical security, Ideally I’d like FDE unlocked via TPM2 with secure boot. But this is non-trivial in proxmox (or any Linux distro for that matter) and above my level of competence, so I probably won’t go this route.

2 Likes

As someone who is still just at the educating myself stage of the whole self-host thing, I sure wouldn’t be against knowing your choice of brand and model.

It is actually super easy with modern systemd:
PCRs

  • 0 = UEFI firmware
  • 1 = UEFI settings
  • 4 = bootloader/grub
  • 7 = secure boot state
  • 8 = kernel command line
  • 14 = shim state

Initial

  • systemd-cryptenroll --tpm2-device=auto --tpm2-pcrs=0+7 /dev/$DEVICE
  • add to kernel command line: rd.luks.options=UUID=tpm2-device=auto
  • add to crypttab ,tpm2-device=auto

Reseal on shim update

  • systemd-cryptenroll /dev/$DEVICE --wipe-slot=tpm2 --tpm2-device=auto --tpm2-pcrs=0+7+14

Reseal on shim/grub/kernel/cmdline update

  • systemd-cryptenroll /dev/$DEVICE --wipe-slot=tpm2 --tpm2-device=auto --tpm2-pcrs=0+4+7+8+14

I only recommend using 0 and 7, unless you want headaches. 14 works if you’re fine with occasional resealing and have working console access.

1 Like

Here is my single consolidated home server for NAS and VM host.
Specs are 5900x/128GB with 36TB HDD & 1.5TB SSD, a LSI 9607-8i, and Mellanox ConnectX-3 40Gb.

It runs Fedora, and all services are each in their own full Fedora VM.
I manage the VMs using virt-manager, and I keep template VMs for different purposes.

Configuration is generally the same as what I document here: https://divested.dev/pages/infrastructure
Largely through Brace: Divested Computing Group / Brace · GitLab


I take a bit of a different path for network storage, it is solely OpenSSH based with dedicated nologin/chrooted users with only public key authentication.
I plan to write a guide up on that eventually, but here are the basics:

Script to create a user, use groupadd sftponly once only, then this:

#!/bin/sh
cd /mnt/share-1;
useradd -G sftponly -s /bin/nologin -d /mnt/share-1/$1 $1;
chown -v root:$1 $1;
chmod -v g+rx $1;
cd $1;
mkdir .ssh data backup sync;
chown -v $1:$1 . -R;
systemctl restart sshd;

You’ll also need this for Fedora’s selinux policy:

semanage fcontext -a -e /home /mnt/share-1
restorecon -rv /mnt/share-1

The sshd_config bits

Match Group sftponly
	ChrootDirectory %h
	ForceCommand internal-sftp
	#brace
	AllowTcpForwarding no
	GatewayPorts no
	PermitListen none
	PermitOpen none
	PermitTTY no
	PermitTunnel no
	PermitUserRC no
	X11Forwarding no
	#Don't require 2FA
	AuthenticationMethods publickey
	ChallengeResponseAuthentication no

additionally if you want to allow use of borg:

Match Group borgonly
	ChrootDirectory %h
	ForceCommand borg serve --restrict-to-path $HOME --append-only
	AllowTcpForwarding no
	PasswordAuthentication no
	PermitEmptyPasswords no
	X11Forwarding no

and on the client end for /etc/fstab:

user@server:/data/ /mnt/remote fuse.sshfs rw,defaults,allow_other,noauto,x-systemd.automount,_netdev,noatime,nosuid,nodev,reconnect,dir_cache=yes,max_conns=4,ConnectTimeout=20,ServerAliveInterval=10,Compression=no,Ciphers=aes256-gcm@openssh.com,IdentityFile=/home/user/.ssh/id_ed25519.whatever,uid=1000,gid=1000 0 0

Other things:

  • I personally use restic however for all backups of all machines.
  • I use ZFS on the drives, I have a cheat sheet here: https://divested.dev/misc/zfs.txt
  • All the hard drives are used enterprise SATA/SAS 6TB ones from eBay, they’re only $28-35 and I’ve only had about 1/13 fail on me so far. Even if one fails, it is only $30 to replace, as opposed to expensive larger sized drives. But you’ll have to do the TCO analysis for your usecase.
  • If you’re buying LSi cards, be sure to avoid fakes, there are some videos on YouTube comparing them.
  • Be sure you allocate enough PCIe bandwidth to your cards, do the math before buying, etc.
  • The Mellanox CX3 is an extremely cheap way to get high speed 10g/40g networking, they’re both $30 on eBay. Combine it with a quiet/efficient 10g ICX6450 ($80) or louder/power hungry 10/40g ICX6610 ($50) for switching. Or just direct attach even, great thread here: Brocade ICX Series (cheap & powerful 10gbE/40gbE switching) | ServeTheHome Forums
  • Make backups, always make backups, do not push them off.
  • Make backups of your backups.
  • And actually verify your backups work!
  • For passing folders into a VM, virtiofs is amazing for most use cases, something that you should try if you haven’t.
9 Likes

I’ve just gone throught the process of researching, narrowing down, and purchasing hardware. I’m happy to share what I’ve learned. My budget was <$150, excluding storage, with a target of <$100, so I can’t give much input on more expensive systems.

Right now, used enterprise mini-pc’s are about the best price/performance ratio you will find. I considered two other form factors (1) a NAS (2) a ‘network appliance’/fanless mini PC, but ultimately went the used enterprise route because the value was much better. One of the best resources for info on these systems is ServeTheHome’s “Project TinyMiniMicro”.

There are 3 brands (Lenovo, HP, and Dell) they all offer mostly comparable systems afaict. One brand or another may have a slight edge for a given generation, but mostly they are pretty comparable.

If you plan on media encoding / streaming, something with at least a 7th gen intel CPU is advisable (because of Intel Quicksync Video, and support for H265). You can find really cheap 6th and 7th gen intel systems with RAM and SSD for about $60-90. Bumping up to 8th Gen will roughly double the cost ($100-200) but you get a 6 core CPU, and with HP, this generation had 2 NVME slots which is really nice (example) To me, this generation of HP (Elitedesk 800 G4) is the sweet spot between price, performance, and low power. The system I got has space for 2xNVME + 1x2.5" drive bay, which is really good for ~$100 and the size of a box of chocolates. With these used enterprise systems, you’ve got one other choice, they are all available in the ‘Tiny/Mini/Micro’ size, and a larger “Small Form Factor” size (example) which is larger but has some expansion slots and the big advantage is more storage. For example, the SFF version of the HP system with the 8th gen CPU I mentioned earlier can fit 2 x NVME, 1 x 2.5" drive, and 2 x 3.5" HDD.

Other options I would consider: A NAS (Like Synology, or DIY option with TrueNAS), SBC (like a raspberry pi, or alternative), Fanless Mini PC like this. Do bear in mind that my priorities lead me towards these smaller lower power systems, which will not be a good fit for everyone, but are perfect for me.

2 Likes

Good grief mate, this was exactly the information I needed as my self-host needs are almost too low to justify doing it in the first place. You probably just saved me weeks of work and serious head scratching :heart:

1 Like

I really want to buy a killernas soon, will do when get a chance.

1. Own hardware or VPS? Hardware Specs?

Own Hardware. All used parts off eBay and various other markets or brand new but on sale. i5 in a Micro-ATX with 16gb ram, 2x 8TB HDD in raid, 1tb ssd cache with a ssd for TrueNAS scale OS.
Also have a old Dell Optiplex with a Intel Nic as my OPNSense firewall and 2x 16GB NUCs with proxmox installed which I am still in the process of learning and setting up.

2. What is your base OS?

TrueNAS Scale on my NAS, Proxmox for my other clients and OPNSense for my router. Could virtualise my NAS or Router but I get worried about virtualising such important things.

3. Do you use containers (docker/podman/etc)?

I use docker in a debian LXC on proxmox for my arr and jellyfin containers. Barely uses 1GB of ram and barely any processing power unless it needs to transcode, haven’t had to transcode yet tho and still trying to properly link an NFS share or equivalent from TrueNAS for the media.

4. What are some of the services you self-host?

Jellyfin and Plex mainly at the moment but want to expand.

5. What are some of the steps you’ve taken to secure your server against network and physical threats?

VLAN’s, Firewall Rules, VPN. Nothing exposed to the outside net at the moment but currently looking into NGINX and others.

6.  How do you handle backups?

NFS Share to my TrueNAS server which has mirror drives. Need to setup automatic backups tho.

7. What would you do differently if you were to start from scratch?

Not much I guess. Maybe build a rack mount server instead of having 1 SFF pc, 2x Mini NUCs and a Micro-ATX.

Circling back to my own question/post. I’m still in the early stages of planning+learning+implementing. and at the moment I’m feeling pretty overwhelmed to be honest. The sheer number of choices and approaches to every choice is overwhelming. But I have got my server up and running.

I’m using Proxmox, I adapted a blog post from over at privsec.dev on setting up Proxmox with ZFS Native encryption which can be unlocked over SSH. I would ideally like some form of secure/measured/verified boot, but I’m not sure if that is feasible considering that I’m trying for a somewhat KISS approach and Proxmox doesn’t officially support secure boot and the ZFS module possibly complicates things. I still need to figure out backups, and networking/vlans, and how I want to connect to the server/network remotely (tailscale, wireguard, cloudflare tunnel, reverse proxy). I am taking notes and plan to write something up when I arrive at a setup I am happy with (if that ever happens).

Dell Mini PC at home, mainly because of ridiculously high electricity prices where I live. Otherwise I would use more powerful hardware. Uses only 3W in idle (without HDDs).

OMV (Debian). Mainly because I didn’t want to deal with setting up NAS functionality myself, TrueNAS (ZFS) isn’t suited for low-energy systems, and unRAID had a few bugs which I couldn’t live with. I don’t like Debian and will switch to a distro close to RHEL like Oracle Linux or a container focused distro, if I ever find the time and motivation to manually add NAS functionality, also because of Selinux.

Yes. Docker for most services, sometimes with different runtimes (e.g. gvisor)

OMV, Nextcloud, Adguard Home, Photoprism, Paperless-ngx and more to come.

Access only via Tailscale. Home network has no open ports. The usual low-hanging best-practices (up-to-date, firewall config, …). HDDs with FDE.

USB HDD for backups, most important files additionally backed up to a a cheap cloud service (encrypted).

Move to a country with much lower electricity prices. Makes hardware selection sooo much easier and you can get more powerful hardware without paying ridiculous monthly bills. More powerful hardware means more room for better solutions like TrueNAS and spinning up VMs for more secure isolation.

1 Like

Own hardware, old Gaming PC repurposed.

Ryzen 5 1600, RX 5500 XT (8GB), 16GB RAM.

OpenMediaVault, which is Debian 11 with an easy-to-use WebUI

Yes, Docker Compose all the way!

Nextcloud (File storage and calendar), Jellyfin (Netflix-like local media streaming), Gitea (Github-like git website), Invidious (YouTube frontend) and many more services!

Administration backend is only accessible with a VPN connection to my laptop, reverse proxy tightly configured, Fail2ban for supported services to IP-ban people trying to bruteforce passwords.

I don’t worry as much about physical attacks, but as all my devices, the server and all it’s disks are encrypted.

You’re funny, I like you!

That’s something I still need to improve, a lot. Currently, I have RAID redundancy on a 2TB drive that’s where important / irrecoverable stuff go (nextcloud data, gitea repos). Yes I know RAID isn’t backup, but it works for me.

All the other 20TB have zero backups. I plan to get redundancy for all the data, but that costs a lot, specially in my currency.

Go for Docker directly. In the beginning, Everything I self hosted needed to have a native package because I was lazy to learn docker.

Migrating was a pain in the arse, but fully worth it.

1 Like

Interesting method. I would like to know more about it. Did you write this guide?

This method seems to be security-focused. Why not simply using a Wireguard tunnel in conjunction with other solutions like TrueNAS or Nextcloud?

How do you access it from other OSes, like Android? How do you deal with situations where you don’t have access to your server? Do you implement some kind of two-way sync to have some files synced to your end-devices?