Proxmox

It’s finally happening - I’m splitting compute and storage up, and using k8s to do so. I bought two more Dell R620s, bringing my total to three - behold, quorum. This will be a multi-part series, focusing on different things I did wrong. I want HA, or as close as I can. I know I still have SPOF with power (aside from my UPS), internet, and switching, but my real concern is the ability to drop a node and not notice. I often find myself wanting to tweak something at an inopportune time, i.e. during the day, and this disrupts the household internet. A capital crime, to be sure. In order to accomplish HA without having a truly ludicrous amount of physical servers, I need three servers. Since I don’t need to dedicate any of them strictly as control planes, and since the R620 is dual-socket, I’ll install Proxmox as the hypervisor and run a control plane and worker on each node. ...

I’ve had my dev VM upgraded to Debian 11 for a few months, and have fiddled with from time to time. It’s not that I distrust Debian stable in the slightest to not be stable, but I also am not going to just throw prod a sudo apt full-upgrade (yes, with updating sources first…) and hope for the best. As I didn’t experience any issues with workflow with the upgraded VM, I decided it was time to bake some new images. ...

I’ve now accomplished one of my previously-mentioned desires, namely, to use Packer to make VMs for Proxmox. After much battling with YAML and esoteric bash commands, I have succeeded in being able to spawn endless VMs, ready to go just how I like them. As with many projects I’ve done, this was thanks to someone else’s hard work; I merely customized it. The repo[s] are in two parts, and for my forks, only the Debian template has been customized. It comes with templates for Alpine and Ubuntu as well, but I’ve not done any work to them. ...

It’s coarse and rough and irritating and… it sucks. There’s a reason why cloud providers are wildly popular. Scaling in AWS is incredibly easy; not so much when you’re rolling your own. God help you if you’re doing this by cobbling together disparate groups of enterprise and consumer hardware. Oh wait, that’s exactly what I’m doing. I’ve had a Linux box since 2016 or so. I mean, I dual-booted every distro known to man with Windows when I was a teenager (including bootstrapped Gentoo, and I managed to get a crotchety HP printer to work with it), but I didn’t have a dedicated Linux box until then. I had a Synology DS413 in 2012, but busybox barely counts. In 2016, the IT manager at my employer was kind enough to donate a Dell T310 to me that the company no longer had use for. After buying an H200 HBA and flashing it to support pass-through, I installed Debian and was off to the races. A friend convinced me to learn Docker, and everything after that just kind of came naturally. ...