Failed NVMe drive? Better change your upstream DNS resolver.

This is not entirely an “it’s always DNS joke,” I promise. But for the record, it is always DNS. I have an (three, actually - two are boot drives/VM storage) NVMe drive in my server that serves multiple functions - a Plex cache, a ZFS Intent Log (I should get rid of that, as it’s shown to have 0% gains for me), and a general scratch disk for anytime I need a fast disk....

2021-02-03 · 3 min · Stephan Garland

Hardware Sucks

It’s coarse and rough and irritating and… it sucks. There’s a reason why cloud providers are wildly popular. Scaling in AWS is incredibly easy; not so much when you’re rolling your own. God help you if you’re doing this by cobbling together disparate groups of enterprise and consumer hardware. Oh wait, that’s exactly what I’m doing. I’ve had a Linux box since 2016 or so. I mean, I dual-booted every distro known to man with Windows when I was a teenager (including bootstrapped Gentoo, and I managed to get a crotchety HP printer to work with it), but I didn’t have a dedicated Linux box until then....

2021-01-03 · 12 min · Stephan Garland

Ascertaining Gender of Github Users to Determine PR Comment Sentiment, or How to Spend Money in the Cloud and Pretend you're a Data Scientist

This is not so much of a SRE post, be ye warned, but it does discuss solving problems creatively. WARNING: Blindly following this will result in incurring costs from cloud providers, potentially quite a bit. It’s not my fault if you don’t calculate how much your task will cost. As an example, had I used AWS Comprehend for my sentiment analysis, I calculated the cost at ~$15,000. Why on earth would you need to know the gender of Github users, you ask?...

2020-11-10 · 7 min · Stephan Garland

Safely Saving Secrets

Because alliteration. Moving on. If you’re interacting with APIs of any kind regularly, you probably have the credentials saved somewhere. Maybe you’re already using a solution to securely store these, in which case congratulations, you’re better than most. I, for one, was not. I assuaged my guilt with the knowledge that my Mac’s disk encryption meant that they were protected, but the whole thing still felt icky. This was briefly discussed in Slack, and this method of dealing with the problem came up....

2020-11-10 · 4 min · Stephan Garland

Refrigerator Monitoring for Fun and Profit

I’m not sure if this is common, but it should be - a second fridge. While ours ostensibly began its life for holding cakes (my wife is a home baker), it’s morphed into extra food, beer, and the like. It lives in our garage, which isn’t an ideal space for a fridge with respect to its lifespan, but it’s the only reasonable spot in the house we can fit it....

2020-09-17 · 5 min · Stephan Garland