I've been taking it easy on social media lately - for all the usual reasons: useless time-suck, less interesting than it used to be, anonymous arguments held behind the shield of internet-based anonymity making me lose faith in humanity, etc., etc. Perhaps ironically, I've instead been occupying more and more of my online downtime on Reddit.
That may sound nuts to some, since Reddit is often regarded as a notorious cesspool of online hooliganism, but in my experience, you can fairly easily tune your home page to your interests and wind up with a social media experience that's far superior to what Facebook or Twitter offer. And depending on the types of subjects you're into, you might be a lot less likely to encounter pointless flame-wars. With my current subscription diet, I experience virtually none.
Given the content of many of my blog posts, it should be no surprise that some of my favorite subreddits are /r/unRAID, /r/PleX, /r/HomeServer, and more recently, /r/homelab.
The latter is full of guys (I think it's safe to assume that 99%+ are guys) who are way more ambitious than I am in terms of setting up home server labs. My home "lab" is very modest by their standards:
This is in the "electrical closet" in my basement which has the circuit breaker panel, the sump pump, and where all the Cat-5 wiring for the house comes together. One of the first things I did when I moved in was to convert many of the phone jacks into ethernet jacks and wire them into a switch which is up on the top shelf. Also up there is the cable modem, router, and UPS backup battery, which only the networking equipment is connected to.
The rack isn't a real rack in the IT sense of the word, this is just a cheap garage type rack I picked up at Home Depot.
Next to the monitor on the second shelf is a small "server" which runs Debian Linux. This machine predates my UnRAID box, which is on the bottom. Over time, the UnRAID machine has taken over more and more of the server duties, but the little Zotac box still houses the webserver that runs this blog and my main website, plus most of the custom server type apps that I've coded myself, like the home automation events server and home notifications server.
Anyway, with these machines and others strewn throughout the house, I started toying with the idea of a dashboard to monitor them. And the zealots on /r/homelab didn't disappoint.
Enter Grafana
I had never heard of Grafana before reading about it on /r/homelab, but it's a fairly common tool for doing this sort of thing. By the sound of the name, it sounds like it was originally designed to be used in conjunction with Graphite, a fairly simple time-series database. You use various agents on your machines to monitor things like CPU and memory utilization and log them to your Graphite server, and then graph them with Grafana.
Nowadays it seems like other tools like InfluxDB are more standard than Graphite, but in toying around, I started with Graphite, so I've kept it thus far. I actually moved the whole setup to a docker container on my UnRAID server which runs all the necessary monitoring, storage, and graphing bits in the one container.
Right now my main dashboard panel looks like this:
I haven't figured out how to do network utilization with Graphite yet, which is next on my list.
I'm obviously monitoring the disk space on various important drives, as well as the up/down status of various machines in the house.
I had never really played much with time-series databases before, it took me a while to understand the way they time slice and archive the info over time (you usually don't need detailed info from a long time ago, so they have strategies for aggregating as it ages).
And Now For My Obligatory Comcast Reference
One of the less-common items I wanted to be able to see at a glance was my current utilization of my monthly Comcast data cap. Comcast notoriously imposes a 1TB monthly cap on bandwidth for consumers customers, after which you pay $10 for each 50GB you go over, up to a limit of $200 per month. They give you two "grace period" months, one of which I've already used. Looking back at historical data, before I ever paid attention I was usually right around that 1 TB mark each month, sometimes a bit under, sometimes a bit over.
Comcast of course does this purely as a way to further milk their customers - they are not in danger of running out of bandwidth.
One of bit of indirect evidence I have for this is that Comcast doesn't make it terribly easy for you to check your usage. Originally when they imposed the cap in some markets, there wasn't a way to check at all. Later they supposedly created an API for you to query it, which is great, but they've since retired it. Now, you have to log in through their slow and clunky website to check it. But I argue that if they really cared about people not using too much bandwidth, they'd make it a lot easier to check. Quite the contrary, it's in their interest to let you go over, so they can charge you for doing so.
However, never fear, the internet is still awesome despite Comcast's attempts to ruin it: someone has written a Python-based Selenium script called xfinity-usage to grab the info and optionally log it to Graphite. As a result, I now check every three hours and have a current readout gauge on the dashboard as well as a graph over time - very cool!