AI in the Homelab, with Ben Thomas

Series two has mostly stayed away from AI and that was deliberate, I wanted to talk about the backup stack, the origin story, my wife’s ongoing tolerance of my cupboard full of gear. Then I got Ben Thomas on for this episode and he dragged me straight back into it, and honestly I’m glad he did. We recorded this one in a hotel lobby in Sydney, which is roughly the safe middle ground between Perth and Christchurch, so if you can hear a bit of background noise on the episode then that’s why.

Ben and I have been mates and colleagues for years now, across a few different companies and these days across the Tasman, and somehow we’ve ended up on completely opposite ends of the homelab spectrum. His lab has a Pure Storage array in it. Mine does not.

The reason I wanted this one on the record is that we’ve both built platforms professionally, we both know infrastructure, and we’ve still landed in different places on how much rope you give an autonomous agent. Not miles apart, but far enough that it’s worth talking about, and that’s the part I keep chewing on.

Garage Cloud vs a Couple of NUCs in a Cupboard

Ben’s lab is, in his own words, excessive to any extent. We used to joke at work that it was bigger than most of the production environments we were looking after for customers, and having now heard the full inventory I don’t think that was much of a joke at all. Five production servers, each with half a terabyte of RAM. A Dell 10 gig network stack running BGP. FortiGate firewalls and a Pure Storage array which is just crazy. Most of the boxes are five to ten years old at this point, which is sort of the whole trick.

He calls it Garage Cloud, there’s merch, and I’ve got the coasters to prove it.

My lab by comparison is a couple of small NUCs tucked into a cupboard, same hobby, wildly different scale, and it’s a decent reminder that there’s no single right answer for what a homelab is supposed to look like. Ben got there through more than a decade of accumulation, and a Celeron box with 16 gig of RAM was apparently the starting point, believe it or not. A fair bit of good fortune along the way as well, becoming friends with a data centre operator who’d tip him off whenever a customer was decommissioning gear, so Nexus switches and all sorts of random kit landed in his lap that would otherwise have gone straight to a skip as e-waste.

The power bill tells its own story. He’s sitting at around 1.2 kilowatts constantly running just to keep the core of it alive, which at his New Zealand rates works out to a couple of hundred bucks a month, and he frames that as a training budget. That logic holds up, to be honest. A course might cost you that, and then you’d still need somewhere to actually run things, a couple of VPSs or some VMs in Azure, and you’d burn through two hundred bucks easily. Try renting a hundred cores and several terabytes of RAM anywhere for that and see how far you get.

And that training budget line isn’t just a way of justifying the power bill to himself, it’s genuinely the use he gets out of the thing. Ben’s shifted disciplines a fair bit over his career, managed services, consulting, building platforms, and now data protection at Veeam, and the technology has changed underneath him the whole way through. So every time the question came up at work, how hard is it to do X, is Y even possible, there’d be the usual debate about whether anyone actually had an environment to test it in, and he could just go and spend a weekend on it at home instead. When VCF started becoming a thing and everyone was trying to work out what NSX looked like sitting inside that stack, he went and rebuilt his three servers and ran the whole VCF stack at home, because he had the RAM and the cores to do it, rather than waiting on someone at work to free up hosts we could decommission.

That’s the bit my cupboard can’t do, and I get reminded of it every time we talk. Ben and I built a Tanzu cluster together a while back, 96 cores and however many gig of memory, and it was a genuinely fun journey, but there was never a version of that running in my lab. Physics doesn’t stretch that far. Same hobby and the same instinct to go break something in order to understand it, his just has a lot more room in it to say yes.

But how does this all translate to AI? Seeing as that’s what we love talking about, and not just AI for the fun of it or because we can, but using AI to actually solve some real world issues he was facing whilst still keeping it all running in his homelab, bridging that gap from homelab to AI.

Three Hermes Builds and a Sassy Bot

The AI side of the lab is where this episode really earns its keep. Ben’s running Hermes Agent as the framework, currently paired with GLM-5.2 through a third party provider on a per token rate. He landed on Hermes over something like OpenClaw because it felt like it had a bit more security and a bit more of a production focus behind it, and because it has routines built in, so the more he uses it the more it goes off and updates and builds its own skills out of those interactions. He’s built at least three genuinely useful things on top of it.

The one that stuck with me is almost mundane and that’s exactly why it works. He runs a full NetBox instance at home for tracking what’s in the racks and what cables are plugged into what, and he uses it for IPAM as well, so he’s hooked an agent into it and now when he’s spinning up a new VM he can just message a Telegram bot and ask what IP is free in a given range. It’ll check NetBox, find something free, create the entry, populate the DNS and hand it back, so no more pinging ranges to see what’s alive. That’s the kind of task AI is genuinely good at, small and bounded and repetitive, and low stakes if it gets something wrong.

The budgeting build was actually his first go at any of this, and it’s a bit more personal. Ben runs an app to track his own spending and the perennial problem was just remembering to sit down and categorise transactions properly, so he built an agent that reviews the day’s transactions each night, makes a recommendation based on how he’s categorised similar spending before, and messages him on Telegram to confirm. He says yes, it updates, and the quality of his household budget data has genuinely got better because the friction of doing it properly basically disappeared. He was hesitant to bring AI into the house at all until he had a real problem for it to solve, which is the same way he approaches everything else in that lab.

It also got a bit cheeky. The bot apparently pulled up his fish and chip shop visits, forty seven times in three years, and messaged him about it completely unprompted. He wasn’t thrilled about the judgment and fair enough really, nobody asked the AI for a lifestyle review.

The kitchen management setup came out of that same idea. It started as a shared shopping list over Telegram, message the bot when you’re out of milk or cat food and it updates the list so nobody’s relying on a scrap of paper stuck to the fridge that never makes it into the car. But it grew from there into something more interesting, because the agent already knows what’s in the pantry and what’s been bought, so he can ask it what he could realistically cook on a lazy Thursday night with a Mexican craving and half an hour to spare. That’s AI doing what it’s actually good at, creative recommendation against a known inventory, rather than being asked to invent facts out of nowhere.

Where We Actually Differ

Right, so this is where the two of us start to pull apart a bit, and it’s the same tension I wrote about after my last AI episode. Trust the tool, but verify what it’s actually doing.

Ben’s comfortable letting his agents operate with a fair amount of autonomy provided the guardrails are right. His automated jobs are locked to read only where it matters, and the bit I really liked is that he won’t let an agent write its own ad-hoc Python to answer a question. If it tries, he stops it and makes it use the pre-vetted commands they’ve already built together, or extend that toolkit properly, rather than freelancing a script he’d then have to read line by line on his phone to work out whether it’s about to do something destructive. That’s a pretty sensible middle ground between full autonomy and micromanaging everything.

The thing is I’ve watched an agent do exactly what he’s guarding against. I had Claude Code go off and run homebrew commands on my Mac a while back and install things I never approved, so I put an explicit guardrail in place, no homebrew without asking me first. Then the other day I asked it to run an nmap against one of my servers, tell me what ports were listening, what was on them, the usual. It didn’t have nmap installed. It wasn’t allowed to go and install it. So it decided the logical move was to write a Python script that mimicked nmap and run that instead, which was honestly a bit mind blowing to watch. I’d given it the what and not the how, and it just found another road.

So I’m still very much on the human in the loop side of this and I don’t think that’s changing any time soon. I want to be the checkpoint, whether that’s approving an action, reading through generated code, or just being the one who says yes before something runs, I want that step to exist.

His actual containment strategy is smart though, and it’s one I should probably steal. The Hermes agent runs on a dedicated box in its own isolated VLAN, only able to reach explicitly allowed things, public internet endpoints or specific firewall rules he’s added himself. Even his Claude usage mostly happens on a remote Linux box rather than his own laptop, which means he’s not worried about an agent trawling his whole machine for whatever random documents he’s got on there, and he can background a long running session in tmux over SSH, shut the laptop, and check in on the code from his phone later.

I run mine on my Mac still. I’ve thought about moving it a few times, spin up a dedicated box, chuck it in the isolated VLAN, lock it down properly, and after hearing him talk through his setup I think that’s the bit where I need to practice what I preach.

That’s the honest gap between us. Ben’s willing to give an agent real autonomy provided the blast radius is contained by the environment it’s sitting in, and I want a person in the loop regardless of the blast radius. Neither of us is wrong or right exactly, we’ve just made different bets about where the risk actually sits.

What We’d Both Tell You

Ben’s advice for anyone starting out was simple and I agree with basically all of it, don’t chase the biggest, baddest gear from day one, get something small in and start experimenting. You don’t need to run models locally on day one either, there are cheap models available through providers like OpenRouter where you can experiment for a few dollars at a time before you commit to anything bigger.

And the hardware doesn’t need to be new. His actual production environment for the things he wants stable, Home Assistant and the household stuff that genuinely can’t go down, isn’t the excessive rack at all, it’s a couple of clustered Dell OptiPlex minis tucked into a network cabinet. Old, boring and reliable, which is much the same thing I’ve said myself about NUCs, the form factor available now means you can do properly useful things without spending big or chasing the newest silicon.

So we’ve built two completely different labs off the same instinct, and we’re never going to fully agree on how much rope to give an AI agent. That’s fine by me. If two people who’ve worked infrastructure together for years across companies and countries can land in different places on this, then there’s clearly no single correct answer, just the one that lets you sleep at night. Mine happens to involve me reading the code before it runs, Ben’s involves a VLAN and a locked down box in a garage that hums like a nuclear reactor in summer. Both of them work, and that’s kind of the point.

Thanks for listening, and as always, keep on learning.