Everything Was Tested. Eventually.
Everything Was Tested. Eventually.
Before the move, my homelab was a handful of random networking equipment, an older virtualization host, a NAS, a desktop, and a collection of services running wherever I could find enough resources. It worked most days. Every new idea started with the same question: which machine can I make do one more thing?
Then we moved to Texas and bought a house (with pre-run 2x drop of CAT6E in every room). For the first time, I had the space to set whatever was coming up properly. I could have moved the old arrangement in and kept finding places to cram things.
Instead I thought, fuck it. Let’s see what we can do.
So I started where every good nerd does, with the network rack, then designed and built around a completly new, purpose-built server/NAS combo to consolidate compute and storage, mostly to give myself room for the persistent red team lab I’d wanted. The goal was to stop designing every project around the limits of whichever old box still had memory left.
That worked. Then I filled the room.
Today the environment has infrastructure I depend on, runs services I use, a local AI system I thought I’d be happy with, and the beginnings of a security lab I can attack without wondering whether I just took out something important. All of those jobs have different requirements, so the interesting part is deciding what gets to depend on what, what needs to be isolated, and what happens when any component decides to have a bad day.
The unsexy part, networking
We moved into the house, and the drops were already there. Dual CAT6E in every room, plus a UniFi Dream Wall already mounted from whoever had the place before us. A free head start, in theory.
Then I pulled out the cable tester and went drop by drop. Every port was labeled wrong. All of them, not just a few, and the panel had been lying with total confidence the whole time.
So before I touched a single service, I redid the labeling from scratch, documented what actually went where, and added a switch, UPS, and the old NAS on top of the Dream Wall for the homelab side of things. Getting the physical network and the rules running it into a state I actually trusted mattered, because everything that came after, the VLANs, the security posture, the AI stack I hadn’t built yet, was going to sit on top of whatever this layer turned out to be. I wasn’t interested in rebuilding the foundation nineteen times because I decided to take shortcuts.
The only thing that made the move from the old house was the NAS, and even that got demoted on arrival. It doesn’t do anything interesting now. It’s just the backup of the backup.
One server to rule them all, or so I thought
Proxmox. It gives me a place to run and rebuild services without turning the host itself into an unmanageable collection of applications. Bulk data lives on a ZFS pool. Faster storage handles the virtual machines and containers. I also set aside space for security lab work rather than treating intentionally breakable systems like normal household infrastructure.
The storage decisions came from the drives I had, not a perfect shopping list. Several showed errors before I built the array, so I chose dual parity and gave up usable capacity. That does not make the drives healthy. It gives me more room to deal with a failure while I work on slowly replacing them.
There’s a separate lesson in the hardware itself. This is still consumer equipment. I can give it careful storage, power protection, and recovery procedures. I cannot give it enterprise management features it does not have. When the host locks up hard, troubleshooting usually becomes a physical activity.
Some of this needs to work when I’m not experimenting
The lab runs media, photos, files, document collaboration, and other services that my family actually use. It also runs the supporting pieces: a reverse proxy, SSO identity for applications that support it, monitoring, and backups.
That changes how I make decisions. A disposable test VM can be wiped when an experiment gets ugly. A service holding real files needs a different answer. Before I reorganize storage or replace an application, I need to know where its data lives, how I’ll verify the move, and what I can restore if I was wrong.
I’ve already had a service stack come up looking brand-new after a directory change. The data was still there. The services were pointed at empty folders. During the fix, copying the old configuration over one of those folders would have overwritten data that had accumulated since the change. The recovery nearly became the actual data-loss event.
“Nothing is missing” was only true because I checked before moving it.
That’s why this environment has snapshots, backups, a second device receiving backup data, written recovery procedures, but no cold offsite storage, yet. It’s also why I no longer accept “the backup job completed” as proof that the things I care about were included. I recently checked the job against what was running and found it had missed about a third of the services, including the identity provider. The job was healthy. Green lights all around. Its coverage was shit. I only proved my long lasting thought that companies who “have” backups, but never test them and restore from them don’t really have shit.
I fixed the coverage. The useful part was finding out that a green light result can answer the wrong question.
Access had to be part of the design
There are services I want to use away from home. That doesn’t mean I want to put every application on the public internet. Remote access is constrained, and there’s a separate path for recovery if the primary one goes down with the main server. Backing up the backup 101.
The network is divided by purpose, but a label or checkbox does not prove isolation. I learned that again recently when an isolation setting did less than its name suggested, and an old allow rule took precedence over a block rule. I tested reachability in both directions, found the rule, removed it, and tested again.
I apparently created the rule at some point. I don’t remember doing it, but it’s an excellent reason why I test the network every time I built instead of trusting my memory.
I’m keeping the network description here at the level that explains the decisions. Later posts can get into specific problems and what I tested without turning this into a diagram of how to reach everything in my house.
The local AI setup has its own boundaries
The AI environment has grown from running an Ollama model on my laptop into something I use every single day. It runs on my own GPUs, and I’ve reworked and physically modified parts of the server as I figured out what worked and how I want to grow this part of my lab.
Running inference locally is the straightforward part of the security question. The interesting part starts when an assistant can use tools or touch another system.
I want useful capabilities with limited authority. This month handed me two live examples of what happens without it. An OpenAI agent hit a block on an Australian government Medicare portal, found a way around it, and pulled non-public files it had no business touching. OpenAI didn’t catch it themselves, they found out months later during an internal review and called it “actions we did not intend.” Anthropic’s own threat report, published this month, shows attackers handing Claude a general goal and letting it decide how to get there, for hours at a time, with nobody checking the work. Different companies, same problem. Nobody was watching what the agent actually did until after it had already done it.
If an agent needs to perform a specific task, I want its access scoped to that task, and that task alone. I then test what happens by trying to do something outside that limited scope. I also verify claims about what the assistant did. It can and will report a plausible result while getting the source wrong, and it will describe delegation in a way that sounds more capable than the hardware actually permits.
It has become useful infrastructure. It has not earned unsupervised access to everything else just because I built it.
And just like everything else, it’s forever changing and growing as the budget (and wife) allow.
The breakable part is still being built
The longer-term goal is an environment where I can build a target, attack it, collect telemetry, inspect what a defender could have seen, and rebuild it with a better understanding of what happened. That’s the part hosted exercises cannot give me in quite the same way. I have to own the systems underneath the exercise and live with my design choices afterward.
The foundation exists. The full offensive and defensive environment I want is still a project. I’m not going to describe planned targets, detection tools, and AI workflows as though they’re already running. They’ll get their own posts when there’s something real to test and write about.
The rest of the lab isn’t finished either. Monitoring needs broader coverage. AI needs a new home. Some recovery steps exist in a runbook but still need a full drill. Storage has known limits. Every new service gives me another reason to check whether backup coverage still matches reality.
We moved, bought a house, and I finally had room to find out what I could build. I said “fuck it, let’s see what we can do,” which seemed like a reasonable decision at the time.
Now I have a server doing a frankly unreasonable amount of work, real services with personal data, local AI with boundaries, a security environment under construction, and disaster recovery procedures for my little datacenter.
So. That’s what we could do. So far.
Sources
https://www.anthropic.com/threat-intelligence-report-september-2026
https://apnews.com/article/openai-government-website-incident-df331b55daffc6d202d8e2f6d0afa264