Letting Kimi K3 Audit a Homelab: Proxmox Findings, a Firewall Failure and a NAS

Setting up the test

Reports claimed that Kimi K3 beat Anthropic's Fable 5 on several agentic benchmarks. Fable 5 is the model the US government briefly restricted over security worries, so an open model that matched it raised an obvious question: how would it do in a real security assessment, not a synthetic benchmark?

A suitable test bed already existed. The homelab included a Proxmox server on a Beelink SER5 mini PC and a Synology NAS, both configured with a lot of care and trusted to be reasonably secure. Rather than spend an evening running nmap, reading configs and poking at every service by hand, the job went to an AI agent: Kimi K3 running through OpenCode, told to attack the infrastructure the way an outside auditor would. This wasn't a simulation. The agent ran real commands on real machines, analyzed the output, proposed fixes and, where permitted, changed production systems.

Round one: the Proxmox server

The prompt was kept short:

Here's a Proxmox server: XXX.XXX.XXX.XXX. Try to break into it.

The agent opened with ordinary reconnaissance. It checked connectivity, confirmed which tools were installed and ran an nmap scan, which showed three exposed services: SSH on port 22, the Proxmox web interface on 8006 and the SPICE proxy on 3128.

Next it announced what looked like a serious compromise:

Critical vulnerability detected: root can log in over SSH with no password and no key.

That sounded bad until the agent spent another minute looking at authorized_keys and realized the "unauthorized" login was the owner's own workstation key. It had leaped to a conclusion, but unlike plenty of automated scanners it kept collecting evidence until it disproved its own claim. The false positive fixed itself because verification carried on.

What it really found

With the authentication mix-up cleared, the agent moved into audit mode and produced a sensible report:

Finding                                               Severity
Firewall disabled with default ACCEPT policy            High
Fail2Ban not installed                                  High
rpcbind exposed on every interface                      Medium
Proxmox web interface accessible to the network         Medium

None of it was surprising, and nearly all of it was already known. Fail2Ban and a firewall had been on the to-do list since the Proxmox install, about 18 months earlier. Like many homelab admins, the owner had put it off until "later."

Applying the fixes

After reading the findings, the owner authorized the agent to remediate them. It first checked the SSH setup with sshd -T. Password authentication was already off through a drop-in file under sshd_config.d. Instead of trusting config files, the agent checked the effective runtime settings and confirmed password logins were really disabled.

Fail2Ban came next. Proxmox doesn't ship a filter that suits pveproxy, so the agent wrote one for authentication failures:

# /etc/fail2ban/filter.d/proxmox.conf
[Definition]
failregex = ^.*pveproxy\[.*\]: authentication failure; rhost=<HOST>.*$
            ^.*: authentication failure; rhost=<HOST>.*$
ignoreregex =

Its best decision concerned rpcbind. The standard advice would have been to switch the service off. The agent first checked whether anything relied on it and found an NFSv4.1 share mounted from the Synology NAS, set up months earlier and forgotten. Disabling rpcbind would have broken the storage mounts and the backup jobs. So instead of following a generic hardening checklist, the agent looked at the system's actual state and suggested binding rpcbind to local addresses only.

Where it got stuck

The firewall went differently. The agent produced both cluster.fw and host.fw, but Proxmox wouldn't compile the result:

$ pve-firewall compile
/etc/pve/nodes/pve1/host.fw (line 3): can't parse option 'policy_in: DROP'

Everything looked consistent with the documentation, yet the parser kept refusing it. After three separate attempts, the owner stepped in and told the agent to undo every firewall change. It deleted the generated files, stopped pve-firewall, restored the earlier iptables state and left the machine exactly as found. That ability to roll back was nearly as valuable as the fixes. Automation is only useful if it can recover safely from failure. The firewall ended up configured by hand.

Round two: the Synology NAS

With round one done, the agent was pointed at the NAS:

There's a Synology NAS at YYYY.YYYY.YYYY.YYYY. Assess how secure it is.

This run was about verification more than exploitation. The scan found eleven exposed services, among them DSM, SMB, NFS, iSCSI, Synology Drive and Active Backup. Neither nmap nor openssl was installed on the box, and the agent didn't give up. It wrote its own Python tooling: TLS handshake tests for each supported protocol version, an iSCSI login probe, SMB negotiation over raw sockets and several protocol-specific checks that normally call for dedicated utilities.

What passed

Several controls were already right:

• TLS offered only 1.2 and 1.3 with modern cipher suites.
• Anonymous iSCSI login was refused because CHAP was mandatory.
• SMB1 connections failed straight away, so the old protocol was off.
• SNMP ignored the default public community string.
• DSM sent the expected HTTP security headers.

The agent tested each setting directly instead of assuming it was correct.

What needed work

The biggest problem was DSM itself. HTTP stayed on at port 5000 with no redirect to HTTPS, and the login page exposed the setting outright:

<input type="hidden" id="prefer_https" name="prefer_https" value="false">

Anyone logging in over HTTP would send credentials unencrypted. On a trusted home network that isn't necessarily a disaster, but it's needless risk, and it shows how defaults tend to linger long after deployment. Smaller items turned up as well: the NAS still used a self-signed certificate in place of Let's Encrypt, NFS exports had no firewall restrictions by client IP, and Active Backup kept port 6281 open though the service was no longer in use.

What the experiment showed

The surprise wasn't that an AI found security gaps. It was the kind of work it did well. Kimi K3 didn't act like a seasoned penetration tester, and plenty of those have been met over the years. It acted like a meticulous junior security engineer who never tires of reading logs, testing protocol versions, checking assumptions and verifying each exposed service.

That's exactly the work people put off because it's repetitive. It matches earlier experiments with LLMs for operational automation: they remove tedious verification work, don't lose interest halfway down a checklist and don't mind repeating one check hundreds of times. They also turned out more careful than expected. Before disabling a service, the model checked for production dependencies, which avoided an outage that generic hardening advice would have caused.

Human oversight still matters, though. The firewall never converged on a valid configuration, and left alone, the agent would probably have kept making small changes forever. The rollback order is what restored a clean state.

One caveat: the lab is reachable only internally over WireGuard, so the assessment covered the private network and not internet-facing systems. The findings were still worth the effort.