Evaluating and Benchmarking Pentest Agents
If you can't measure an offensive agent repeatably, you're doing demos, not engineering. Our harness: success rate, cost per flag and scope adherence.
Tag
lang-en
gVisor vs seccomp-bpf: Sandboxing Offensive Tools
Letting an agent run exploits demands serious isolation. When seccomp-bpf is enough, and when we stack gVisor on top.
Persistent Memory for Offensive Agents
A pentest does not fit in a context window. How we give our agents operational memory that survives across phases without leaking scope.
Graph RAG and MITRE ATT&CK
Flat RAG retrieves paragraphs; an operation thinks in relationships. We wired Beorn to an ATT&CK knowledge graph to decide the next move.
ReAct Prompting for Kill Chain Orchestration
Why a single LLM cannot run an entire pentest end-to-end, and how we extended the Thought-Action-Observation loop to coordinate agents in Gandalf CLI.
Seccomp-bpf for Autonomous Agents
How we built minimal seccomp-bpf profiles so that the exploits an LLM runs don't turn into an accidental rm -rf on the host.
Observability in Offensive Operations
Why a red team without traces is indefensible: how we instrument every decision of our agent with OpenTelemetry, eBPF and spans mapped to MITRE ATT&CK.
Secure Tool Calling in Air-Gapped Environments
How a kill-switch, a seccomp-bpf filter, and CIDR rules cut off the silent leak of an LLM agent in a lab with no internet. Lessons from the field.
Reinforcement Learning for Exploit Generation
We train a PPO agent to turn crashes into control flow hijacking. Rewards with eBPF, honest failures and real code. What we learned along the way.
Tactical RAG: From Writeups to Action
Indexing 9115 HTB writeups isn't building a search engine: it's giving operational memory to an agent in the middle of an exploit. Here's what we learned.
Multi-Agent Debate for Vulnerability Triage
Why we make three agents (critical, evidential, and technical) debate each finding before closing it, with real metrics and trade-offs.
LLM-Guided Fuzzing: More Coverage, Fewer Silly Crashes
How we combined AFL++ with LLM-generated seeds in Gwaihir CLI to fuzz complex parsers without drowning in initial validation crashes.