PoCForge: A Closed-Loop Multi-Agent Framework for Autonomous Exploit Synthesis and Verification

Penetration testing remains a critical yet resource-intensive task in modern cybersecurity practice, as traditional approaches heavily rely on expert-driven manual workflows that are difficult to scale across diverse and rapidly evolving systems. Recent advances in large language models (LLMs) and multi-agent systems have opened new opportunities for automating complex security assessment processes. However, existing LLM-based pentesting frameworks remain fundamentally limited by their reliance on either static public Proofs-of-Concept (PoC) or open-loop reasoning pipelines that lack executable validation and adaptive refinement. In this paper, we propose PoCForge, an exploit-centric multi-agent framework that reformulates automated penetration testing as a closed-loop process of exploit synthesis, execution, and verification. The framework coordinates specialized agents across reconnaissance, vulnerability analysis, exploit synthesis, and validation, with reasoning explicitly structured around executable exploit artifacts. To organize exploit generation when a public Proof-of-Concept may or may not be retrieved, PoCForge introduces a three-tier knowledge hierarchy that integrates reusable exploit templates, retrieval-augmented external knowledge, and curated domain expertise, with a fallback mechanism for cases where no suitable public PoC is retrieved. The proposed system is evaluated on two widely used benchmarks, CVE-Bench and AutoPenBench, covering real-world vulnerabilities deployed in containerized environments. Experimental results based on 5 repeated runs per target, demonstrate that PoCForge achieves the highest exploitation success rate among the evaluated systems, with statistically significant improvements over both baselines. On CVE-Bench and AutoPenBench, PoCForge achieves exploitation success rates of 51.0\% and 89.1\%, respectively, significantly outperforming PentestAgent and VulnBot, both of which remain below 10\% on CVE-Bench and 32.7\% on AutoPenBench. These findings demonstrate the effectiveness of multi-agent orchestration, guided reasoning, and runtime validation for practical autonomous penetration testing within the benchmarked containerized environments.