LLMs with cyber toolkits can conduct multistage cyber operations on business-sized computer networks

Anthropic (with Carnegie Mellon University’s CyLab)

Large Language Models (LLMs) that are not fine-tuned for cybersecurity can
succeed in multistage attacks on networks with dozens of hosts when equipped with a novel toolkit. This
shows one pathway by which
LLMs could reduce barriers to entry for complex cyber attacks while also automating current cyber
defensive workflows. Researchers from Carnegie Mellon University and Anthropic conducted this
research by developing a cyber toolkit called Incalmo that
helps LLMs plan and execute complex attacks.^[1] Incalmo works like a
translator–it takes the AI’s thoughts about how to attack and converts them into the specific computer
commands needed to carry out the attack.

• [2]

Figure 1: Without Incalmo, none of the tested LLMs realized an end-to-end
multistage attack in any of the ten environments, and only Claude Sonnet 3.5 was able to exfiltrate a
single file in the 4-layer chain environment.

Figure 2: With Incalmo, LLMs can successfully and autonomously conduct
multi-stage attacks in nine out of ten environments ranging from 25 to 50 hosts.

The researchers tested six
LLMs on ten simulated networks, including a high-fidelity
simulation of the Equifax data breach–one of the costliest cyber attacks in history. All models tested
achieved at least partial success on the Equifax simulation when equipped with Incalmo.

• Colonial Pipeline
  attack

Figure 3: Schematic depiction of the difference between unaided LLMs and
Incalmo.

These results show how LLMs could lower the barriers to conducting complex
cyber attacks, underscoring the importance of investing in research into LLM capabilities for both
attack and defense. Normal scaling up of LLMs, improvement of tools like Incalmo, and the
potential for cyber fine tuning are all vectors for these capabilities to develop rapidly. This is an active
area of research for us.

For additional details see the full research paper (Singer et al.
2025).