Waqar Uddin

Signs of life: the slow, cheap AI worm

October 16, 2026 (1d ago)7 views

What was published

On 12 October 2026, Anthropic published a research post by William Zhou and Daniel Freeman titled "AI models may be capable of self-replicating without access to a datacenter". The subject is what safety researchers call autonomous replication and adaptation, informally "AI worms": software that copies and spreads itself with no human in control.

Self-spreading code is not new. What is new, in their framing, is that the model weights travel as part of the payload. The agent does not phone home to a big server for its intelligence. It carries its own brain and runs on whatever hardware it lands on.

I want to summarise what they reported, separate it from what they predict, and spend most of my words on what it means for the people who defend networks and write rules for devices. I will keep this at the level of findings. Nothing here is a method, and the authors themselves avoid publishing one.

What is new: two lines converging

Read this first. This is a set of demonstrations under simplified, contained conditions. The replication test ran on eight emulated Android nodes in a firewalled lab, not on the public internet. The authors say plainly that there is no full rogue replicator today. Their timeline rests on a proxy capability index, so the date is a rough estimate, not a forecast to plan a budget around.

The core argument is geometric. A replicator has to clear a capability floor: it must be smart enough to work out an unfamiliar device and get a model running on it. It also has to stay under a size ceiling: it must be small enough to fit on the devices it targets. Big models clear the floor but do not fit. Small models fit but, so far, fall short of the floor.

The researchers report that those two constraints do not intersect today. Their claim is that small models are improving fast enough that they soon will, and they put "soon" at months rather than years. Their headline examples of targets are deliberately mundane: smart ovens and off-brand Android TV boxes.

They also describe the shape such a thing would have. It would be hard to shut down, because there is no central server to switch off. It would be slow, with a doubling time measured in days. And unlike conventional malware, which is written for known targets, it could reason its way through hardware it has never seen.

The demonstration

The flagship result is not about spreading at all. It is about capacity. The team ran a 26B parameter model across fifteen low-cost, off-brand TV boxes costing about 55 dollars each, and reported roughly 0.39 tokens per second. That is painfully slow, and it is the point: cheap consumer devices, pooled, can host a capable model.

A few reported details change how I think about the risk. The model does not need administrative privileges, because inference can run in ordinary user space. Low-security devices that restart can simply be re-infected. And the authors chose pipeline parallelism, splitting the model into stages across devices, for fault tolerance rather than speed.

Put together, weights as payload turns the familiar botnet into something different. A classic botnet is a set of hands waiting for orders. This is closer to a set of hands that brings its own judgement. Take away the command server and, in principle, it keeps thinking.

The four stages, and the one they studied

The authors break the cycle into four stages: finding targets, gaining execution, establishing inference, and continuing propagation from the new host. They set the first two, which together amount to intrusion, explicitly aside. That ground is already covered by prior published work, and they had no reason to add to it.

Their focus is stage three, which they call the least studied: preparing a machine to serve a model. They describe it as a long chain of fiddly, hardware-specific setup decisions where a failure early on is hard to recover from later. That is exactly the kind of work small models used to be bad at.

On this stage, the results are mixed in an instructive way. A 9B model set up inference on 8 of the 9 hardware types they tested. Models of 30B parameters or fewer could already complete many of the individual steps, but made mistakes stitching them together end to end. In simplified evaluations, GLM-5.2 and stronger models completed the full cycle, reaching 8 of 8 nodes in 4 of 5 trials.

The authors also describe a harness they call Memento. When an agent fixates on a wrong idea over a long task, it restarts a fresh agent with its predecessor's workspace and a short note of what was tried. Anyone who has watched a coding agent dig itself into a hole will recognise the problem it addresses.

The crossing point

This is the chart that carries the paper's timeline. The authors plot open-weight models of 30B parameters or fewer against a general capability index, by release date. The trend climbs about 13.7 index points a year, with an R-squared of 0.95. Projected forward, it reaches the level they think a replicator needs around the end of 2026.

I would hold this chart loosely, and so do the authors. A capability index is a summary of benchmarks, not a direct measure of "can set up inference on a strange box". A high R-squared tells you the trend has been steady, not that it will continue, and not that the threshold sits where they drew it. Move the dashed line up a few points and the date slides into 2027.

But the direction is hard to argue with. Small open models have improved steadily, and nobody is planning to stop making them better. The more useful question is not the exact month. It is whether defenders are ready for the window once it opens.

Adaptation without training

The part I found most interesting is the adaptation result. The authors let each generation of agent inherit its predecessor's notes, code and prompts. No weights change. They call this Lamarckian evolution, after the old idea that acquired traits pass to offspring.

With a 9B model, they report that setup success rose from 45% in the first generation to between 61% and 84% in later ones, purely from inheriting notes.

The second result here is a quiet surprise. Giving a small model access to a frontier model as an adviser produced no measurable improvement. The small models, the authors write, did not know the right questions to ask. Their reading is that close coordination, if it comes, is more likely to look like a large model setting strategy and directing small models as subagents, rather than small models phoning a clever friend.

For defenders, the inheritance finding matters more than the headline percentages. It means a population can get better at a specific job without anyone retraining anything. Every failure that leaves a useful note makes the next attempt a little more likely to succeed.

Scale, honestly stated

Here the authors do something I wish more safety writing did: they put a ceiling on their own scenario. They anchor on the Kimwolf botnet, which reached over 2 million devices, mostly Android TV boxes. If every group of fifteen boxes ran one copy of a 26B model, that gives about 133,000 copies, producing about 80,000 tokens per second in total.

Forty GPUs. That is a mid-sized research lab, not a superintelligence. The authors' own image is a slow, distributed "slime mould", not a fast takeover. Real throughput on the TV boxes was around 0.3 to 0.39 tokens per second. A full working context takes tens of hours to process. A single replication cycle for a 9B model can take one to two days.

They also look at the upper bound. Some edge accelerators can serve a whole model on one device, and there are on the order of 90 product lines and 9.5 million fielded units in that class. But the authors note that those devices are overwhelmingly reachable through outbound management tunnels rather than inbound ports, and that there has been no historical mass compromise of that class of device.

Slow replicators would be a bellwether for faster, more capable ones.

Paraphrasing the authors' conclusion, Anthropic, 12 October 2026

That, to me, is the real reason to care about a forty-GPU slime mould. It is not dangerous because it is powerful. It is worth watching because it is persistent, because it adapts, and because whatever detection and clean-up habits we build against it are the ones we will need for whatever comes next.

The authors list five threat models, and I reproduce them compactly because the list is useful for planning. Their stated worst case is a mix of them.

What it does not mean

It does not mean an AI worm exists. The authors say there is no full rogue replicator today. The end-to-end replication ran on emulated nodes behind a firewall. The intrusion stages were not studied. The timeline rests on a proxy. And even the scary version, as they size it, is slow and small. Their own phrase is "signs of life", which they compare to where coding agents were in 2024: clearly working in pieces, clumsy as a whole.

Their forward prediction, a world with agents running on all kinds of hardware, some outside human control, is labelled by the authors as going beyond their experiments. I take it seriously as a direction, not as a finding.

What defenders and regulators can do

The good news is that most of the useful responses are things we already know how to do. The authors frame the goal in epidemiological terms: drive the reproduction number below 1, so that each infection produces fewer than one new one and the replicator dies out rather than persisting. You do not need to catch every copy. You need to make spreading harder than dying.

From where I sit in Karachi, the last point is the one I would underline. Cheap, unbranded TV boxes and smart appliances are everywhere in our markets, often imported with no security baseline at all. Rules like the UK's are only as protective as the pool of devices they cover, and a lot of the world's pool sits outside them.

What I am watching

Sources

  1. William Zhou and Daniel Freeman, "AI models may be capable of self-replicating without access to a datacenter", Anthropic, 12 October 2026. All figures, numbers and threat models in this article are drawn from this post.
  2. Prior published work on the intrusion stages, which the authors set aside, is cited in the Anthropic post; see its reference list.
  3. The Mirai notification and quarantine figures (92% versus 74% within 14 days) are as reported in the Anthropic post, which draws on ISP clean-up research; see the post's references for the original study.
  4. The Kimwolf botnet figure (over 2 million devices) and the UK default-password rule are as stated in the Anthropic post.

Figures in this article are my own schematic redraws of reported results. Where the source gives a range rather than individual values, the figure shows the range.