Agent evaluation

lamm-mit/

SwarmWorld

A deterministic research environment where societies of LLM agents discover, exchange, and physically embody technologies across simulation ticks.

What’s new here

SwarmWorld studies technological evolution at the society level, not the agent level. Agents perceive locally, deposit persistent artifacts, and pass executable programs to successors. The simulator is deterministic: traces can be replayed and state digests verified for compatible engine revisions. Counterfactual replays can remove individual contributors to isolate their causal role.

What it does

The Python simulator runs societies of LLM agents (or scripted policies) in a shared 2-D world. Each agent observes only its local neighborhood, acts through a bounded contract, and can create artifacts whose effects continue on subsequent ticks. A PettingZoo interface exposes the environment to external policies.

The repo includes:

  • A biofoundry CLI for simulating, replaying, serving, and analyzing runs
  • OpenAI-compatible LLM policy support, plus open-weight serving via mistral.rs or vLLM
  • A study runner that generates paired condition/seed traces and writes the protocol before execution
  • Analysis commands for scaling, cultural gain, spatial mobility, artifact knockouts, and technology atlases
  • A Three.js Observatory for live and seekable playback, an optional Godot client, and a human-in-the-swarm game server
  • Declarative scenario packages (the default biological world and the Ashen Realms example)

The approximately 8.4 GB paper dataset, including authoritative traces and final figures, is released separately on Hugging Face.

Who it’s for

Researchers studying collective intelligence, cultural evolution, or emergent behavior in multi-agent systems. It suits teams who want a reproducible computational substrate: independent simulation seeds are the unit of replication, and traces ship with integrity checks. It also fits anyone who wants to run LLM agents through a persistent shared world.

Try it

Requires Python 3.10 or newer (Python 3.12 is the reference environment).

git clone https://github.com/lamm-mit/SwarmWorld.git
cd SwarmWorld
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ".[dev,analysis]"
biofoundry doctor --config configs/demo.yaml

Run a no-key deterministic simulation:

biofoundry simulate \
  --config configs/demo.yaml \
  --policy scripted \
  --ticks 128 \
  --output runs/demo.jsonl
 
biofoundry replay runs/demo.jsonl

No model is downloaded or contacted by configs/demo.yaml. API keys are read only from the environment variable named by the selected YAML profile.

How mature is it

Created and last pushed September 2026. 21 stars, 7 forks, 1 contributor, 4 commits in the past 90 days. No releases tagged. One open issue and one open pull request. Licensed Apache-2.0. The repo is a clean source release accompanying a preprint (arXiv 2608.26081); the bulk of the scientific data lives in a separate Hugging Face dataset.