- Qwen 3.8 27B (a 17GB version) running locally identified Pac-Man, Pong and Breakout from raw weights alone, and failed on Freeway.
- Runs took 46m 12s to 1h 4m 54s and used 43,506 reasoning tokens on the Pac-Man run. Kimi K3.1 Pro via OpenRouter solved Breakout in 9m 41s.
- The test comes from a paper on 'Weight-Space Programming'. Critics say it mixes two separate questions, and Simon is not ready to call it a big shift.
What happened
Simon Willison reproduced one experiment from the paper 'Sparks of Weight-Space Programming' using Qwen 3.8 27B on his own laptop. The paper reports that frontier models, GPT-6 Astra in particular, can read and modify the raw weights of a machine learning model. The test here is the no-tools part: give a model the weights of a small network trained with reinforcement learning on a grid game, and ask which game it was.
Qwen got Pac-Man (72% stated confidence, 46m 12s), Pong (75%, 1h 4m 54s) and Breakout (88%, 52m 25s) right. It failed Freeway and guessed a Pac-Man-like game. Simon says that is not surprising since Freeway is more obscure. For comparison, Kimi K3.1 Pro through OpenRouter guessed Breakout after 9m 41s and 19,493 reasoning tokens.
Simon also notes the paper mixes two topics: what models can do with raw weights without tools, and what they can do once given code. spaCy author Matthew Honnibal called the paper misleading for conflating the two. Simon says he is not ready to declare the age of source code over.
What this means for founders
- No product use yet. This is a research demo. Nothing in the source shows a workflow a founder can ship from it.
- Local runs are slow. Each Qwen run took roughly 46 minutes to over an hour, versus under 10 minutes for Kimi K3.1 Pro on a hosted API. Budget your time accordingly.
- Local cost is hardware, not tokens. The model version Simon uses is 17GB and runs in LM Studio. Exact laptop specs are not given in the source, so check your own machine before planning around it.
- Setup has traps. The default 8192 context failed with errors, and Simon had to raise the timeout to 900 seconds. Expect some tuning.
- Be careful with the hype. Qwen missed one of four games, and the paper's wider claims are disputed. Do not build plans on 'models edit their own weights'.
How to try it today
- Install LM Studio and load the Qwen 3.8 27B build Simon uses (about 17GB). Set a long context window, because the default 8192 failed.
- Install the
llmtool and thellm-lmstudioplugin. - Run one of the paper's prompts straight from GitHub with a longer timeout:
LMSTUDIO_TIMEOUT=900 \
llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/pacman/prompt.txt \
-m lmstudio/qwen/qwen3.8-27bExpect to wait 45 minutes or more. To compare with a hosted model, install llm-openrouter and swap the model to openrouter/moonshotai/kimi-k3.1-pro.
Related tools on Launch Llama
Tools from the Launch Llama directory that fit with this news:
| Tool | What it does | Why it matters here |
|---|---|---|
| What's my local ai? | For AI engineers: What's my local ai? detects your hardware and recommends local models so you run LLMs privately without cloud costs. | Detects your hardware and recommends local models, useful before you try a 17GB model like this one on your laptop. |
| LLMxRay | See what your local AI is actually doing, step by step | Lets you inspect reasoning chains from local models on Ollama, which is the kind of long trace this experiment produced. |
| Qwen3 | For developers: Qwen3 is an open-weight multimodal agent with 397B capacity but 17B active parameters for fast inference. | A Qwen3 listing in our directory with open weights, though it is a different model from Qwen 3.8 27B and not confirmed to be the same one. |
Browse all tools or list your own tool.
Llama's take
I would try this if you already run local models and want to see how far a 27B model can go on a hard reasoning task. It is a fun test and the prompts are public, so the setup cost is low. The result is real: three of four games identified, with no tools.
If you are shipping a product this week, wait. The runs take close to an hour, one game failed, and the source does not show a practical use. The paper's broader claims are also contested by people who know the field.
The thing to watch is whether the paper's two questions get separated and tested properly: what models can do with raw weights alone, versus what they can do with code. That split decides if any of this matters for builders.
FAQ
Can Qwen 3.8 27B run on a laptop?
Yes. Simon Willison runs a 17GB version of it in LM Studio on his laptop. He had to use a long context window, since the default 8192 failed, and raise the timeout to 900 seconds.
How well did Qwen 3.8 27B identify games from raw weights?
It correctly identified Pac-Man, Pong and Breakout, with stated confidence of 72%, 75% and 88%. It failed Freeway and guessed a Pac-Man-like game.
How long did the Qwen 3.8 27B runs take?
Pac-Man took 46m 12s with 43,506 reasoning tokens, Breakout took 52m 25s, and Pong took 1h 4m 54s. Kimi K3.1 Pro via OpenRouter solved Breakout in 9m 41s.
Sources
- Qwen 3.8 27B on my laptop can interpret model weights, too (simonwillison.net)
- Hacker News discussion