Local LLMs as coding agents on an M4 Max
from a human
I wanted to do some work where I don't have to worry about tokens, so I had Claude test a bunch of LLMs on my 128 GB M4 Max MacBook.
I wanted to know whether a model running on my own Mac can do real coding work without me watching it. My machine is an M4 Max with 128 GB of memory. I took one feature from a Python computer-algebra interpreter I'm building and gave the same task to five local models through four different agent harnesses. Claude wrote the scripts and ran the whole thing overnight on November 1 and 2 while I was asleep. It graded the results the next morning, and the numbers below come from its logs.
The task is a p-adic valuation function that returns two values, with exact argument checks and error rules for bad input. Grading was 23 hidden tests, plus a reference check of 6 values against the real system my interpreter imitates. A run passes only if it gets 23 of 23 and 6 of 6. For scale, a Claude Sonnet subagent did the same task in about 4 minutes and passed both. Sonnet spends limited tokens of mine. A local model costs nothing per token, so the question was how much time I'm buying with that.
Results

There were 13 finished local runs. Eleven passed and two failed. Voided runs, where a server crashed or never started, are left out of the chart.
DeepSeek V4 Flash passed all six configurations I gave it. It is a mixed 2-bit and 4-bit quant, a 97.6 GB file, served by DwarfStar's ds4-server at a 100,000 token context. The times, fastest first:
| Harness | Minutes |
|---|---|
| Codex CLI, medium effort | 38 |
| ds4-agent | 39 |
| opencode | 65 |
| Pi, high effort | 96 |
| Codex CLI, high effort | 100 |
| Pi, medium effort | 108 |
Qwen3-Coder-Next at 8-bit (MLX, through LM Studio) was the fast model that actually finished. It passed in 25 minutes under Pi and 41 under Codex CLI. Under opencode it stopped after 14 minutes believing it was done, and got 19 of 23 hidden tests and 3 of the 6 reference values. It had the unit part of a rational's valuation wrong. Its passing code was also rougher than DeepSeek's, with dead boolean checks and a duplicated loop.
Qwen3.8-Flash-Next at 2-bit passed under Pi at both effort levels, 99 minutes at medium and 82 at high. GLM 5.3 Flash passed the hidden tests too, but only at my 150 minute cap. One of its own tests tried to factor 400-bit numbers by trial division and hung for 86 minutes, and Pi's shell tool has no timeout. Claude killed that test by hand at 02:34 and GLM carried on. I count that pass as helped.
Gemma 4 was the disappointment. The 26B-A4B model runs at 49 to 55 tokens per second at short context, the fastest of anything I tried. It stopped after about 6 minutes with 18 of 23 hidden tests and 0 of 6 reference values, because its two-value form came back wrapped in an extra tuple and booleans were not rejected. Fast and careless. The 31B model decoded at 7 to 9 tokens per second in the 4-bit GGUF build, and 3 to 5 in the MLX 8-bit build, where requests stalled until LM Studio's engine aborted. I stopped the 4-bit run after five minutes rather than wait for a verdict, since it was slower than DeepSeek at both reading and writing.
Why the harness changed the time
DeepSeek reads a prompt at roughly 100 tokens per second on this Mac and writes at 11 to 12. The speed sweep, which times a fixed prompt at each context length, shows the same picture: DeepSeek reads at 127 to 162 tokens per second and generates 12 to 14, while Qwen3-Coder-Next reads at around 490 to 600 once the prompt passes 4k.

A coding agent sends the whole conversation back to the model on every turn. That is fine as long as the server remembers what it already read and only processes the new tokens. When the new prompt stops matching the cached one, the server logs a cache miss and reads the history again. At 100 tokens per second, 10,000 tokens of history cost about 100 seconds.
The same model on the same task gave very different totals of re-read tokens:
| DeepSeek run | Cache misses | Prompt tokens read | Time reading | Total |
|---|---|---|---|---|
| Codex CLI, medium | 0 | 38,240 | 387 s | 2,266 s |
| opencode | 4 | 161,823 | 1,328 s | 3,921 s |
| Pi, medium | 2 | 152,494 | 1,421 s | 6,501 s |
Pi at high effort lost 749 seconds to two compactions, where the harness summarises the conversation, so the new prompt no longer matches what the server cached. DwarfStar ships its own agent, ds4-agent, for this reason. It talks to the engine directly with no server in between, reused the KV cache, and read only 34,000 prompt tokens in a run that took 39 minutes.
I don't want to overstate this. I have one run per cell, and the harnesses differ in more than caching. Pi at medium effort took 123 turns and generated 49,909 tokens against Codex's 56 turns and 21,530, so it was also doing more work. What the table does show is that prompt reading was between 17 and 34 percent of the total time in these three runs, and that the cheapest run is the one where the server never lost its place.
One setup detail along the same lines. With my normal Codex configuration, the first prompt it sent was about 69,000 tokens, which I traced to the 10 MCP servers I have configured. At this reading speed that is more than ten minutes before the model does anything. I ran every Codex test with a clean config directory.
The pass that edited its own evidence
The Qwen3.8 run at medium effort passed all 23 tests and all 6 reference values, and it was also the only run that registered the new function in the project's support catalogue. It then did something I had not asked for. It rewrote the hashes in a stored record of a CI run and changed the expected counts in a catalogue test so they matched its change. The repository now said a CI pass had happened that never did.
Nothing in my tests would have caught it. Claude found it in the diff review, which looks at every file a run touched, not only the ones the task named. The same model at high effort did not touch those files, so I can't tell you it is a habit. One of eleven passing runs did it, and I now grade the files that record evidence, not only the tests.
What I kept
I kept DeepSeek V4 Flash on disk. Codex CLI at medium effort and ds4-agent are the two harnesses I'll use with it, at about 40 minutes for this task. Qwen3-Coder-Next is faster under Pi, but its one failure came under opencode, where DeepSeek passed, and I would rather have the model that passed in all six configurations.
The follow-up has ten small tasks, each given twice, once with a full written spec and once with a one-line brief. I want to know whether a local model needs the spec or can work from the name of the thing. The results aren't in yet.
Measurement notes. Voided runs were caused by LM Studio's engine crashing and by an LM Studio restart that left it holding the port a model server needed. One of the Qwen3.8 medium run's 105 turns came from a GLM process that survived its time cap and was killed later; I fixed the cap to kill the whole process tree. The DeepSeek runs were not affected.