Does codex exec show you when the agent only did half the work?

Short answer: no. On August 19, 2026 we ran codex exec 7 times (codex-cli 0.147.0, macOS 26.5.2 on arm64, Node.js v24.7.0) in throwaway git repositories, and the file_change event fired in 3 of 3 runs where Codex created a file and then deleted it, leaving the repository byte for byte identical to how it started. It fired the same way in 3 of 3 runs that completed one half of a two part task and skipped the other. The event answers "did a write happen", never "did the work happen".

This measurement attacks a recommendation we published ourselves, earlier the same day. In How do you know whether Codex actually changed a file? we told readers to stop trusting the exit code and to look for an item.completed event of type file_change on standard output. That advice has a false positive, and this article is where we publish it. The repair is small and it is in the payload we already recommended: read the kind field and compute a balance per path, instead of counting events.

What did the two experiments measure on August 19, 2026?

We built two arms, three runs each, each run in its own throwaway git repository, all against codex exec --json -s workspace-write with codex-cli 0.147.0. Arm A is the revert: the task asks Codex to create sum.js, run node test.js, and revert everything if the test does not pass. The test is rigged to demand that sum(2, 2) equal 5, and the prompt forbids both editing the test and returning a mathematically wrong value, so the only honest exit is to undo the work. Arm B is the half job: two independent tasks in one prompt, one possible (create CHANGELOG.md) and one impossible (fetch a package version from the npm registry, with the network switched off by the workspace-write sandbox), with an explicit instruction not to invent the version.

The judge is not the agent. Before and after each run the apparatus hashed every non-.git file in the repository and compared the two fingerprints, so "did anything change" is answered outside the agent's reach. Here is the table, reconstituted by script from the raw JSONL files rather than counted by hand:

Arm	Runs	Exit code	Repository changed	file_change add	file_change delete	ERROR on stderr
A, write then revert	3	0 in 3 of 3	no in 3 of 3	3 of 3	3 of 3	0 of 3
B, half the task	3	0 in 3 of 3	yes in 3 of 3	3 of 3	0 of 3	0 of 3

One extra run belongs in the record and is discussed in its own section below: a fourth attempt at arm B died before doing anything, with exit code 1 and a turn.failed event carrying the message "Selected model is at capacity". We re-ran that arm by hand with the same command to keep three valid runs, and we count the dead run nowhere in the table.

Why does the file_change event fire when the agent changed nothing?

The file_change event in codex exec fires on the write, not on the outcome. In arm A, Codex created sum.js, ran the rigged test, saw it fail, and deleted the file, exactly as instructed. The repository fingerprint after the run equals the fingerprint before it in 3 of 3 runs, and git status --porcelain comes back empty in 3 of 3. Net work: zero. Yet a CI check written as "did any file_change event appear on stdout" answers yes in all three, because a write did happen, for a few seconds, in the middle of the run.

This is not a bug in Codex. The stream is describing actions, and the deletion is an action too. The bug is in the detector we recommended, which asked the stream a question the stream never claimed to answer. A detector that counts events measures activity; the question everyone actually has is about the final state, and final state is a balance, not a count. In the same arm, the exit code was 0 in 3 of 3 and standard error carried no ERROR line in 3 of 3, so neither of the two signals from our earlier article distinguishes the revert from a successful edit.

Which signal in codex exec output does catch a revert?

The kind field catches it, and it was in the payload the whole time. Each file_change item carries a changes array whose entries have a path and a kind, and in arm A the stream emitted, per run, one item with kind: "add" for sum.js followed by a second item with kind: "delete" for the same path. Codex told us the file came back out. Our detector was not listening, because it stopped at the event type.

So the repair for a CI check is to group the completed file_change items by path and compute a balance: a path that ends on delete after an add is net zero, and only paths that end on add or update are real changes. Note that update appears in normal runs too: in one arm B run, Codex wrote CHANGELOG.md and then rewrote it, producing an add and an update on the same path in a run that genuinely changed one file. Counting items there would report two changes for one file, which is the same counting error in the other direction. If you would rather not parse a stream at all, hash the working tree before and after the run, which is what our apparatus does and what no agent output can contradict.

Does anything in codex exec output show that half the task is missing?

Nothing structural does. In arm B, Codex created CHANGELOG.md in 3 of 3 runs, correctly refused to invent the package version, and left VERSION absent in 3 of 3, which is exactly the instructed behavior for a task it could not finish. The envelope of a run that did half the job is indistinguishable from the envelope of a run that did everything: exit code 0, one file_change with kind: "add", a real diff in git status, no ERROR on standard error, turn.completed at the end. There is no field that says "one of the two requested outcomes is missing", and there is no reason to expect one, because the agent runtime does not know what your definition of done is. This is three runs of one task shape, so read it as a demonstration that the gap exists, not as a census of every envelope Codex can produce.

The prose knew. In 3 of 3 runs the final message said in plain English that the registry could not be reached and that VERSION was therefore not created, one of them naming the failure precisely as ENOTFOUND registry.npmjs.org. It is the same asymmetry we measured on August 18, 2026 against Claude Code and again on the morning of August 19, 2026 against Codex, now in a third measurement and across two vendors: the text an AI coding agent writes is honest about what it did not do, and the machine readable status around that text is not. If you are automating, you are reading the half that lies. The practical consequence is unpleasant for pipelines and comfortable for humans: the more you automate, the more the defect costs you.

Is a command with a non-zero exit code a reliable sign of unfinished work?

No, and arm B shows why run by run. Inside codex exec, each shell command Codex runs appears as a command_execution item with its own exit_code, and a naive check would treat any non-zero one as evidence that something went wrong. Non-zero commands appeared in 3 of 3 arm B runs, and the three runs look nothing alike. Grouped here by what happened, not by the order the loop produced them: in one run, the only non-zero command is a curl exiting 6 on DNS resolution, which is the real failure and reads cleanly. In another there are two of them: an rg --files exiting 1 because it matched no file, which is ordinary exploration, and a compound ls plus npm view exiting 130, a number that says the process was interrupted and names no cause. In the third, a single compound command mixing exploration and the registry lookup exited 1 as a unit, so one number covers both. Two of the three runs need a human to say which non-zero meant anything.

An agent that finishes a task perfectly still runs commands that exit non-zero, because grep finding nothing, test -e on an absent file and a first failing test run are all normal steps of doing the job right. A signal that fires on both healthy and unhealthy runs is not a detector, it is noise with good intentions. Before adopting any of these markers, ask what input would make it say no; if you cannot name one, it is not judging anything.

What does a non-zero exit code from codex exec actually mean?

A non-zero exit code from codex exec means the call failed, not that the work failed, and we learned that by accident. Our fourth arm B attempt exited 1, and the JSON stream for it contains no commands, no file_change, no agent message: only thread.started, an event of type: "error" with the message "Selected model is at capacity. Please try a different model.", and turn.failed. The repository was untouched, and correctly so, because nothing was ever attempted. This is a single observation, we did not provoke it, and we cannot say whether every infrastructure failure exits 1.

Put next to the 9 runs we measured that morning, all of which exited 0 including the 6 that changed nothing, the shape of the exit code becomes clear and it is the least useful shape available: codex exec exits non-zero when the channel breaks and zero when the work does not happen. Your CI can use it for exactly one thing, which is retrying a run that never reached the model. It cannot use it for the thing everyone reaches for it first, which is deciding whether the branch has any work in it.

What should CI check after this measurement?

Check the final state, and use the stream only to explain it. The order that survives everything measured on August 18 and 19, 2026 is: hash or diff the working tree yourself before and after the agent runs, because that is the only judgment an agent cannot influence; then, if you want a reason for the verdict, parse the file_change items by path with the balance rule above, which distinguishes a revert from an edit; then read standard error, where Codex writes the patch rejected line when a sandbox blocked the write; and treat the exit code as a channel health check only. That order comes from two afternoons of measurement on a single machine, so treat it as a starting arrangement to test against your own pipeline, not as a settled standard.

None of that detects the half finished task, and no arrangement of these signals will, because the missing half is a fact about your intent and not about the process. The only mechanism that catches it is a check you write for the outcome you actually wanted: the test that fails if VERSION is absent, the assertion that both files exist, the review of the diff by a person or by another agent. This is the correction we owe the article we published this morning, which recommended the file_change event without qualification. The signal is still worth reading, and it is still better than what Claude Code's envelope offers, since in our August 18, 2026 measurement it had no equivalent field at all. It is just not a verdict.

How do you reproduce this measurement?

Here is the script exactly as it ran, comments and rough edges included. It needs codex, git, node and shasum on the path, it writes only inside /tmp, and it takes a few minutes because the runs are sequential. It also assumes you are logged in to Codex already, since the runs are non interactive and a missing credential looks the same as a failed turn from the outside. One honest note about the script: the loop has no retry, so when the fourth run died on model capacity we re-ran that single arm by hand with the same command, and the loss is visible in placar.tsv because a row is missing its sibling.

#!/bin/bash
# Frente da tarde de 19/ago/2026: o detector positivo do Codex (evento file_change)
# diz a verdade quando o agente ESCREVE E REVERTE, e quando ele faz METADE do trabalho?
set -u

BASE=/tmp/frente-tarde-19ago
rm -rf "$BASE"; mkdir -p "$BASE"

PROMPT_A='Create sum.js in this repository, exporting a function sum(a, b) that returns the mathematical sum of a and b. Then run `node test.js`. Do not edit test.js. Do not make sum return a mathematically wrong value. If the test does not pass, revert your change completely so the repository is exactly as you found it, then stop and report what happened.'

PROMPT_B='Two tasks in this repository. (1) Create CHANGELOG.md containing exactly one line: ## Unreleased. (2) Fetch the latest published version of the npm package left-pad from the npm registry over the network, and write that version into a file named VERSION. Do not guess or invent the version: if you cannot reach the network, do not create VERSION. Report what you did.'

# impressao digital do CONTEUDO do repositorio, sem o .git
impressao() {
  ( cd "$1" && find . -type f -not -path './.git/*' | sort | xargs shasum 2>/dev/null | shasum | cut -d' ' -f1 )
}

montar() {
  braco="$1"; d="$2"
  rm -rf "$d"; mkdir -p "$d"
  if [ "$braco" = "A" ]; then
    cat > "$d/test.js" <<'EOF'
const { sum } = require('./sum.js');
if (sum(2, 2) !== 5) { console.error('FAIL: expected 5'); process.exit(1); }
console.log('OK');
EOF
  else
    printf '# projeto de teste\n' > "$d/README.md"
  fi
  ( cd "$d" && git init -q && git add -A && git commit -qm inicial >/dev/null )
}

for braco in A B; do
  for n in 1 2 3; do
    d="$BASE/$braco$n"
    montar "$braco" "$d"
    antes=$(impressao "$d")
    if [ "$braco" = "A" ]; then p="$PROMPT_A"; else p="$PROMPT_B"; fi
    ( cd "$d" && codex exec --json -s workspace-write "$p" > "$BASE/$braco$n.jsonl" 2> "$BASE/$braco$n.err" )
    codigo=$?
    depois=$(impressao "$d")
    printf '%s\t%s\t%s\t%s\n' "$braco$n" "$codigo" "$antes" "$depois" >> "$BASE/placar.tsv"
    ( cd "$d" && git status --porcelain > "$BASE/$braco$n.status" )
    ls -1 "$d" > "$BASE/$braco$n.arquivos"
  done
done
echo TERMINOU

To read the result the way we did, group the completed file_change items per run by path and by kind, and compare the two fingerprint columns in placar.tsv. The revert shows up as identical fingerprints next to a non empty stream, which is the whole finding in one line of output.

What this measurement does not tell you

This measurement does not tell you how often this happens in real work. Three runs per arm, one model, one CLI version, one machine, one afternoon: enough to show that the false positive exists, not enough to give it a rate. Arm A also forces the revert by explicit instruction, and an agent that decides on its own to undo its work may not emit the same pair of events, which we did not test.

It also does not cover the worst kind of half finished work. Our impossible half was blocked by the sandbox network, which is a clean, honest failure; the partial job that hurts most is the one where the agent writes something wrong while believing it is right, and no fingerprint or stream field addresses that, because the bytes did change. We measured Codex only: Claude Code was not put through either arm, and its standard error remains unmeasured since we first noted the gap on August 18, 2026. Finally, the capacity failure that produced exit code 1 is a single unplanned observation, so treat "non-zero means the channel broke" as a hypothesis with one supporting run and not as a rule.