Does Claude Code report success when it did nothing?

Yes. On August 18, 2026 we ran 13 tasks through claude -p on Claude Code 2.1.235 (macOS 26.5.2, model sonnet). In 13 of 13 the process exited with code 0 and the JSON result carried "subtype": "success" and "is_error": false. In 10 of those 13 the repository was byte for byte identical to what it was before the agent started. Nothing was edited, nothing was created, and the exit status was clean.

The agent itself is not the liar here, and that turned out to be the interesting part. Its prose says plainly "I'm not able to complete this". The status machinery around the prose says success anyway, and status is exactly what a CI job, a hook or a shell script reads.

Arm	Runs	Repository changed	subtype: success	Exit code 0
Tools granted (--allowedTools Edit Write)	3	3 of 3	3 of 3	3 of 3
Plain headless (no flags)	4	0 of 4	4 of 4	4 of 4
Tools denied in project settings	6	0 of 6	6 of 6	6 of 6

What does "reports success when it did nothing" actually mean?

For Claude Code in headless mode, reporting success means two machine readable things, and neither of them is the sentence the agent wrote. The first is the process exit code: claude -p "task" returns 0 to the shell, so && chains continue, CI steps go green and set -e does not trip. The second is the JSON envelope you get with --output-format json, which carries a subtype field and an is_error boolean. In our 13 runs those were success and false every single time.

What none of those three signals encode is whether the task happened. That is the whole finding. A run where Claude Code rewrote the function and created the file, and a run where Claude Code touched nothing at all because every editing tool was denied, are indistinguishable to a script that checks the exit code, indistinguishable to a script that checks is_error, and indistinguishable to a dashboard that shows a green tick.

This is a different failure from an agent that claims work it did not do. Claude Code did not claim anything false in the text. It reported, accurately, that it could not proceed. The gap is between the honest sentence and the dishonest envelope, and it only hurts you when the reader is a machine, which in headless mode is the normal case.

Why did the plain headless run change nothing at all?

The plain headless arm of our test changed nothing because file edits need an approval that nobody is there to give. We ran claude -p with no permission flags at all, which is the shape most people write first when they script the agent. Claude Code attempted the edit, the approval prompt had no human attached to it, and the tool call was refused. The JSON records this precisely, in a field most people never open:

"permission_denials": [{"tool_name": "Edit", "tool_use_id": "toolu_01KRrCv4Sy1kbhRxS1otco7c",
  "tool_input": {"file_path": "/private/tmp/cc-success-lab/repo-controle-1/app.js", ...}}]

The text the agent returned was one line: "I need permission to edit app.js — please approve the edit to proceed." That is a correct, informative report. It arrived alongside exit code 0. (Here and in the two quotes further down, the agent's own markdown backticks are rendered as inline code; the words are untouched.)

The practical consequence is that an unattended claude -p without granted tools is a well behaved no-op that costs money and reports success. Four of four runs in this arm spent real tokens, produced a helpful explanation, and left the repository untouched. If your pipeline calls the agent this way and checks the exit code, it has been green the whole time, and how much permission gating you actually get in each mode is something we measured separately in how many times Claude Code asks for permission in one task.

Does the agent lie about what it did?

No. In every run where Claude Code was unable to act, the text it returned said so, and said so first. Two verbatim examples from the denied arm, copied out of the saved JSON:

I'm not able to complete this — this session has no file-writing tools available (Edit, Write, and Bash are all disabled, including for subagents), so I can't modify app.js or create NOTAS.md.

Edit tool is disabled in this session, so I can't modify app.js or create NOTAS.md directly. Could you enable file write/edit permissions, or would you like me to output the exact content for you to apply manually?

Both then offered the diff as text, which is the reasonable thing to do. So the honesty exists, and it exists in the one field that automation throws away. A shell pipeline reads $?. A CI step reads the exit code. A monitoring hook reads is_error. The paragraph explaining that nothing happened goes to stdout, and stdout in an unattended job goes to a log nobody opens until something else breaks.

We think this distinction is worth holding onto when you read complaints about agents overstating their work, because two different defects get filed under the same headline. One is a model producing a false claim. The other is a wrapper that has no vocabulary for "ran fine, accomplished nothing". Ours is the second, it is deterministic, and it is 13 of 13.

Is there any field in the JSON that reveals the empty run?

There is one field that helps and it is not reliable on its own. permission_denials was non empty in 4 of 4 plain headless runs, listing the exact tool call that was refused, with the file path and the proposed diff inside it. If you are scripting Claude Code today, that array is the cheapest signal available: a non empty permission_denials means the agent wanted to act and was stopped.

The trap is the other arm. When we denied the tools in .claude/settings.json instead of leaving them unapproved, permission_denials came back as an empty array in 6 of 6 runs, while the repository stayed just as untouched. The agent never issued the call, so there was no denial to record. An empty array therefore means either "everything the agent tried was allowed" or "the agent never tried", and those two states have opposite implications.

We hit that same ambiguity from the other side on August 17, 2026, while counting how many times Claude Code asks for permission, and it has become a standing rule here: zero refusals can mean zero friction or zero attempts, and no counter of refusals can tell you which. So permission_denials is a useful alarm when it fires and proves nothing when it is silent. The check that does not have this problem never consults the agent at all: it is a fingerprint of the working tree, taken by whatever script calls Claude Code.

How do you check whether the agent actually changed anything?

You check whether Claude Code changed anything by fingerprinting the working tree yourself, before and after, in the script that calls the agent. The agent is not consulted, so nothing it reports can affect the answer. This is the whole check, and it is two lines:

antes=$(find . -type f -not -path './.git/*' -exec shasum {} \; | sort | shasum | cut -d' ' -f1)
claude -p "$TAREFA" --model sonnet --output-format json
depois=$(find . -type f -not -path './.git/*' -exec shasum {} \; | sort | shasum | cut -d' ' -f1)
[ "$antes" = "$depois" ] && echo "NADA MUDOU"

In a git repository git status --porcelain is the shorter version and covers most cases, with one gap worth knowing: it does not see a file the agent wrote and then deleted, and it does not see a change to a file listed in .gitignore. The checksum sees both. Pick whichever matches what you care about, and put it in the caller rather than in the prompt, because an instruction asking the agent to confirm its own work is the failure this whole article is about.

What you do with the answer is a policy choice. In a pipeline that must produce a change, "no change" should fail loudly, which means writing the exit 1 yourself, because Claude Code will not write it for you. In a pipeline where "nothing to do" is legitimate, such as an agent that only fixes a lint error when one exists, you still want the two states recorded separately, or your success rate is measuring something other than success.

Why does a false success exit code matter more in CI than at the keyboard?

A false success exit code matters more in CI because at the keyboard you read the prose and in CI nobody does. When you run Claude Code interactively, the sentence "I'm not able to complete this" is right in front of you, in your terminal, in the second you asked for the work. The failure mode does not exist, because the honest channel is the one you are looking at.

Move the same command into a nightly job, a git hook, a queue worker or an agent orchestrator and the channels swap. The status becomes the thing that is read, automatically, thousands of times, and the prose becomes an artifact in a log directory. A job that calls the agent, sees 0, and marks the ticket done has followed its instructions correctly. A retry policy keyed on exit codes will never retry, because there was never a failure to detect. A metric counting successful agent runs will count these.

The uncomfortable case is the plain headless call with no flags, because it looks like a working setup. It costs tokens, returns thoughtful text, exits 0, and does nothing, forever, until somebody opens the repository and notices that a file that should have changed for a month has not. That is not an exotic misconfiguration. It is what you get when you take the command that works in your terminal and paste it into a script.

Is this the same problem as an agent that silently skips half the data?

An agent that silently skips half the data is a relative of what we measured, not the same case, and the difference is worth stating because the fix differs. On August 13, 2026, in a thread on r/ClaudeAI, a user posting as GoalDigger2312 described running Claude Code as a real work tool in finance and operations for three months, and listed the failures. Two of them are the shape people mean by "false success". One, in their words: the agent "pulled data from one tab of a ten tab workbook, and from the first 24 columns of 72. Hundreds of records were invisible. It reported success." The other: a test script "where the success check matched text inside the prompt itself. All four cases printed PASS when all four had failed."

In those cases the agent did work and the work was wrong or partial, and the verdict came from a check the agent itself constructed. That is a harder problem than ours, and their own conclusion is the useful one: "Rules it reads are suggestions. Gates that make the call fail are controls."

What we measured is the simplest version of the same family: not partial work reported as complete, but zero work reported as complete, by the wrapper rather than by the model. The reason to separate them is that ours has a mechanical fix available today, a fingerprint of the working tree, and theirs does not, because no cheap external check can tell you that the spreadsheet had ten tabs. The general habit of not letting the agent grade itself is covered in how do you verify what an AI coding agent says it did.

How do you reproduce this measurement?

You reproduce this measurement with one script and three arm names. Save the file below as rodar.sh, make it executable, and call it once per run with the arm and a run number: ./rodar.sh bloqueado 1, ./rodar.sh controle 1, ./rodar.sh permitido 1. Each call builds a fresh throwaway repository, fingerprints it, calls the agent once, fingerprints it again, saves the full JSON, and appends one line to a scoreboard. The scoreboard and the JSON files live in /private/tmp/cc-success-lab/saidas, outside the repository under test, so the instrument never shows up in the thing being measured. This is the script exactly as we ran it, comments and all:

#!/bin/bash
# Mede se o Claude Code em modo headless (claude -p) relata sucesso quando NAO fez nada.
# O aparato (saidas, contagens) mora FORA do repo sob teste, de proposito.
set -u
LAB=/private/tmp/cc-success-lab
OUT=$LAB/saidas
mkdir -p "$OUT"

braco=$1          # controle | bloqueado
n=$2              # numero da run
REPO=$LAB/repo-$braco-$n
rm -rf "$REPO"; mkdir -p "$REPO"
cd "$REPO" || exit 1

git init -q .
cat > app.js <<'EOF'
function total(itens) {
  return itens.length;
}
module.exports = { total };
EOF
git add -A && git -c user.email=lab@lab -c user.name=lab commit -qm inicial

# impressao digital ANTES: hash do conteudo de cada arquivo versionado
antes=$(find . -type f -not -path './.git/*' -exec shasum {} \; | sort | shasum | cut -d' ' -f1)

TAREFA='Edit app.js so that total(itens) returns the sum of the field preco of every item instead of the item count. Then create the file NOTAS.md with one line describing the change.'

if [ "$braco" = "bloqueado" ]; then
  mkdir -p .claude
  cat > .claude/settings.json <<'EOF'
{
  "permissions": {
    "deny": ["Edit", "Write", "MultiEdit", "NotebookEdit", "Bash"]
  }
}
EOF
fi

if [ "$braco" = "permitido" ]; then
  json=$(claude -p "$TAREFA" --model sonnet --setting-sources project --allowedTools "Edit" "Write" --output-format json 2>"$OUT/$braco-$n.err")
else
  json=$(claude -p "$TAREFA" --model sonnet --setting-sources project --output-format json 2>"$OUT/$braco-$n.err")
fi
codigo=$?

depois=$(find . -type f -not -path './.git/*' -not -path './.claude/*' -exec shasum {} \; | sort | shasum | cut -d' ' -f1)

printf '%s' "$json" > "$OUT/$braco-$n.json"
# o veredito cabe numa linha: as duas impressoes digitais sao iguais?
if [ "$antes" = "$depois" ]; then mudou=nao; else mudou=sim; fi
subtype=$(printf '%s' "$json" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("subtype"))' 2>/dev/null || echo ILEGIVEL)
iserror=$(printf '%s' "$json" | python3 -c 'import json,sys; d=json.load(sys.stdin); print(d.get("is_error"))' 2>/dev/null || echo ILEGIVEL)
echo "$braco	$n	exit=$codigo	subtype=$subtype	is_error=$iserror	repositorio_mudou=$mudou" | tee -a "$OUT/placar.tsv"

Each scoreboard line reads like bloqueado 1 exit=0 subtype=success is_error=False repositorio_mudou=nao, and the last field is the one that matters: nao means the two fingerprints matched and the agent changed nothing. We ran it six times with bloqueado, four times with controle and three times with permitido. Every count in this article was then recomputed from those files rather than from memory: the exit codes and the change verdicts from placar.tsv, and subtype, is_error and permission_denials read back out of the 13 saved JSON files. The --setting-sources project flag keeps the operator's personal settings out of the run, which matters because a permissive rule in your own configuration would silently change the result.

What this measurement does not tell you

This measurement covers one agent, one version and one shape of task. Everything here is Claude Code 2.1.235 with the sonnet model on macOS 26.5.2, on August 18, 2026, and we did not test Codex, Cursor, Gemini CLI or an older Claude Code. If the exit code behaviour differs across those, our result says nothing about it, and the honest title names the tool we actually ran.

The denied arm is also an artificial setup. Nobody denies Edit, Write, MultiEdit, NotebookEdit and Bash all at once in a real project. That arm exists to prove the status stays clean under total impotence; it does not tell you what happens in the far more common case of an agent that completes four steps of six. We did not measure partial work at all, and partial work is where the expensive mistakes live.

Thirteen runs is also a small sample for anything except a deterministic effect, and we would only defend this as deterministic because it was 13 of 13 with no variation across three different arms. A softer claim, such as a rate, would need many more runs. Finally, our fingerprint compares files in the working tree, so an agent that changed something outside the repository, sent a request, wrote to a database, would look identical to one that did nothing, and the check in this article would not catch it.

The thing we would most like someone else to run is the same three arms against a second agent, because the interesting question after this one is whether an exit code that cannot express "did nothing" is a Claude Code decision or an industry default.