An agent tool call is made idempotent by the deterministic shell around the model, never by the model itself. A deterministic shell is the non-model code that decides whether an effect happens — it fixes the key, writes the ledger, calls the tool, and records the outcome, and the model can neither see it nor override it. The model never mints the key and never gets to decide that a retry is a good idea.
Where does the key come from?
From the attempt, not from the call site. Attempt identity is the tuple that names one intended effect — run id, step index, tool name, argument digest — fixed and durably recorded before the effect is attempted, and unchanged by every retry of that attempt.
Popular advice says the mistake is a random UUID. Popular advice is wrong. The expired IETF draft recommends one, the Agentic Commerce Protocol from OpenAI and Stripe marks its required: true header "UUID v4 recommended", and AWS's sample calls randomUUID() inside a step, warning that "a key generated outside a step changes on replay, which defeats deduplication and doubles up on retry".
What separates a checkpointed UUID from a broken one is where the value gets pinned. Temporal derives instead: workflowRunId + '-' + activityId, constant across retries, unique across workflows. Random is fine. Recomputed on the retry path is fatal. Cite RFC 9562 for the UUID, not 4122: UUIDv7's embedded timestamp makes pruning trivial, and pruning is on you. Stripe prunes at 24 hours; the draft sets no lifetime at all.
The idempotency header every payments API ships has no standards-track home. draft-ietf-httpapi-idempotency-key-header expired at version -07 on 18 April 2026 after seven revisions and was never published as an RFC.
Why not just hash the tool name and arguments?
Because a content hash cannot tell a retry from a person who genuinely wants the second reminder sent. Add the step index, a monotonic position in the plan, so the key names an attempt rather than a wish.
Content-derived keys have a nastier problem: low entropy makes them guessable. The IETF draft's security considerations warn that attackers "MAY determine other keys and use them to fetch existing idempotent cache entries, belonging to other clients". Hash an invoice number and an amount and your dedupe store is an oracle. Compose the lookup key as the expired IETF draft and the Agentic Commerce Protocol both do: a high-entropy caller value, scoped to authenticated identity and endpoint, which the resource combines with server-side attributes the caller never sees. On your own side of the wire, HMAC the tuple with a secret so nobody guesses a key from the arguments.
Scope belongs to the key's identity. An EC2 client token is idempotent per Region, so the same token replayed in another Region launches a second instance by design.
Is exactly-once achievable?
No, and yes, and the distinction is the whole discipline. Exactly-once delivery is impossible over an unreliable channel: Two Generals, Akkoyunlu, Ekanadham and Huber, 1975, popularised by Jim Gray in 1978. Exactly-once effect is routine. Kafka's exactly-once semantics are real and bounded strictly to Kafka-to-Kafka; they do not reach your outbound SMS.
Exactly-once delivery is impossible over an unreliable channel. Exactly-once effect is ordinary engineering: at-least-once delivery into a receiver that refuses to act twice.
Which layer is actually retrying?
Five layers can repeat a side effect, and the layer everyone blames cannot.
| Layer | What it repeats | Default | Cost of a duplicate |
|---|---|---|---|
| Anthropic Python SDK | POST /v1/messages | DEFAULT_MAX_RETRIES = 2 on 408/409/429/5xx | Tokens and latency. No side effect. |
| LangGraph node retry | The node, tool call included | retry_policy = None until you opt in; then 3 attempts, 0.5s, ×2 backoff, 128s cap | Whatever the tool does. Again. |
Your loop around tool_use | The tool handler | Whatever you wrote | Whatever the tool does. Again. |
| The model | Nothing; it re-decides | Not configurable. Ambiguity reads as "it did not happen" | A second, sincere email. No attempt-derived key stops it; only a content index in the same ledger notices. |
| Proxy or transport | The HTTP request | RFC 9110: a proxy MUST NOT auto-retry non-idempotent requests | A silent duplicate you do not own |
An idempotency key protects you from the same call being transmitted twice. It does nothing about a model that reads an ambiguous result and freshly decides to send the email again — that is a new intent, and it will produce a new key.
The fix for a model that re-decides is state, not transport. Hand it "already sent, message id X", because an error string reads to a language model as permission to try harder.
How do agent tool calls duplicate in production?
In recognisable, nameable ways. Unnamed failure modes never get budget.
- Key regenerated on the retry path. Computed at call time, so attempt two carries a different value and the dedupe store never matches.
- The poisoned key. Stripe caches the first request's status and body "regardless of whether it succeeds or fails", so a stored 500 replays as a 500 for the life of the key and a blind retry loop burns the run's budget against a dead value. The
Idempotent-Replayed: trueheader exists so you can tell replay from reality. - Unknown treated as failed. A timeout is an absence of information. Marking it failed and retrying is how one intent becomes two effects.
- The unrecognised exception. LangGraph's
default_retry_ondenies a list of programmer errors and ends with a fallbackreturn True, so the moment you attach a retry policy, a custom exception from your tool is retried three times with backoff. - Retrying a retry. RFC 9110, STD 97 rules it out in one line: "A client SHOULD NOT automatically retry a failed automatic retry." The proxy version of that sentence is a MUST NOT.
- The metering loophole. Idempotence "only applies to what has been requested by the user", so a conformant server may still meter and bill every attempt.
What if the tool has no key support?
Then choose which failure you prefer, because no safe answer exists. Temporal names three honest options: execute at most once and accept that zero executions becomes possible; write a compensating action; or read back before treating the call as complete. Read-back narrows the window and never closes it, because another actor may have changed state between your failed write and your read.
Does MCP handle any of this?
No. MCP supplies the vocabulary and none of the mechanism. idempotentHint exists in the tools specification, defaults to false, and means anything only when readOnlyHint is false. The same spec tells clients to "consider tool annotations to be untrusted unless they come from trusted servers", so idempotentHint is a self-declaration the spec tells clients not to believe.
Below the hint, nothing. tools/call carries no key and no replay semantics. Streamable HTTP resumability covers server-to-client SSE replay after a Last-Event-ID and says nothing about a re-POSTed tools/call, and a reconnecting client re-issuing a call is indistinguishable from new intent. The commerce protocol's own MCP binding smuggles the value in as meta.idempotency_key, downgraded from required in REST to optional over MCP.
What does the shell look like?
Small, boring, and outside the model's reach. The key is the stable prefix; the argument digest rides in the ledger row, so a changed body against a fixed key surfaces as a 422 rather than a silent overwrite. The ledger doubles as the trail that answers did we send it, when, and on whose instruction: raw material for an evidence layer.
THE DETERMINISTIC SHELL — the non-model code that decides whether an effect happens
The Hopium Lab · v1.0 · 21 July 2026 · at-least-once delivery, at-most-once effect
# Attempt identity is the tuple that names one intended effect — run id,
# step index, tool name, argument digest — fixed and durably recorded before
# the effect is attempted, and unchanged by every retry of that attempt.
def call_effectful_tool(run_id, step_index, tool, args):
key = hmac(server_secret, f"{run_id}:{step_index}") # stable prefix only
claim = ledger.claim_if_absent(key, tool, digest(args)) # atomic; IN_FLIGHT
# leases into UNKNOWN
if claim.state == SUCCEEDED: return definitive(claim.result)
if claim.state == TERMINAL_FAILURE: return stop(claim.error) # poisoned
if claim.state == IN_FLIGHT: raise Backoff(claim.retry_after) # 409
if claim.state == BODY_MISMATCH: return stop(claim.error) # 422
if claim.state == UNKNOWN: return escalate(key) # a human decides
try:
out = tool.invoke(args, idempotency_key=key, sdk_retries=0, timeout=T)
except (Timeout, TransientError): # 5xx, reset connection, killed process
ledger.mark(key, UNKNOWN) # unknown is NOT failed; it may have landed
return escalate(key)
if out.replayed and out.failed: # Idempotent-Replayed: true
ledger.mark(key, TERMINAL_FAILURE, out.error)
return stop(out)
ledger.record(key, out) # durable BEFORE the model sees it
return definitive(out)
PRE-FLIGHT — before an agent touches anything irreversible
[ ] Key fixed before the call, stable across every retry
[ ] Key HMAC'd with a server secret, scoped to identity, endpoint, region
[ ] Ledger write atomic, and before the effect
[ ] IN_FLIGHT claims leased, expiring into UNKNOWN rather than limbo
[ ] Replayed failures separated from transient ones, both escalated
[ ] Timeouts land in UNKNOWN, never in FAILED
[ ] Exactly one layer retries: SDK retries off on write paths
[ ] A human, not the model, resolves anything left in UNKNOWN
Whether a step needs the shell at all is the agent or process question: most steps that move money or messages should never have been agentic. When one genuinely is, the model proposes the effect and the shell owns whether it happened.
The model is allowed to be wrong about whether the email was sent. The shell is not.
Ross Jones, Founder, The Hopium Lab. Last modified 21 July 2026. Engineering commentary, not legal advice.