Response recovery for non-idempotent Notecard requests

[m-mcgowan]

Problem

Consider this situation:

1. host sends a non-idempotent request to the Notecard over serial or I2C (e.g. web.post)
2. the Notecard processes it successfully, but
3. the response is lost in transit (CRC mismatch, I2C read failure, serial framing error, intra-byte timeout etc..)

The host is left in an unrecoverable state:

• Retrying is unsafe — the action already happened (e.g. a Note was added, a web.post was dispatched)
• Not retrying may lose data — the host has no confirmation the request succeeded
• There is no way to re-fetch just the response — the Notecard protocol has no concept of "resend last response"

Affected requests

Some non-idempotent requests are truly unrecoverable — there is no after-the-fact query that can confirm whether the request was processed:

Request	Impact of blind retry
note.add	Duplicate Note — no way to check if a specific Note was already added
hub.log	Duplicate log entry — fire-and-forget by design
web.post / web	Duplicate HTTP request — side effect is external to Notecard

Others have workarounds involving extra round-trips, which add complexity, latency, and their own transport failure risk:

Request	Impact of blind retry	Workaround
card.binary.put	Binary data written twice or corrupted	Check buffer state with card.binary — extra round-trip
card.binary.get	Data consumed on read	Check buffer state — but data is gone if consumed
note.get (with delete)	Note consumed on read	Query the Notefile — but the Note is gone
note.changes (with delete)	Change tracker reset	No way to undo the tracker advance

The serial/I2C transport is inherently unreliable at the physical layer. CRC detection catches corrupted responses, but detection without recovery just converts a silent error into an explicit one with no better outcome.

How HTTP REST APIs solve this

The standard pattern is idempotency keys (popularized by Stripe, adopted by AWS, Google Cloud, etc.):

1. Client generates a unique key and sends it with the request (Idempotency-Key: <uuid>)
2. Server checks if it has seen this key before
3. If yes → returns the cached response without re-executing
4. If no → executes the request, caches the response, returns it

Retries with the same key are always safe, regardless of the request's inherent idempotency.

Proposed solution

The Notecard request format already supports an "id" field. This can be extended to enable response recovery:

1. Client sends a monotonically incrementing "id" with non-idempotent requests:

   {"req":"note.add","id":42,"file":"sensors.qo","body":{"temp":22.5}}

2. Notecard processes the request normally and caches {id, response} for the most recent request.

3. On response loss, the client retries with the same "id":

   {"req":"note.add","id":42,"file":"sensors.qo","body":{"temp":22.5}}

4. Notecard recognizes the repeated ID → returns the cached response without re-executing.

5. On a new request, the client increments the ID. The Notecard can discard the previous cache entry.

Why this is lightweight

• Only one response needs to be cached — serial/I2C is a single-threaded, sequential protocol. There is never more than one request in flight.
• The ID is a simple integer counter, not a UUID. The Notecard only needs to compare against the single most-recent ID.
• Backward compatible — requests without "id" behave exactly as today. The feature activates only when the client opts in.

Client library integration

The client library (e.g. note-c, note-python) can automate this:

• Track a per-transport sequence counter
• Auto-attach "id" to requests where Safety is NonIdempotent or Destructive (or all requests for simplicity)
• On ResponseLost errors, automatically retry with the same ID
• Expose the retry/recovery behavior transparently — the caller sees either success or a confirmed failure

Alternatives considered

• Application-level deduplication (e.g. check if Note was added before retrying): Pushes complexity to every application developer. Doesn't work for web.post where the side effect is external, or note.add where there's no dedup key.
• Sticky response buffer (Notecard keeps last response available for re-read): Similar to this proposal but without the ID match, so it can't distinguish "re-read last response" from "this is a new request that happens to be identical."
• Per-request workarounds (check state after suspected failure): Workable for some requests (card.binary.put → check card.binary), but adds round-trips with their own failure risk, and impossible for truly non-idempotent requests like note.add.