From bd3e112a4e171ff8ec4233570669f9e86016a70c Mon Sep 17 00:00:00 2001 From: Nate Moore Date: Fri, 9 Oct 2026 09:57:57 -0500 Subject: [PATCH 1/4] spec: add image element specification MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Focused feature spec for the void img() directive: a term-level image registry (setImage/removeImage), a fidelity ladder of kitty graphics, shape-aware ascii art, and alt text, resolved deterministically from capability evidence. Kitty tier: lazy chunked RGBA transmission with delete-before- re-transmit, cursor-placed letterbox-free placements via same-pair replacement, scoped lowercase deletions for stale placements, data-freeing uppercase removal, immediate cleanup bytes from update() on capability denial and resize, and transmission-state resets grounded in the protocol's storage-quota rules. Ascii tier: per-character shape vectors over six staggered internal sampling circles with ten external circles, per-component max normalization, directional-then-global contrast enhancement, and 95-character nearest-match glyph selection, all baked at build time and golden-locked. Memory: no-malloc arena discipline — fixed-capacity entry table, compaction-only pixel pool, placements table, and O(1) per-frame scratch, carved at Term creation with closed RangeError failures. Includes amendments appendix (A1-A10), testing obligations, and open decisions. Synthesized via a three-candidate design arena; protocol facts verified against the kitty graphics protocol documentation. --- specs/image-spec.md | 1528 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1528 insertions(+) create mode 100644 specs/image-spec.md diff --git a/specs/image-spec.md b/specs/image-spec.md new file mode 100644 index 0000000..b4ab9ad --- /dev/null +++ b/specs/image-spec.md @@ -0,0 +1,1528 @@ +# Clayterm Image Specification + +**Version:** 0.1 (draft) **Status:** Proposed. Normative for the image element +upon adoption. + +--- + +## 1. Purpose + +This specification defines Clayterm's image element: a declarative, void +directive that renders a caller-supplied raster image into the terminal, +degrading through a fidelity ladder of output tiers — Kitty graphics protocol +pixels, shape-aware ASCII character art, and alt text. + +The image element fulfills the commitment made in +[Renderer Specification](renderer-spec.md) §7.8, which defers "Kitty graphics +emission" to a focused feature specification. It consumes the terminfo +foundation (PR stack #131–#133): tier selection reads the `RuntimeCapabilities` +snapshot that `term.update()` maintains, and the kitty tier is gated on the +`kitty-graphics` capability probe (query 9). + +The division of labor follows the renderer's founding invariants: + +- **The caller owns pixels.** The renderer performs no IO and decodes no file + formats. The caller decodes an image by any means available and hands the + renderer decoded RGBA pixels through a Term-level registry. The renderer + transcodes those pixels into whatever the current terminal can display. +- **The renderer owns bytes.** Tier selection, scaling, protocol encoding, + placement, cleanup, and cell-buffer integration are renderer concerns. The + caller declares *what* to show and *where*; the renderer decides *how*. +- **The renderer owns no memory allocator.** All image state lives in regions + carved once at Term creation, under the same fixed-capacity discipline as the + cell buffers (Section 5). + +## 2. Scope + +### In scope (normative) + +- The `img()` directive: its shape, its void (self-closing) form, and its + validation rules +- The image registry: `term.setImage()` and `term.removeImage()` +- The no-malloc memory discipline for image state: the carved registry, pixel + pool, and placements table, including failure behavior at capacity +- Tier resolution: the fallback ladder and its deterministic resolution from + capability evidence and directive properties +- The emission contract for each v1 tier: kitty, ascii, alt +- Cell-buffer integration: covered-cell representation, diff interaction, + damage, and cleanup +- Capability-change invalidation rules +- Line-mode interaction and the error surface +- Testing obligations + +### In scope (non-normative, descriptive) + +- Placement-id derivation (requirements normative; the hash itself is + implementation-defined, §9.2.3) +- The transfer encoding of the new directive (current implementation surface; + §15) +- Exact sampling-circle geometry and contrast exponents (structure normative; + values calibration-pinned by golden tests, §9.3.4) +- Glyph-lookup acceleration and per-frame memoization (§15) + +### Out of scope (v1) + +- The sixel tier. Its design sketch and required capability evidence are in + §16.1. The v1 ladder is kitty → ascii → alt. +- iTerm2 inline images (OSC 1337 `File=`) +- Animation (multi-frame formats, kitty animation placeholders) +- Kitty unicode-placeholder placements (§16.4) +- Cell-size geometry evidence (XTWINOPS 14/16) and pixel-space scaling (§16.3) +- Z-layering between graphics and overlapping text or floating overlays beyond + the §10 damage rule +- Behavior when an image's cells scroll within a scroll region (no caller-facing + scroll API exists; renderer-spec §14) +- Transitions on image properties +- Cropping, source rectangles, and sub-image placement + +### Out of scope (indefinitely) + +- Image decoding (PNG, JPEG, WebP, …) inside the renderer. The renderer + consumes decoded RGBA pixels only (INV-I1). +- File paths, URLs, or any IO-bearing reference in a directive. Pixels enter + through the registry or not at all. +- Screen-reader presentation of the alt tier. The alt tier is a visual + fallback rendered as ordinary cells; accessibility-tree semantics are a + higher-level framework concern. + +## 3. Terminology + +**Image element (`img` directive).** A void (self-closing) directive that +participates in layout as a leaf element, is a pointer hit-test target, is +reported in `RenderInfo`, and has no children. By analogy with HTML's ``. + +**Registry.** The per-Term store of decoded images. Callers put pixel data in +with `setImage()` and take it out with `removeImage()`, keyed by caller-chosen +integer ids. The registry is a resource store, not UI tree state. + +**Registry id.** A caller-chosen integer in `[1, 4294967295]` identifying one +image in one Term's registry. It is transmitted verbatim as the Kitty graphics +image id (`i=`), whose valid range is identical — the protocol requires +positive ids up to 4294967295 and forbids zero (§9.2.1). + +**Image data.** Decoded raster pixels: width, height, and a `Uint8Array` of +straight (non-premultiplied) RGBA8 bytes, top-to-bottom rows, exactly +`width * height * 4` bytes. + +**Tier.** One of the output forms an image element may take: `kitty` (Kitty +graphics protocol pixels), `ascii` (shape-aware character art in the cell +grid), `alt` (alt text as plain cell text). + +**Ladder.** The fidelity-ordered tier sequence used for automatic resolution: +kitty, then ascii, then alt. + +**Evidence.** A capability fact (see +[Terminfo Specification](terminfo-spec.md)) that gates a tier. The v1 gate for +the kitty tier is `RuntimeCapabilities.kittyGraphics === true`. + +**Layout box.** The element's computed bounding box in character cells, as +produced by the layout engine (the `ElementInfo.bounds` value, renderer-spec +§12.3). + +**Painted footprint.** The subset of the layout box that the selected tier +actually paints. For the kitty and ascii tiers, the aspect-fit region of §8.2; +for the alt tier, the whole layout box. + +**Half-row pixel convention.** This specification's source-pixel geometry +convention: one source pixel is one cell column wide and one half-row tall, so +one cell is two pixels tall and the cell aspect ratio `A = cell_height / +cell_width = 2` (height:width 2:1). The convention assumes a typical terminal +cell; §16.3 refines it with cell-size geometry evidence when that evidence +exists. + +**Contain.** Scaling semantics fixed for v1: the image is rendered into the +largest rectangle within its layout box that preserves its aspect ratio, centered +on both axes (§8.2). There is no stretch mode in v1 (§16.7). + +**Placement.** One terminal-side display of one kitty image at one cell box, +identified on the wire by the pair (image id, placement id). Placements persist +on the terminal until deleted or overwritten by text. + +**Placements table.** The renderer's fixed-capacity record of the previous +frame's live kitty placements (§5.4). It is retained diff state, not UI tree +state. + +**Damage.** Any event that makes an image element's previous emission stale: +registry replacement, footprint change, tier change, removal, or a capability +change. + +**Dirty-marking.** Forcing the next render's diff to emit bytes for a cell +region. The mechanism is implementation detail; the described approach is +overwriting front-buffer cells with the default cell value (§10.4). + +**Quiet emission.** Emitting protocol bytes in a mode that provably generates +no terminal responses, so the input parser's `scan()` stream stays clean +(INV-I5; input-spec §6.2 is the consumer that must not be disturbed). + +**Sampling circles.** Fixed cell-relative regions — six internal, ten external — +over which both candidate characters and source-image cells are measured +(§9.3.2). + +**Shape vector.** A per-character vector recording how the character's ink +distributes across the internal sampling circles. Computed at build time and +compiled into the module (§9.3.2). This is a fixed table of fractions, not a +font system: the renderer holds no font machinery of any kind at runtime. + +**Sampling vector.** The per-cell counterpart: luma means sampled from the +source image over the same circles, plus luma means over the external circles +for boundary awareness (§9.3.1). + +**Contrast enhancement.** The pre-matching transform on sampling vectors — +directional, then global — that sharpens region boundaries in the art +(§9.3.4). + +## 4. Architectural Model + +_This section is normative._ + +### 4.1 Relationship to the frame-snapshot model + +The directive array continues to fully describe each frame's UI tree. Pixels +are not part of the tree; they are resources bound to the Term through the +registry, referenced by id from `img` directives — the same relationship +dimensions and capabilities have to the Term. + +The renderer retains the following state between render transactions, all of it +implementation detail invisible to the caller except through reduced output +size and correct cleanup: + +1. The front/back cell buffers used for diffing (pre-existing). +2. The image registry: caller-uploaded pixel data, keyed by registry id (§5.3). +3. The **placements table**: a fixed-capacity record of the previous frame's + live kitty placements — for each, its registry id, wire image id, placement + id, painted footprint box, tier, and transmission state. It exists so that + cleanup and tier switching can be computed without scanning cells and + without storing element-id strings. It is updated during each render + transaction exactly as the front buffer is (§5.4). + +Items 2 and 3 extend what renderer-spec §4.3 enumerates as retained state; the +corresponding amendment to INV-3 is listed in Appendix A. None of this state +is a UI tree: no parent pointers, no component identity, no reconciliation, no +cross-frame directive memory. The placements table is derived from and bounded +by the previous frame's directives, exactly like the cell buffer. This +preserves the spirit of renderer-spec INV-3 and §10.4 (no cross-frame identity) +while admitting the resource-and-record state that graphics cleanup requires. +Consequently, tier selection, footprint computation, and all emission decisions +are pure functions of the directive array, the capability snapshot, the +registry state, the placements table, and the front cell buffer — no ambient +state, no clock, and no heuristic beyond the evidence table (§9.1) and the +damage rules (§10). + +### 4.2 Data flow + +``` +caller decodes file ──▶ term.setImage(id, {width, height, pixels}) + │ + ▼ (copied into the carved pixel pool; no IO) +img directives ──▶ render() ─▶ tier resolution (§9.1) + │ + ┌───────────────┼──────────────────┐ + ▼ ▼ ▼ + kitty tier ascii tier alt tier + delete/tx/place shape-vector art plain cell text + (APC graphics) in the grid (wrapped) + │ │ │ + └───────────────┴──────────────────┘ + ▼ + ANSI bytes (write now) +``` + +- `setImage()` performs no IO and emits no bytes. It stores pixels and, when it + replaces a live id, bumps the entry's data version (§4.3). +- `removeImage()` returns cleanup bytes where the terminal state requires them + and dirty-marks previously covered cells for the next render (§7.2). +- The first render transaction that places an image at the kitty tier + transmits its data. Transmission is keyed by per-entry transmission state + (§9.2.2), so unchanged, still-persisted data is never re-sent. +- All graphics emission happens inside the normal render transaction's byte + output. There are no side-channel writes (renderer-spec INV-1, INV-2). + +### 4.3 Registry lifecycle + +- `setImage(id, data)` inserts or replaces. Replacing a live id bumps its data + version. The pixels are copied into the carved pixel pool; the caller's + buffer is not retained (§5.3). Storage failure raises `RangeError` (§7.2). +- `removeImage(id)` removes the registry entry, returns kitty deletion bytes + when terminal-side state may exist (§7.2), and dirty-marks every cell in + every footprint that referenced the id so the next render repaints them. + Removing an unknown id is a no-op returning empty bytes. +- **Terminal-side persistence.** Kitty images and placements persist on the + terminal after the Term that placed them is discarded. Clayterm has no Term + disposal callback (renderer-spec §7.4), so images that outlive their Term are + the caller's to clean up — for example by deleting placements before + teardown, or by exiting the alternate screen. This matches the existing + boundary: the renderer emits bytes, the caller owns terminal state + (renderer-spec §11.2). + +## 5. Memory Model + +### 5.1 The no-malloc discipline + +_This section is normative for behavior._ + +The renderer runs freestanding (wasm32) with no heap allocator. `malloc`, +`free`, and any heap-based dynamic structure MUST NOT be used by the renderer +for image state. All image storage and bookkeeping MUST fit the Term's +fixed-capacity memory plan: + +- The Term's linear memory is carved **once at creation** into fixed regions + sized by a pre-computed size query over the requested dimensions and options + (§5.2). Region carving is descriptive; the behavioral requirements are + normative. +- Every image-side structure is a **fixed-capacity array or a bump-carved + extent within a pre-sized region**. There is no dynamic allocation, no + free-and-reallocate of per-image memory, and no per-frame scratch region. +- When a fixed-capacity structure is exhausted, the behavior is the closed + failure behavior of §7.2 and §13 — never silent truncation and never + eviction of caller-owned resources (INV-I2). +- Reclamation of image pixel bytes happens only by **compaction of the pixel + pool** (§5.3): dead extents are reclaimed by copying live entries to the + pool start and resetting the bump pointer. Compaction is deterministic, + order-preserving for live entries, and triggered by `setImage` on demand. +- All image state lives on the Term's C-side state struct (pointers into the + carve), never in global mutable state. Multiple Terms in one process are + fully isolated. +- Resize (renderer-spec §7.7) re-carves the dimension-dependent regions (cell + buffers, placements table) and grows linear memory if required. The registry + and pixel pool are **independent of terminal dimensions** and MUST survive a + resize unchanged: an image registered before a resize remains registered and + renderable after it. The placements table is discarded by resize together + with the other diff state, with the consequence documented in §10.5. +- `setImage` and `removeImage` MUST NOT grow linear memory and MUST NOT create + a new WASM module or instance. The reason is view safety: growing linear + memory detaches `Uint8Array` views over it, and output views are known-valid + only until the next `render()` or `update()` call (renderer-spec §7.3) — a + registry-triggered grow would silently detach output views outside those + transactions, where nothing in the contract permits it. Memory growth is a + Term-creation and resize concern only (renderer-spec §7.7 discipline). + +### 5.2 Carved regions (descriptive) + +``` +linear memory at Term creation: +┌────────────────────────────────────────────────────────────┐ +│ Clayterm struct │ +│ front cells (width × height cells) │ +│ back cells (width × height cells) │ +│ output buffer (capacity, §5.6) │ +│ Clay arena (fixed capacity; bump) │ +│ image entry table (IMAGE_ENTRY_CAP × entry, §5.3) │ +│ image pixel pool (imagePoolBytes, §5.3) │ +│ placements table (PLACEMENT_CAP × entry, §5.4) │ +└────────────────────────────────────────────────────────────┘ +``` + +The size query computes each region's capacity before the single carve. The +layout, ordering, and padding of regions are implementation surface. + +### 5.3 Image storage + +_This subsection is normative for behavior._ + +The registry is a fixed-capacity entry table plus a shared pixel pool. + +- `IMAGE_ENTRY_CAP` is 256 entries (normative). Each entry holds: registry id, + data version, width, height, pool offset, payload byte length, transmission + state (§9.2.2), and an in-use flag. +- `imagePoolBytes` is a `createTerm` option (Appendix A7) with normative + default `4194304` (4 MiB). The pixel pool is one contiguous region of that + size. +- `setImage(id, data)` copies the payload into the pool: the required + `width*height*4` bytes are bump-carved from the pool. If the pool's + remaining space cannot hold the payload, the renderer first **compacts** + (copies all live payloads to the pool start in offset order, resets the bump + pointer, and rewrites live entries' offsets) and retries. If the payload + still does not fit, `setImage` throws `RangeError` identifying the capacity + condition (§7.2). +- Replacing a live id allocates the new payload and marks the old extent dead; + the dead extent is reclaimed at the next compaction. Until then the pool + holds both payloads. This is observable only through earlier capacity + exhaustion and is accepted behavior. +- An entry's data version is a monotonically increasing counter, incremented + on every replace. It is the deduplication key for transmission (§9.2.2) and + the memoization key for ascii art (§15). +- The renderer MUST NOT evict, expire, or garbage-collect live entries + (INV-I2). Removal is explicit via `removeImage`, or by the Term being + discarded. + +### 5.4 Placements table + +_This subsection is normative for behavior._ + +- `PLACEMENT_CAP` is 64 entries (normative). Each entry holds: registry id, + wire image id, placement id, the painted footprint box, the resolved tier, + and the data version last transmitted. +- At the end of every render transaction, the table is rewritten to describe + exactly the kitty placements that transaction emitted, in previous-frame + directive order — the same retained-state discipline as the front buffer. +- When a frame's kitty placements exceed the capacity, the excess image + elements are **degraded to the ascii tier for that frame** in directive + order, and one `"IMAGE_PLACEMENTS_EXCEEDED"` error is surfaced (§13). No + kitty placement is ever created without a table entry: cleanup correctness + (INV-I3) outranks fidelity. +- The table stores no element-id strings. Placement identity is the wire pair + (image id, placement id) derived per §9.2.3. + +### 5.5 Per-frame scratch + +_This section is normative for behavior._ + +There is no per-frame scratch region. Every per-frame image computation is +either: + +- **O(1) working state** (stack locals): base64 encoder state, per-cell + sampling accumulators (six internal + ten external luma means), contrast + vector components, and the footprint arithmetic; or +- **Direct output**: base64 encoding streams into the carved output buffer at + the current write cursor; ascii sampling writes directly into the back cell + buffer; per-cell work never buffers more than one cell. + +The renderer MUST NOT build per-frame intermediate buffers for image +processing. This is what keeps the ascii pipeline inside the no-malloc +discipline: sampling, matching, and cell emission all stream. + +### 5.6 Output-buffer sizing for transmission + +_This section is normative for behavior._ + +The output buffer's capacity MUST be at least the sum of: + +1. the foundation's text-redraw capacity (the capacity required by an + ordinary complete redraw at the Term's dimensions), and +2. the worst-case single-frame graphics encoding: the base64 encoding of the + largest payload the pixel pool can hold — `ceil(4/3 × imagePoolBytes)` + bytes — plus, for every 3072 payload bytes (one maximum chunk), at most 128 + bytes of control-block framing. + +Formula: `output_capacity ≥ text_capacity + ⌈4/3 · imagePoolBytes⌉ + 128 · +⌈imagePoolBytes / 3072⌉ + 64` (the final 64 bytes cover the trailing +placement and cursor bytes). If a render transaction nevertheless exceeds the +output capacity, the renderer MUST throw a `RangeError` identifying the +condition as an output-capacity overflow, in the manner of renderer-spec §9.2 +— never a raw host-level TypedArray error. + +## 6. Core Invariants + +_This section is normative._ + +**INV-I1. Pixels are data, not IO.** Image pixels enter the renderer only +through `setImage()`. The renderer MUST NOT read files, decode compressed +formats, fetch, or otherwise perform IO to obtain image content. The directive +carries a registry id, never a path. + +**INV-I2. Caller-owned resources.** The registry holds exactly what the caller +stored, keyed by caller-chosen ids. The renderer MUST NOT evict, expire, +shadow, or garbage-collect live registry entries, and MUST NOT invent images. +Removal is explicit. Capacity exhaustion fails loudly per §7.2/§13. + +**INV-I3. Stateless cleanup.** All graphics cleanup — deleting stale +placements on move, resize, unmount, tier change, data replacement, or image +removal — MUST be computable from the current directive array plus the +retained diff state (cell buffers, registry, placements table). No callbacks, +no cross-frame UI tree, no imperative delete API beyond `removeImage()`. + +**INV-I4. Determinism and measurement agreement.** Tier selection and +footprint computation are pure functions of the directive, the capability +snapshot, the registry state, and the layout box. The painted footprint +computed for layout purposes MUST equal the footprint actually painted, for +every tier, on every capability state. An image MUST NOT claim cells it does +not paint, nor paint cells it does not claim. + +**INV-I5. Quiet emission and parser independence.** Every graphics-protocol +control block the renderer emits MUST set the quiet flag (`q=2`), which +suppresses both OK and failure responses. Rendered output MUST NOT cause the +input parser's `scan()` stream to receive probe-like or protocol-ack responses +(input-spec §6.2 defines what the parser recognizes). This preserves the +renderer and input parser's architectural independence (renderer-spec INV-7): +the renderer consumes capability facts, and nothing it emits is shaped to be +parsed by the input parser. + +**INV-I6. Cell-buffer honesty.** While an image covers cells at a graphics +tier, the cell buffer's content for those cells MUST be the covered-cell value +(§10.1) and MUST NOT contain other elements' content: pixel-tier coverage wins +over text (§10.3). When coverage appears, moves, or disappears, the renderer +MUST delete stale graphics state terminal-side and restore the affected cells +so that, after the caller writes the transaction's bytes, the terminal's +visible state matches the cell buffer. + +**INV-I7. Transaction boundaries preserved.** `setImage()` and `removeImage()` +are registry operations outside the render transaction. Each render +transaction remains a single TS→WASM crossing (renderer-spec INV-2). Registry +calls MAY cross the boundary individually, but they MUST NOT render, and +render MUST NOT require a preceding registry call beyond what its directives +reference. + +**INV-I8. No-malloc arena discipline.** All renderer-owned image state — +registry, pixel pool, placements table, and any per-frame working memory — +MUST satisfy §5.1: fixed capacities carved at Term creation, bump-carved +extents with compaction-only reclamation, O(1) per-frame scratch, and closed +failure behavior at capacity. The renderer MUST NOT link or invoke a heap +allocator for image state. + +**INV-I9. Atomic failure.** A render transaction that throws MUST leave all +retained state — cell buffers, registry, placements table — exactly as it was +before the transaction, and MUST return no bytes. A terminal that has received +no bytes is consistent with unchanged retained state; the caller may retry the +render after correcting the condition. The same holds for a throwing +registry call: validation MUST occur before any mutation (§7.2). + +## 7. Public API Additions + +_This section is normative for the shapes shown. Adoption amends +[Renderer Specification](renderer-spec.md) §8 (Appendix A)._ + +### 7.1 `img()` + +```ts +function img(id: string, props: ImgProps): Img + +interface ImgProps { + /** Registry id of the image to render. Omit for an alt-only element. */ + image?: number; + /** Required. Text fallback and human description. "" = decorative. */ + alt: string; + width?: SizingAxis; // default fit() — see §8.3 + height?: SizingAxis; // default fit() — see §8.3 + variant?: "auto" | "kitty" | "ascii" | "alt"; // default "auto" + /** Element background, same value space as open({ bg }). See §8.4. */ + bg?: number; +} + +interface Img { + directive: typeof OP_IMG; // opcode value is implementation surface + id: string; + image?: number; + alt: string; + width?: SizingAxis; + height?: SizingAxis; + variant?: "auto" | "kitty" | "ascii" | "alt"; + bg?: number; +} +``` + +The directive is **void**: one `img()` value in the array is a complete +element. It MUST NOT be paired with `close()` and MUST NOT have children. It +participates in layout as a leaf: its `width`/`height` sizing axes resolve like +any element's (§8), it is a pointer hit-test target under its element `id` +(renderer-spec §12.4), and its layout box is reported by `RenderInfo` +(renderer-spec §12.3). Its `id` MUST be unique among all element ids in the +frame, per renderer-spec §9.3 — duplicate ids are undefined behavior. + +The directive has no `caret` property. The renderer's hardware-cursor +machinery is text()-bound (renderer-spec §7.6); the only cursor bytes an image +frame may contain are the placement-positioning bytes of §9.2.3. + +A frame MAY contain any number of `img` directives. Two elements MAY reference +the same registry id; each receives its own placement (§9.2.4). + +The returned value is a plain object — no classes, no methods, no prototype +chain (renderer-spec §9.1). + +### 7.2 `term.setImage()` and `term.removeImage()` + +```ts +interface ImageData { + width: number; // positive integer + height: number; // positive integer + /** RGBA8, straight alpha, top-to-bottom rows, width*height*4 bytes */ + pixels: Uint8Array; +} + +interface Term { + setImage(id: number, data: ImageData): void; + /** Returns bytes to write now (kitty deletion); empty when nothing to emit. */ + removeImage(id: number): Uint8Array; +} +``` + +- `id` is an integer in `[1, 4294967295]` — the Kitty protocol's own image-id + range, which forbids zero — unique among the Term's live registry entries. + Re-setting a live id replaces the image (§4.3). +- `setImage` MUST throw a `RangeError` when: `id` is not an integer in + `[1, 4294967295]`; `width` or `height` is not a positive integer; + `pixels.length !== width * height * 4`; or storage fails — either no free + entry exists in the registry table, or the pixel pool cannot hold the + payload even after compaction (§5.3). The message SHOULD identify the + condition, mirroring the transfer-buffer error discipline of renderer-spec + §9.2. +- `setImage` returns `void` and emits nothing. Transmission is lazy (§9.2.2). +- `removeImage` returns a `Uint8Array` to write to the terminal immediately — + the same write-now discipline as `update()` output (terminfo-spec TINV-5). + The bytes, when the registry id was live, delete the image, all its + placements, and the terminal's stored copy of the data: one + `ESC _ G a=d,d=I,i=,q=2 ESC \` control block. The uppercase + form is the protocol's data-freeing deletion; the lowercase variant deletes + placements but retains stored data "so that the images can be re-displayed + without needing to resend" — retention is what §9.2.5's scoped deletions + exploit, and removal is the one place it is wrong. + Deletion bytes MUST be emitted whenever the id was live in the registry — + regardless of the current `kittyGraphics` evidence — because a placement + placed under earlier evidence may still exist terminal-side after the + evidence was denied (§11.3). The result is empty when the id is unknown. +- `removeImage` also dirty-marks (§10.4) every cell of every footprint that + referenced the id, so the next render repaints them; this happens even when + no deletion bytes are emitted. +- Both calls are synchronous. Neither renders, and neither performs IO + (INV-I1, INV-I7). + +### 7.3 Validation + +`validate(ops)` extended rules: + +- An `img` directive MUST carry a string `alt` (possibly empty). +- `image`, when present, MUST be an integer in `[1, 4294967295]`. +- `variant`, when present, MUST be one of the §7.1 literals. +- `width`/`height`, when present, MUST be sizing-axis values. +- `bg`, when present, MUST be a number (packed ARGB per `rgba()`, + renderer-spec §8.5). +- `img` directives are not opened or closed; `validate` MUST NOT treat them as + contributing to open/close balance. + +Malformed directives are invalid input; behavior follows the foundation rule +(renderer-spec §9.1): callers SHOULD validate; renderer behavior on invalid +input is unspecified. + +## 8. Layout and Sizing + +### 8.1 Layout box + +The layout engine computes the element's layout box from its `width`/`height` +sizing axes like any leaf element's. The layout box is what layout, clip +regions (§8.5), pointer hit-testing, and `RenderInfo` report (renderer-spec +§12.3). + +### 8.2 Painted footprint (contain) + +The **painted footprint** is resolved inside the layout box, per tier: + +| Tier | Painted footprint | +| ------ | ------------------------------------------------------------- | +| kitty | contain-fit region (below) | +| ascii | contain-fit region (below) | +| alt | the entire layout box | + +The contain-fit region of a `W × H` pixel source inside a `boxW × boxH` cell +box, under the half-row pixel convention (`A = 2`): + +- `cols = min(boxW, floor(boxH · A · W / H))` +- `rows = min(boxH, round(cols · H / (A · W)))` +- If `cols = 0` or `rows = 0`, the footprint is **empty** and the element + paints nothing at graphics tiers (alt still paints its full box, §9.4). +- Otherwise the footprint is the `cols × rows` rectangle centered horizontally + and vertically in the box, top-left at `x = floor((boxW − cols)/2)`, + `y = floor((boxH − rows)/2)` relative to the box origin. + +The footprint is what a graphics tier paints and what the ascii tier maps +pixels onto. It is always a subset of the layout box, and it equals the layout +box whenever the box already matches the source's aspect ratio under `A = 2` +(within the rounding above) — which is the case for intrinsic `fit()` sizing +(§8.3). + +v1 has exactly one scaling semantic. `scale: "stretch"` is not offered; see +§16.7 for why (the kitty protocol letterboxes unconditionally when both +placement dimensions are given, so stretch cannot be made honest at the pixel +tier). + +### 8.3 Intrinsic size (`fit`) + +`fit()` on either axis resolves to the image's **intrinsic cell size** under +the half-row pixel convention (§3). For image data of `W × H` pixels: + +- intrinsic width = `W` cells +- intrinsic height = `ceil(H / 2)` cells + +This is evidence-free: the convention is an assumption about the host cell, +not a measured value. An `img` with both axes at `fit()` renders at native +terminal pixel density under the assumed cell shape. + +For an alt-only element (`image` omitted), `fit()` resolves to the alt text's +wrapped size, like a `text()` element with wrapping enabled. + +### 8.4 Background and pointer + +- `bg` fills the element's layout box with the given packed ARGB background, + following the element-background rules for `open({ bg })` (renderer-spec + §12.2). Image pixels composite above it: kitty graphics alpha-blend over the + cells, so transparent source regions leave the background visible; ascii art + leaves the background of opaque cells untouched (§9.3.5). +- The element is a pointer target under its `id` over its **layout box**. The + existing pointer event surface (renderer-spec §12.4) applies unchanged; + coverage does not affect hit-testing. + +### 8.5 Clipping + +The effective clip region rules of renderer-spec §7.5 apply to image elements +unchanged. The element's **effective footprint** is the intersection of its +§8.2 painted footprint with its effective clip region; the effective layout +box is the intersection of its layout box with the region. + +- When the effective footprint is empty, the element paints nothing: no + placement is emitted and any previous placement is stale (§10). +- The ascii and alt tiers paint only within the effective region — their + per-cell output is ordinary cells and is suppressed outside it by the + existing clip machinery. +- Placement-origin and `c=`/`r=` values are computed from the effective + footprint, so a clipped image places exactly its visible cells. + +## 9. Tier Emission Contract + +### 9.1 Ladder resolution + +For each `img` directive, the renderer resolves exactly one tier: + +1. If `variant` is `"alt"`, or `image` is omitted, or the registry id is not + live — the tier is `alt` (with `IMAGE_NOT_FOUND` surfaced per §13 in the + missing-image case). +2. If `variant` is `"kitty"` or `"ascii"`, that tier is selected when its + evidence gate (§11.1) passes; otherwise resolution proceeds as if `variant` + were `"auto"`. +3. Otherwise (`variant` omitted or `"auto"`), the ladder is walked in order: + `kitty` when its evidence gate passes, then `ascii` (always available), + then `alt`. + +The resolution is deterministic (INV-I4): the same directive, capability +snapshot, registry state, and placements table always yield the same tier and +footprint. An evidence-gated override that fails its gate produces no error — +the ladder is the defined fallback, not an exception path. + +### 9.2 Kitty tier + +The kitty tier emits Kitty graphics protocol bytes (APC framing, +`ESC _ G [;] ESC \`), conformant with the protocol +specification. Every control block the renderer emits MUST set `q=2` (INV-I5). + +#### 9.2.1 Wire identities + +- The wire **image id** is the registry id, transmitted verbatim. The + protocol's id space — positive integers up to 4294967295, zero forbidden + (a zero or omitted id means "no id") — is exactly the registry id range of + §3, so the identity mapping keeps every registry id addressable with no + transformation to test or explain. +- The wire **placement id** is a pure function of the directive's element id + and the image's registry id, in `[1, 4294967295]`. The protocol treats + `p=0` (and omission) as "unspecified", which would create accidental + additional placements — placement ids MUST NOT be 0. + - Requirements: deterministic (same inputs → same id across frames and + across Terms' sessions with the same element ids), uniform over its range, + and cheap to compute. The specific hash is implementation-defined (§15). + - Collision handling: two elements whose (element id, registry id) pairs + collide in the hash would share a placement id and each other's cleanup. + The renderer MUST detect same-frame collisions and resolve them + deterministically: the affected elements resolve to the ascii tier for + that frame, emit no placements, and surface + `"IMAGE_PLACEMENT_COLLISION"` per affected element (§13) — a collision + can never leave undeletable graphics debris on the terminal. A + renderer-side id counter would reintroduce cross-frame state and is + deliberately avoided (INV-I3). +- Every placement is identified on the wire by the pair (image id, placement + id); image ids alone never address a placement this renderer created. + +#### 9.2.2 Transmission + +- Pixels are transmitted as RGBA direct data: `a=t,t=d,f=32`, with source + dimensions `s=,v=` and wire image id `i=`, base64-encoded. +- Transmission is **lazy**: it occurs in the render transaction that first + places the image, not at `setImage()` time. +- **Transmission state.** Each registry entry tracks the data version last + transmitted (0 = never). Before a placement is emitted, the renderer + transmits the image iff `transmitted_version ≠ version`. After a successful + transmission, `transmitted_version = version`. +- `transmitted_version` MUST be reset to 0 — forcing re-transmission on the + next placement — when: the image's last live placement is deleted (unmount, + tier switch, collision demotion, registry removal); a `kitty-graphics` + capability denial deletes its placements (§11.2); or the Term is resized + (the placements table, and with it all assumptions about terminal-side + image persistence, is discarded). +- **Re-transmission is preceded by deletion.** Any transmission of data for a + wire image id that may already exist terminal-side MUST be preceded by + `ESC _ G a=d,d=i,i=,q=2 ESC \` (delete image and all its placements). + The protocol requires this for id re-use: re-transmitted data does not + update existing placements, and the old image must be removed first to avoid + divergent behavior when ids are re-used in a session. +- Payload chunking: chunks MUST be at most 4096 bytes of base64; every chunk + except the last MUST have a length that is a multiple of 4; all chunks + except the last MUST set `m=1`, the last MUST set `m=0`. The first chunk's + control block carries the full transmission keys; continuation chunks carry + only the `m` key, in the protocol's documented form. The renderer MAY choose + smaller chunks. +- The renderer MUST NOT re-transmit image data while `transmitted_version = + version` (INV-I2 discipline: no redundant uploads) — subject to the resets + above, which exist because the protocol's storage quota preferentially + deletes images without live placements, so persisted-data assumptions MUST + be dropped whenever this renderer drops a placement. + +#### 9.2.3 Placement positioning + +Placements are positioned by the hardware cursor: the protocol renders a +placement at the cursor's current cell, from its top-left. Without pixel +geometry evidence (§16.3) this cursor-positioned form is the protocol's only +cell-addressable placement path, and it is the one v1 uses. + +- Before each placement control block, the renderer MUST move the hardware + cursor to the effective footprint's origin cell with the same absolute CUP + form the renderer uses for cell writes, including the render `row` option + (renderer-spec §8.2.1). +- The placement control block is + `a=p,i=,p=,c=,r=,C=1,q=2`: the footprint's cell + dimensions on `c=`/`r=`, and `C=1` to pin the cursor movement policy — the + protocol's default post-placement cursor jump by the box size would leave + the cursor at an implementation-defined position when the box exits the + screen. `C=1` does not position the placement; the preceding CUP does. +- With both `c=` and `r=` given, the protocol letterboxes to prevent + distortion. Because the footprint is the contain-fit region (§8.2), the + letterbox is a no-op and the terminal's painted pixels fill exactly the + footprint — the agreement INV-I4 requires. +- Placement ids are reused across frames for the same (element id, registry + id). Re-sending a placement with the same (image id, placement id) pair + replaces the previous graphic without flicker — the protocol's documented + move/resize path — and §10.2 uses it normatively: a footprint change of an + unchanged image is a pure re-place with no intervening deletion. + +#### 9.2.4 Covered-cell blanking + +Cells in the effective footprint are blanked as part of the frame's cell +output — spaces with the element `bg` applied (§8.4) — so no text shows +through transparent pixels and no stale text is trapped under the image +(§10.1). + +#### 9.2.5 Cleanup + +- Before a frame's cell writes for a region whose placement is stale — the + element unmounted, the tier changed away from kitty, the element demoted by + a placement-id collision (§13), or the image removed or replaced — the + renderer MUST emit deletion of the stale placement: + `ESC _ G a=d,d=i,i=,p=,q=2 ESC \`. The lowercase form deletes the + placement while retaining the terminal's stored data, which the protocol + documents as re-displayable without resending; whether the data survives is + governed by the transmission-state resets of §9.2.2, since stored data is + never trusted. Deletions MUST precede the frame's other output for that + region. A delete command received while an image's chunked upload is + incomplete aborts the partial upload (per the protocol); §9.2.6's + deletions-before-transmissions ordering makes that unobservable. +- The renderer MUST NOT rely on the terminal's text-overwrites-graphics + behavior for cleanup; it deletes explicitly (INV-I6). + +#### 9.2.6 Frame ordering + +Within one render transaction's output, the renderer MUST emit, in order: + +1. **Stale-placement deletions** — every placement in the placements table + that this frame does not re-create and whose image's data version did not + change, in previous-frame directive order. (Data-version deletions ride + with the transmission block below, not here.) +2. **Transmissions** — for each image requiring (re-)transmission: the + delete-id block, then its chunks, in registry id order. The delete-id + block subsumes any scoped deletions for that image's placements: the + protocol requires existing image and placements to be deleted before + re-transmission, and the same frame re-places them (§10.2). +3. **Cell writes** — the ordinary changed-cell diff bytes, including covered + cells' blanks and any repair bytes from §10. +4. **Placements** — for each kitty placement, in directive order: the + positioning CUP, then the placement control block. + +The two ordering constraints that matter: deletions precede the cell writes +that repaint their footprints, and blanking precedes the placements that +overwrite those blanks. Both hold in the order above. After the last +placement, the renderer MUST restore the post-frame cursor state required by +renderer-spec §7.6 (the §10.6 carve-out). + +### 9.3 ASCII tier + +The ascii tier renders the image as character art in the cell grid. It uses +only ordinary cells, so it works on every terminal and participates in normal +diffing. + +Glyph selection is **shape-aware**, adopting the approach described in +[ASCII characters are not pixels](https://alexharri.com/blog/ascii-rendering) +(Harri, 2026). A 1-D luminance ramp treats characters as pixels: their shapes +are ignored, edges blur into averaged blocks, and contours jag. Instead, each +candidate character is modeled by a shape vector that measures how its ink +distributes across regions of the cell; each source cell is measured over the +same regions; and the character whose shape best matches the measurement is +emitted. Characters then follow image contours, giving the art far higher +effective resolution than block-averaged downsampling. The pipeline below is +deterministic (INV-I4). + +#### 9.3.1 Source-rectangle sampling + +The image is mapped onto the effective footprint cell by cell. Cell `(cx, cy)` +of a `cols × rows` footprint maps to the source rectangle starting at +`round(cx * W / cols)`, `round(cy * H / rows)` and ending at +`round((cx+1) * W / cols)`, `round((cy+1) * H / rows)`. When this rectangle is +empty — the mapping rounds to zero width or height at fine footprints — it +consists of the single pixel at the start coordinates, clamped into the image. + +Three measurements come from each cell's source rectangle: + +- **Color sample** — the per-channel arithmetic mean of the rectangle's + pixels. Drives foreground color (§9.3.5). +- **Alpha sample** — the mean alpha of the rectangle's pixels. Drives + transparency (§9.3.5). +- **Glyph sampling vector** — one luma value per internal sampling circle + (§9.3.2): the mean of `Y = 0.2126·R + 0.7152·G + 0.0722·B` (channel values + 0–255) over the rectangle's pixels whose centers fall inside that circle. + Alpha does not participate in glyph sampling (Open Decision 3). + +All sampling happens in a cell-relative coordinate frame: a point `(u, v)` in +the unit cell maps to the source plane by `x = x0 + u·(x1 − x0)`, +`y = y0 + v·(y1 − y0)`. Internal circles sample `u, v ∈ [0, 1]`; external +circles sample outside the unit cell (§9.3.2), and the same map extrapolates +them onto the source plane. + +The same luma formula, sampled over the **external** sampling circles — which +reach beyond the cell into neighboring source regions — yields the cell's +**external sampling vector**. Pixels outside the source image are excluded +from an external circle's sample; a circle with no in-bounds pixels contributes +nothing to the enhancement in §9.3.4. + +#### 9.3.2 Sampling circles and shape vectors + +Sampling circles are fixed regions of the cell, defined in cell-relative +coordinates so they are independent of footprint size and source resolution: + +- Six **internal sampling circles** arranged in two staggered columns — the + left column lowered, the right raised, radii enlarged — so the pair covers + the cell well while limiting mutual overlap. Staggering follows the cell's + taller-than-wide shape (the `A = 2` convention of §8.2) and captures + top/bottom, left/right, and diagonal structure: `T` vs `L` vs `p` vs `q`, + `-` vs `_` vs `^`, `/`. +- Ten **external sampling circles** around the cell boundary, each reaching + into neighboring cells. They have no effect on a solitary cell, but they let + contrast enhancement see boundaries adjacent to it (§9.3.4), which removes + the staircasing that global enhancement alone produces. + +The circle geometry — centers and radii as fractions of the cell box, plus a +fixed internal/external circle ordering — is a closed parameter set whose +numeric values are calibration constants pinned by golden tests in the +implementation PR (Open Decision 2). What this specification +closes normatively: the counts (6 internal, 10 external), the two-column +staggered arrangement with lowered-left/raised-right and enlarged radii, the +fixed orderings used by the affecting table below, and the per-component max +normalization of shape vectors. + +**Shape vectors.** For each character in the candidate set (§9.3.3), a +6-component shape vector records, for each internal sampling circle, the +fraction of that circle's area covered by the character's ink. Shape vectors +are computed at build time from a rasterization of each character over the +same circle geometry and compiled into the module (§15) — no runtime font +access exists (INV-I1; the renderer has no font machinery). Every component is +then divided by the largest value of that component across all candidates +(per-component max normalization), spreading the candidates across the full +component range. + +Shape vectors model the build-time rasterization face, while the host terminal +renders the final glyphs in its own font. The renderer cannot rasterize the +host's font (INV-I1), so this mismatch is unavoidable; it is why the candidate +set SHOULD favor shape-stable glyphs (Open Decision 1). + +#### 9.3.3 Character set and matching + +The candidate set is a closed table: all 95 printable ASCII characters +(U+0020–U+007E), in ascending code-point order; this order is the tie-break +order. U+0020 (space) is the zero-density anchor; the alpha rule (§9.3.5) +bypasses matching entirely for transparent cells. + +| Range | Characters | +| ------------ | -------------------------------------------------------------- | +| U+0020 | ` ` (space) | +| U+0021–002F | `!` `"` `#` `$` `%` `&` `'` `(` `)` `*` `+` `,` `-` `.` `/` | +| U+0030–0039 | `0`–`9` | +| U+003A–0040 | `:` `;` `<` `=` `>` `?` `@` | +| U+0041–005A | `A`–`Z` | +| U+005B–0060 | `[` `\` `]` `^` `_` `` ` `` | +| U+0061–007A | `a`–`z` | +| U+007B–007E | `{` `\|` `}` `~` | + +Pruning to a curated subset is Open Decision 1. + +The cell's glyph is the candidate whose shape vector has the smallest squared +Euclidean distance to the cell's contrast-enhanced glyph sampling vector +(§9.3.4); the squared distance suffices, no square root is taken. Distance +ties resolve toward the lower code point. +Lookup acceleration — k-d trees over the 6-D vectors, or result caches keyed +by quantized sampling vectors — is implementation-defined (§15); cached and +uncached lookups MUST agree. + +#### 9.3.4 Contrast enhancement + +Before matching, the glyph sampling vector passes through two contrast steps, +in this order. Enhancement increases the separation between differently +shaded regions, so boundary cells pick characters that emphasize the boundary +(the `&` → `b` → `L` style transitions of the reference approach) instead of +mid-density compromises. It barely affects near-uniform vectors, so smooth +gradients survive. Sampling vectors are oriented bright-to-dense: brighter +source regions match denser-ink glyphs. + +1. **Directional** (per component `i`): with + `m = max(v[i], max({v_ext[j] : j affects i}))`, set + `v[i] = (v[i] / m)^γd · m`; when `m = 0`, the component is set to `0`. The + affecting-external sets — normative, from + the reference approach, with internal circles `0–5` and external circles + `0–9` in their fixed ordering — are: + + | Internal | Affected by external | + | -------- | --------------------- | + | 0 | 0, 1, 2, 4 | + | 1 | 0, 1, 3, 5 | + | 2 | 2, 4, 6 | + | 3 | 3, 5, 7 | + | 4 | 4, 6, 8, 9 | + | 5 | 5, 7, 8, 9 | + +2. **Global**: with `m = max(v)`, set `v[i] = (v[i] / m)^γg · m` for every + component; when `m = 0`, the vector is set to all zeros. + +The exponents `γd` and `γg` are baked implementation constants calibrated +together with the circle geometry (Open Decision 2). + +#### 9.3.5 Alpha, color, and layout agreement + +- When the alpha sample is below 128, the cell is transparent: glyph space + (U+0020), no foreground, and the background rules of renderer-spec §8.3.3 + apply (element `bg` when declared, otherwise the underlying background is + preserved). +- Opaque cells (alpha sample ≥ 128) carry the color sample as their + foreground, encoded through the renderer's existing color-encoding path; when + the color-encoding capability work lands, ascii art colors follow it. The + background of opaque cells is not overridden: the glyph's ink density over + the preserved background is what carries the image's lightness, and + equalizing foreground and background would erase the very glyph texture the + shape matcher selected. +- The effective footprint computed in §8 is exactly the art's cell region + (INV-I4). The art is never clipped by its own tier; callers size the element + or rely on clip regions as with any content. + +### 9.4 Alt tier + +The alt tier renders `alt` as plain cell text over the **entire layout box**: +wrapped at the box width, clipped at the box height (overflow is suppressed, +not laid out past the box), with the element's `bg` applied per §8.4 and no +implicit prefix or decoration. Empty `alt` renders an empty box (decorative +image). Alt text is unstyled in v1 — no foreground override, no attributes. + +## 10. Cell-Buffer Integration and Damage + +### 10.1 Covered-cell representation + +A cell inside a live kitty placement's effective footprint holds the +**covered-cell value**: space glyph, no foreground, and the element's declared +`bg` — or the default background when none is declared. Covered cells are +ordinary diff participants: front and back values equal ⇒ no bytes, exactly as +for any other cell. There is no sentinel mode, no special diffing, and no +per-cell ownership map in v1. + +The letterbox margin cells of the layout box (outside the footprint) are +ordinary background cells under §8.4 and diff normally. + +### 10.2 Footprint change + +When an image's effective footprint changes between frames and the image's +data version is unchanged, the placement is **re-placed**: emitted at the new +origin with its same (image id, placement id) pair (§9.2.3). The protocol's +same-pair replacement removes the previous graphic without flicker and without +an intervening delete; the old box's pixels vanish with the placement record, +and the blanks previously written under them remain the terminal's text +content at those cells. The diff then repaints the old box's cells whose +front-buffer value (blank) differs from the back (the ordinary vacate case) +and emits blank bytes for newly covered cells. No deletion occurs for a mere +move or element resize, and no bytes are emitted for old-box cells whose new +content equals the blank value they already hold. + +When the data version changed, the re-transmission path of §9.2.2 applies: +delete, re-transmit, then re-place every placement of that image. When the +placement is not re-created at all — unmount, tier change, collision demotion +— the stale placement is deleted (§9.2.5). + +A placement whose covered cells receive any cell write in the current frame — +a changed blank-state background beneath unchanged coverage, or any other +buffer change inside its footprint — MUST be re-placed with its same +(image id, placement id) pair in that frame: any write under a placement +erases the graphic's cells, so the re-place heals the whole box rather than +poking holes in it. The re-place is the repair; no deletion is emitted for it. + +Because a deleted placement's pixels are erased by the delete itself, the +renderer MUST additionally force the diff to re-emit every cell of a deleted +placement's previous footprint: each such front-buffer cell is dirty-marked +(§10.4) before diffing. This **deletion damage** rule is what keeps the +terminal's visible state matching the cell buffer after an unmount, tier +change, or removal (INV-I6). + +### 10.3 Overlap policy + +Content from other elements that lands in a live placement's footprint is +**suppressed by a post-walk coverage pass**: after the render-command walk — +including border-junction resolution — image elements resolved to a pixel +tier assert their effective footprint into the back buffer, writing the +covered-cell value over whatever content the walk left there. Running the +pass after the walk, not in directive order, is what makes suppression +structural: overlapping text's bytes are never emitted, so no damage-repair +reaction is needed for text overlap, and directive order cannot defeat +coverage. Pixel-tier coverage cannot honor painter's order — the terminal +renders the placement as a layer over the whole box, and a cell buffer that +showed text under the placement would disagree with the terminal's visible +state either way the bytes were ordered (§9.2.6). v1 therefore makes the +image win at the pixel tiers, deterministically. + +- When two pixel-tier footprints overlap in one frame, the later element (in + directive order) **demotes to the ascii tier for that frame** — a ladder + fallback, silent like any other fall-through (§9.1), because the demoted + element remains fully representable. The earlier placement keeps its + footprint intact; the demoted element's art cells are ordinary cells that + the coverage pass overwrites with the earlier placement's blanks. +- Overlap between **ascii**/**alt**-tier images is ordinary painter's order: + later content in directive order wins per cell, like any text. +- The placements-table capacity degradation of §5.4 applies before any + placement is emitted, so suppression never leaves an untracked placement. +- Callers SHOULD avoid overlapping images with other content in v1; the + z-ordered coexistence design is deferred (§16.6). + +### 10.4 Dirty-marking primitive + +Dirty-marking a cell region forces the next render's diff to emit bytes for +it. The described mechanism: overwrite the region's front-buffer cells with +the default cell value, so the next diff necessarily differs. Any mechanism +that guarantees the same emissions satisfies the contract. Dirty-marking is +used by: `removeImage` (§7.2), deletion damage (§10.2), and capability +invalidation (§11.2). + +### 10.5 Resize + +A resize step (renderer-spec §7.7) discards the placements table with the rest +of the diff state. Before the discard, the resize transaction MUST: + +1. Recover the images with live placements from the table and emit, in the + update's returned bytes, one data-freeing deletion per image: + `ESC _ G a=d,d=I,i=,q=2 ESC \`. This is the write-now + discipline (TINV-5): the redraw that follows cleans pixels by writing text + over covered cells, but it cannot clean terminal-side records the way a + deletion does. +2. Reset the affected images' transmission state (§9.2.2) — placements are + gone, so persisted-data assumptions are dropped (the protocol's storage + quota preferentially deletes images without placements). +3. Reconcile the table and the front buffer's covered cells as part of + discarding diff state, so the next render's complete redraw contains no + image-specific bytes beyond the new frame's own paint. + +Registry entries and their pixel data are Term state, independent of terminal +dimensions, and MUST survive a resize unchanged (§5.1). + +### 10.6 Cursor interaction + +Placement positioning requires moving the hardware cursor. This is the sole +sanctioned exception to renderer-spec §7.6's "no cursor-positioning bytes in +caret-less frames": frames containing kitty-tier images MAY include the +cursor-positioning bytes needed for placement, provided the post-frame cursor +visibility state defined in §7.6 is unchanged, and for caret-declaring frames +the post-frame position is the caret's cell as §7.6 requires. The renderer +MUST account for the render `row` option (renderer-spec §8.2.1) when computing +placement positions, exactly as it does for cell CUPs. + +## 11. Capability Evidence and Invalidation + +### 11.1 Evidence table + +| Tier | Gate | Evidence source | +| ----- | ------------------------------------------------- | --------------------------------------------------------------------------------------- | +| kitty | `RuntimeCapabilities.kittyGraphics === true` | Probe query 9 (APC `_G` query, terminfo-spec §9.1), delivered as a `CapabilityEvent` (terminfo-spec §6.3) folded in through `term.update()` | +| ascii | none | Always available | +| alt | none | Always available | + +The v1 kitty gate is the existing boolean. The renderer does not verify +sub-features of the kitty graphics protocol (placeholders, animation) in v1; +it uses only transmission, placement, and deletion — the protocol's core. + +### 11.2 Capability-change invalidation + +When a `CapabilityEvent` folded through `update()` changes a tier's +eligibility — a `kitty-graphics` event arriving true or false — the update +transaction's behavior is: + +- **Denial (true → false).** The transaction returns immediate deletion bytes: + one data-freeing `a=d,d=I,i=,q=2` block per registry image + with live placements (§7.2), recovered from the placements table before any + state is discarded, and resets the affected images' transmission state + (§9.2.2). The table's placement entries are then cleared and the front + buffer's covered cells are reconciled to their blank-state values, so the + next render emits no redundant image bytes. This honors the write-now + discipline (TINV-5): a caller that never renders again does not leave + graphics on the terminal. +- **Grant (false → true).** No immediate bytes: the terminal's visible state + does not change (the lower tier's art is on screen; the kitty tier has + nothing to delete). The next render's own resolution transmits and places + at the kitty tier. + +This is the focused-spec output invalidation that renderer-spec §7.7 and §7.8 +reserve for feature specifications. A capability event that changes only color +encoding (e.g., `colordepth`) affects ascii art colors; the renderer MAY +re-emit affected art on the next render. It MUST NOT re-transmit unchanged +kitty data for color changes. + +### 11.3 Removal and capability edge cases + +- `removeImage()` when `kittyGraphics` is false but the id is live still + returns deletion bytes (§7.2): placements placed under earlier evidence may + persist terminal-side. +- An image placed at the kitty tier whose capability is later denied is + cleaned by §11.2's immediate denial rule, not by `removeImage()`. +- A tier re-arm (denied → supported) re-transmits lazily at the next + placement: the denial deleted the terminal-side data, so the transmission + reset of §9.2.2 forces a fresh upload. No bytes flow at grant time. + +## 12. Interaction with Line Mode + +In `mode: "line"` renders (renderer-spec §8.2.2), tier resolution is capped at +`ascii`: the renderer MUST NOT emit graphics-protocol bytes in line mode. Line +mode output is newline-separated rows written inline; the graphics protocol +requires cursor-addressed placement that line mode does not own. The ladder +above the cap resolves to ascii, then alt, unchanged. + +## 13. Errors + +The render result's `errors` channel (renderer-spec §12.3) gains three +Clayterm-specific types alongside `"CLIP_DEPTH_EXCEEDED"`: + +| `type` | Raised when | Behavior | +| ------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | +| `"IMAGE_NOT_FOUND"` | An `img` directive references a registry id that is not live (registry empty, id never set, or removed) | Renders at the alt tier (graceful degradation); reported once per affected element per render | +| `"IMAGE_PLACEMENTS_EXCEEDED"` | More kitty placements were required in one frame than the placements table holds (§5.4) | Excess elements degrade to the ascii tier for that frame, in directive order; reported once per render | +| `"IMAGE_PLACEMENT_COLLISION"` | Two or more elements in one frame derive the same placement id (§9.2.1) | The affected elements resolve to the ascii tier for that frame and emit no placements; reported once per affected element | + +None of these abort the frame; other elements render normally. The error +`message` fields SHOULD identify the affected element ids and, for +`"IMAGE_NOT_FOUND"`, the missing registry id. All three are specified as +closed fallbacks rather than undefined behavior so that no condition can +leave undeletable graphics debris or an untracked placement. + +Thrown conditions — descriptive `RangeError`s that abort the transaction and +return no bytes (INV-I9), following the transfer-overflow discipline of +renderer-spec §9.2: + +| Condition | Raised by | Message SHOULD identify… | +| ------------------------------------------- | ------------ | ---------------------------------------------------------------- | +| `setImage` validation failure (§7.2) | `setImage()` | the offending field and value | +| Registry table full / pixel pool exhausted | `setImage()` | the capacity limits and the requested size (§5.3) | +| Output-capacity overflow | `render()` | the emission that overflowed (suggest a smaller image or frame) | + +After any throw, the Term remains usable: the next render behaves as a full +re-render of that frame (INV-I9 keeps retained state truthful). + +## 14. Testing Strategy + +All tests live in `test/image.test.ts` — one spec, one test file (AGENTS.md). +Obligations: + +1. **Tier resolution truth table.** For every combination of `variant`, + `image` presence, registry liveness, and `kittyGraphics` evidence, the + selected tier matches §9.1. +2. **Footprint math.** Intrinsic size (§8.3), contain resolution (§8.2) — + including the `ceil`/`round`/`floor` edge cases, empty footprints, and + upscale/downscale paths — agree between a pure reference and rendered + output (INV-I4). +3. **Kitty conformance.** Transmitted frames are valid APC with RGBA direct + encoding (`a=t,t=d,f=32,s=,v=`), chunked at ≤ 4096 base64 bytes with + multiple-of-4 non-final chunk lengths, `m=1` continuation / `m=0` final, + `q=2` on every control block, ST-terminated. Placement blocks carry + `a=p,i=,p=,c=,r=,C=1,q=2` and are preceded by a CUP honoring the `row` + option. +4. **Transmission state.** First placement transmits; re-renders without data + change do not re-transmit; a version bump emits delete-id, re-transmit, + re-place in order (§9.2.2); loss of the last placement (unmount, tier + switch, collision demotion), capability denial, and resize each force + re-transmission on the next placement (§9.2.2, §11.2). +5. **Placement and cleanup.** First placement emits no stale-delete; footprint + change of an unchanged image emits a pure same-pair re-place with no + deletion (§10.2); a version bump emits delete, re-transmit, re-place + (§9.2.2); unmount and tier switch emit the scoped lowercase + `a=d,d=i,…,p=…` form; `removeImage` emits the data-freeing `d=I` form once + and the next render emits no image bytes for the vacated footprint; + placement ids are stable across frames for the same element and distinct + across elements; ids are never 0; wire image ids equal registry ids. +6. **Cell-buffer honesty.** Covered cells hold the covered-cell value and + produce no bytes while unchanged; overlapping text is suppressed (§10.3); + vacated and deletion-damaged cells repaint (§10.2). +7. **ASCII art determinism.** Golden-image tests: fixed pixel input → exact + glyph/fg/bg cells, locking the circle arrangement, affecting table, + shape-vector normalization, tie-breaking, the alpha threshold, and the + empty-rectangle rule (§9.3). Cached and uncached glyph lookups MUST agree. + Edge cases: uniform vectors pick density-appropriate glyphs with minimal + enhancement; external circles crossing the image border contribute + nothing; 1×1-cell footprints. +8. **Alt tier.** Wrap, clip, empty-alt, bg application, and box-width text + painting (§9.4). +9. **Registry API.** `setImage` validation RangeErrors (bad id, bad + dimensions, wrong byte length, table-full, pool-exhausted-even-after- + compaction); replace bumps data version; compaction preserves live entries' + render output; `removeImage` idempotence, byte emission with + `kittyGraphics` both true and false, and dirty-marking. +10. **Capacity behavior.** Registry and placements capacities produce §5.3's + RangeError and §13's degradation respectively, deterministically. +11. **Capability invalidation.** `kitty-graphics` true after ascii renders + switches tier on next render; denial after kitty renders returns deletion + bytes from `update()` and the next render paints ascii (§11.2). +12. **Line mode.** Kitty-tier images render as ascii in line mode (§12). +13. **Snapshot equivalence.** An `img` directive inside `snapshot()` renders + identically to the direct directive (renderer-spec §9.1), and re-resolves + registry liveness and tier at each render rather than at pack time. +14. **Clip interaction.** Clipped images place and paint only their effective + footprint (§8.5). +15. **Resize.** Registry survives resize; the resize update returns one + data-freeing deletion per image with live placements and resets + transmission state; the placements table is discarded; the post-resize + redraw erases prior pixels; re-transmission occurs on next placement + (§10.5). +16. **Errors.** `IMAGE_NOT_FOUND` fallback and once-per-element surfacing; + `IMAGE_PLACEMENT_COLLISION` demotes affected elements and emits no + placements; a thrown render returns no bytes and leaves retained state + unchanged, with the retry render reproducing the frame (INV-I9). +17. **Input-stream cleanliness.** Rendered output containing kitty frames, + fed to `input.scan()`, yields zero events (INV-I5). + +## 15. Implementation Notes + +_This section is non-normative._ + +**Transfer encoding.** `img` adds one opcode to the command buffer in the +§12.1 style: an opcode word, the id string, a property mask, and fields for +the registry id, sizing axes, alt string, variant, and bg. The existing +protocol is extended, not restructured. Because the underlying layout engine +models elements as open/close pairs, the void directive expands to a balanced +open/close pair with leaf sizing at decode time; `validate` sees the void form. + +**Placement-id derivation.** The reference implementation pins FNV-1a over the +element id's UTF-8 bytes and the registry id's decimal digits — XOR-separator +`0x3A` between them; a raw result of 0 maps to 1 — into `[1, 4294967295]`; +golden tests lock the exact values. The derivation MUST satisfy the normative +requirements of §9.2.1 (pure, deterministic, uniform); pinning is what makes +golden tests exact and cross-session reports comparable. The known limitation +is the same-frame collision case of §9.2.1, handled per §13. + +**Ordinary-cell art.** The ascii tier's cell pipeline (glyph + fg + bg per +cell) is the same machinery plain text uses; no new cell representation is +required for art. Kitty-covered cells are the only new cell-buffer concept, +and they are behaviorally just blank cells (§10.1). + +**Ascii cost and shape-vector baking.** Glyph selection samples each cell's +source rectangle per circle and runs a nearest-neighbor match per cell: +O(W·H) sampling once per changed image plus O(cells × candidates) distance +evaluations. The reference approach's accelerations apply — k-d trees over +the 6-D shape vectors, or result caches keyed by quantized sampling vectors — +and are implementation-defined, with cached and uncached lookups required to +agree (§9.3.3). The renderer MAY additionally memoize per-element matched art +keyed by (registry id, data version, footprint box) in fixed-capacity carved +storage; memoized and unmemoized outputs MUST agree. Shape vectors are baked +at build time from a rasterization of the candidate characters — a small +closed table of candidates × 6 circle fractions, not a font system — so no +runtime rasterization exists (INV-I1). Kitty transmission is O(W·H) once per +data version. All costs are bounded by registry image sizes, not by frame +rate. + +**Streaming video ergonomics.** The registry's replace-on-id path is the +natural video-streaming API (one `setImage` per frame). Each replacement +costs a delete/re-transmit/re-place cycle (§9.2.2); terminals batch this +within a write burst, but the protocol does not guarantee flicker-free +updates. Per-frame streaming ergonomics are revisited with §16.4 +(placeholders), which would make moves pure cell diffs. + +**Cursor safety.** `C=1` keeps the cursor at the placement origin; combined +with §10.6's post-frame restore, no frame leaves the cursor somewhere the +caller did not put it. + +## 16. Deferred (v1 boundaries) + +### 16.1 Sixel tier + +The intended ladder position is kitty → **sixel** → ascii → alt. Sixel is +deferred because it cannot be specified correctly without two evidence +sources that do not exist yet: + +1. **Sixel support evidence.** Terminals advertise sixel in their DA1 feature + bits (feature `4`); the terminfo specification currently treats the DA1 + reply as a fence only. A follow-up must extend DA1 parsing into a new + `CapabilityEvent` key (e.g., `sixel-graphics`). +2. **Cell-size geometry evidence.** Sixel paints pixels at the cursor with no + cell-footprint parameter, so both the painted footprint and the required + downscale depend on the cell pixel size (XTWINOPS 14/16, or OSC 776-style + reports). Geometry reporting is a tracked terminfo follow-up. + +When both exist, a focused amendment to this specification adds the tier: +256-color quantization of the source (quantizer choice implementation-defined), +deterministic re-encoding on footprint change, and region cleanup by +overpainting (sixel has no placement model to delete). Until then the ladder +skips sixel unconditionally, and a sixel-capable terminal still receives +kitty/ascii/alt. + +### 16.2 iTerm2 inline images + +OSC 1337 `File=` is a third pixel protocol. Evidence (identity parsing of +DA2/XTVERSION) is a terminfo follow-up; the tier slots between kitty and +sixel by fidelity. + +### 16.3 Geometry-evidence scaling + +With cell-size evidence (`A` measured instead of assumed 2), contain becomes +exact on terminals whose cells are not 2:1, and the intrinsic-size default +can be expressed in true terminal pixels. The change is internal to §8.2 and +gated on the geometry follow-up. + +### 16.4 Kitty unicode placeholders + +Transmitting once and placing via placeholder codepoints in cells would make +kitty-covered cells first-class diff participants (moves become pure cell +diffs, and scroll-region tracking comes nearly free). It requires evidence +that the terminal implements placeholders, which the current probe does not +capture. Deferred to a focused amendment. + +### 16.5 Background-aware art + +The lightness mapping assumes a dark terminal background: dark source regions +map toward low-density glyphs. When theme evidence +(`RuntimeCapabilities.theme.background`) indicates a light background, the +mapping SHOULD invert (complement the luma) so art remains legible. Deferred +until the color-encoding work lands. + +### 16.6 Z-ordering and overlap + +Coexistence of graphics with overlapping text (floating overlays over images) +is unspecified beyond §10.3's policy. A future design would need kitty `z` +semantics (including negative z-index placements, which the protocol draws +under text) or placeholder-based layering. + +### 16.7 Stretch scaling + +`scale: "stretch"` is not in v1. At the kitty tier it is unsatisfiable: with +both `c=` and `r=` specified the protocol letterboxes to prevent distortion, +so a stretched render would claim cells it does not paint (INV-I4). At the +ascii tier stretch is representable (the pixel mapping is renderer-owned) and +could be added as a directive property in a future amendment, ascii-only. + +### 16.8 Animation, cropping, transitions + +Multi-frame formats, source rectangles, and transitions on image geometry are +not designed here. The directive model leaves room: `image` could become a +richer descriptor and `transition` could apply to footprint axes without +breaking §7.1's shape. + +### 16.9 Synchronized-output wrapping + +How graphics bytes interact with DECRQM 2026 frame wrapping is the +synchronized-output follow-up's concern. This specification treats graphics +bytes as ordinary frame output. + +## Appendix A. Proposed Amendments to Existing Specs + +_None of the following is applied yet. This appendix is the proposal; on +approval the amendments land as their own spec-only commit._ + +**A1. Renderer-spec §4.3 / §6 INV-3 (retained state).** Amend "the only state +the renderer retains between frames is the cell buffer used for diffing" to +"the only state the renderer retains between frames is diffing state: the +cell buffers, the image registry, and the previous frame's placements table +(as defined in the Image Specification §4.1)". Frame-snapshot independence of +the directive array is unchanged. + +**A2. Renderer-spec §7.6 (cursor carve-out).** Append to the cursor +visibility/positioning section: frames containing image elements at a +graphics tier MAY additionally include cursor-positioning bytes required by +the graphics protocol (Image Specification §10.6), provided the post-frame +cursor visibility state defined here is unchanged and caret-declaring frames +still end with the caret's cell. + +**A3. Renderer-spec §8.3 (directive constructors).** Add §8.3.5 `img()` +deferring normatively to the Image Specification, noting the void-directive +form and that `img` values do not participate in open/close balance. + +**A4. Renderer-spec §7.8 (capability consumption).** Remove "Kitty graphics +emission" from the deferred list and reference the Image Specification as the +focused feature specification that defines it. + +**A5. Renderer-spec §12.1/§12.2 (current surface).** Note the `img` opcode in +the transfer encoding and the `ImgProps` surface, as current implementation +surface per §5's contract-layer boundary. + +**A6. Renderer-spec §12.3 (errors).** Add `"IMAGE_NOT_FOUND"`, +`"IMAGE_PLACEMENTS_EXCEEDED"`, and `"IMAGE_PLACEMENT_COLLISION"` to the +Clayterm-specific error types. + +**A7. Renderer-spec §8.1 (createTerm options).** Add the optional +`imagePoolBytes` option (Image Specification §5.3), default 4 MiB, sizing the +carved image registry. The memory carve and capacity behavior are normative +in the Image Specification; renderer-spec §2 keeps WASM memory layout out of +scope beyond such behavioral requirements. + +**A8. Terminfo specification.** No changes in v1. The kitty tier consumes the +existing `kitty-graphics` evidence; sixel and geometry evidence are future +amendments (§16.1, §16.3). (The `createTerm` restatement in terminfo-spec +§10.2 lists the same options as renderer-spec §8.1; when A7 lands, that +restatement is updated in the same commit to avoid drift.) + +**A9. Input specification.** No changes. The input parser already recognizes +and silently consumes kitty graphics APC replies (input-spec §6.2); INV-I5 +keeps the renderer's emissions from generating any. + +**A10. Renderer-spec §7.7 (update transaction).** Note that a focused feature +specification may define immediate update-output bytes for its capability or +resize steps; the Image Specification exercises this for graphics cleanup +(§10.5, §11.2). No foundation behavior changes. + +## Open Decisions + +The following are open. They are deliberately not resolved by this draft. + +1. **The ascii candidate set.** §9.3.3 proposes all 95 ASCII printable + characters. The alternative is a curated subset favoring shape-stable + glyphs (`.`, `:`, `-`, `/`, `T`, `O`, …), trading matching precision for + robustness against font variation between the baked rasterization and the + host terminal. The set is a closed normative table; changing it later is a + versioned spec change, not a tweak. +2. **Sampling-circle geometry and contrast constants.** §9.3.2 and §9.3.4 fix + the structure (six staggered internal circles, ten external circles, the + affecting-external table, directional then global enhancement, + per-component max normalization) but defer exact centers, radii, orderings, + and exponents to calibration against the reference implementation, locked + by golden tests in the implementation PR. +3. **Alpha in glyph sampling.** §9.3.1 ignores alpha when sampling glyph + lightness; transparency is decided per cell by the alpha sample. The + alternative — alpha-weighting the luma samples or compositing semi-alpha + color over the element background — would render semi-transparent regions + blended, at the cost of coupling glyph choice to a channel the color path + otherwise ignores. +4. **`removeImage()` return value.** §7.2 returns delete bytes immediately. + The alternative — defer all cleanup to the next render — keeps + `removeImage` void but leaves graphics on screen if the caller never + renders again. +5. **Quiet mode strictness.** INV-I5 fixes `q=2` (all responses suppressed), + so protocol failures are invisible to v1. A debug-mode `q=1` (suppress OK + only) would surface failures — as APC replies the input parser would need + to route somewhere safe — at the cost of violating the silence guarantee + INV-I5 exists for. +6. **Image-id cohabitation and APC passthrough.** Registry ids map directly + onto wire image ids (§9.2.1). A Term sharing its screen with another + program that also uses kitty image ids could collide. Reserving a base + offset or negotiating ids (the protocol's `I=` numbering) is possible; v1 + declares the single-owner screen assumption instead. The probe's own + graphics query is harmless to this assumption: it is a query action + (`a=q`), which per the protocol "will not replace an existing image with + the same id, nor will it store the image" — re-probes leave no image + behind and collide with nothing, so no id reservation exists in v1. + Multiplexers are the related hazard: a mux that swallows APC bytes + silently degrades the ladder to ascii (no probe response arrives), which + is correct behavior; a passthrough mux that mishandles graphics itself is + not detectable in v1. +7. **Pixel-pool capacity policy.** §5.3 makes the pool a `createTerm` option + (default 4 MiB) with compaction-only reclamation. Alternatives: a fixed + normative constant (no option, less API surface, more waste for non-image + apps), or size-classed blocks instead of contiguous extents (no compaction + cost, more access complexity). +8. **Term teardown.** There is no disposal callback; graphics placed by a + discarded Term are the caller's cleanup (§4.3). A `term.dispose()` + returning deletion bytes is the natural completion of the lifecycle and is + deliberately out of scope until the foundation defines Term disposal. +9. **Terminal-state desync.** Caller-owned terminal operations clear images + behind the renderer's wire state: a reset (RIS) clears all images, an + alternate-screen switch clears the screen's images (mode 1049), and the + clear-screen escape (`ED 2J`) is specified to clear images — all + caller-managed operations the wire state does not observe. v1 documents + the ownership boundary (§4.3) and does not detect resets. Candidate + future mitigations: a post-reset companion routine, or re-probe-driven + re-sync. Undecided because resets are caller-owned terminal state, the + same class of problem as alt-screen ownership. From 61e15fd7db418be185e9dc38c1dc489c6cdcee50 Mon Sep 17 00:00:00 2001 From: Nate Moore Date: Fri, 9 Oct 2026 10:07:41 -0500 Subject: [PATCH 2/4] spec: reframe image spec as the graphics substrate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rename to specs/graphics-spec.md and reframe: the pixel-surface registry plus the tier engine is the substrate (§4.5), img is its first consumer, and video/svg/3d/canvas are producers or later focused specs on the same substrate - the renderer stays a transcoder and never grows a drawing API. Canvas's zero-copy surface views lean on INV-I8's fixed carve; kitty animation frames are the deferred fast path for video. No normative changes to the arena-synthesized contract. --- specs/{image-spec.md => graphics-spec.md} | 113 ++++++++++++++++------ 1 file changed, 85 insertions(+), 28 deletions(-) rename specs/{image-spec.md => graphics-spec.md} (93%) diff --git a/specs/image-spec.md b/specs/graphics-spec.md similarity index 93% rename from specs/image-spec.md rename to specs/graphics-spec.md index b4ab9ad..f957870 100644 --- a/specs/image-spec.md +++ b/specs/graphics-spec.md @@ -1,23 +1,30 @@ -# Clayterm Image Specification +# Clayterm Graphics Specification -**Version:** 0.1 (draft) **Status:** Proposed. Normative for the image element -upon adoption. +**Version:** 0.1 (draft) **Status:** Proposed. Normative for the graphics +substrate and the image element upon adoption. --- ## 1. Purpose -This specification defines Clayterm's image element: a declarative, void -directive that renders a caller-supplied raster image into the terminal, -degrading through a fidelity ladder of output tiers — Kitty graphics protocol -pixels, shape-aware ASCII character art, and alt text. - -The image element fulfills the commitment made in -[Renderer Specification](renderer-spec.md) §7.8, which defers "Kitty graphics -emission" to a focused feature specification. It consumes the terminfo -foundation (PR stack #131–#133): tier selection reads the `RuntimeCapabilities` -snapshot that `term.update()` maintains, and the kitty tier is gated on the -`kitty-graphics` capability probe (query 9). +This specification defines Clayterm's graphics substrate and its first +element. The substrate is a per-Term registry of **pixel surfaces** — +caller-supplied decoded raster buffers — plus the engine that downsamples a +surface onto the terminal through a fidelity ladder of output tiers: Kitty +graphics protocol pixels, shape-aware ASCII character art, and alt text. The +first element on the substrate is the declarative, void `img()` directive. + +The substrate is designed for more than images. Higher-level bitmap +producers — video, SVG, 3D rasterizers, a draw-canvas — are consumers of the +same surface registry and tier engine, not renderer features; §4.5 fixes +that layering as part of this specification's contract. + +The image element — this specification's first consumer — fulfills the +commitment made in [Renderer Specification](renderer-spec.md) §7.8, which +defers "Kitty graphics emission" to a focused feature specification. It +consumes the terminfo foundation (PR stack #131–#133): tier selection reads +the `RuntimeCapabilities` snapshot that `term.update()` maintains, and the +kitty tier is gated on the `kitty-graphics` capability probe (query 9). The division of labor follows the renderer's founding invariants: @@ -36,6 +43,8 @@ The division of labor follows the renderer's founding invariants: ### In scope (normative) +- The graphics substrate: the pixel-surface registry, the tier engine, and + the layering commitment of §4.5 - The `img()` directive: its shape, its void (self-closing) form, and its validation rules - The image registry: `term.setImage()` and `term.removeImage()` @@ -74,6 +83,9 @@ The division of labor follows the renderer's founding invariants: scroll API exists; renderer-spec §14) - Transitions on image properties - Cropping, source rectangles, and sub-image placement +- The canvas element, its 2D context, and other producers (video, svg, 3D): + §4.5 fixes their mapping onto the substrate; each lands as its own focused + specification ### Out of scope (indefinitely) @@ -187,7 +199,8 @@ implementation detail invisible to the caller except through reduced output size and correct cleanup: 1. The front/back cell buffers used for diffing (pre-existing). -2. The image registry: caller-uploaded pixel data, keyed by registry id (§5.3). +2. The image registry: the substrate's caller-owned pixel surfaces, keyed by + registry id (§5.3). 3. The **placements table**: a fixed-capacity record of the previous frame's live kitty placements — for each, its registry id, wire image id, placement id, painted footprint box, tier, and transmission state. It exists so that @@ -208,6 +221,41 @@ registry state, the placements table, and the front cell buffer — no ambient state, no clock, and no heuristic beyond the evidence table (§9.1) and the damage rules (§10). +### 4.5 Substrate and consumers (layering) + +_This subsection is normative for the layering commitment; the per-consumer +sketches are non-normative._ + +The substrate is element-agnostic. Nothing in the tier engine (§9), the +memory carve (§5), the cell-buffer integration (§10), or the invalidation +rules (§11) may learn what produced a surface's pixels — not its format, not +its producer, not whether it is a still image or one frame of many. Every +consumer below is therefore a *producer of surface pixels plus ordinary +directives*; the engine stays a transcoder and never grows a drawing API. + +- **img (this specification).** A decoded raster surface plus a declarative + element with alt-text semantics — the reference consumer, specified in + §6–§8. +- **video.** A producer that re-fills a surface per frame (`setImage` per + tick). The substrate already supports it: a revision bump re-transmits and + re-places (§9.2.2; §15's streaming note) with no new API. The Kitty + protocol's animation machinery — multi-frame images, frame composition, + playback controls — is the protocol's designed fast path for this and is + deferred to a focused amendment (§16.8). +- **svg / vector / 3D.** Rasterizer producers: the caller rasterizes — with + any TypeScript-side library, including a WebGL offscreen surface read back + via `readPixels` — and fills surfaces through `setImage`. The renderer + gains nothing: no path, stroke, tessellation, or scene-graph machinery + ever enters the engine. +- **canvas (next focused specification).** A directive exposing a surface as + a layout-participating draw target, plus a user-facing 2D context that is + pure TypeScript drawing through zero-copy views into the carved pool. + INV-I8 is what makes this sound: the pool is carved once and never grows + or moves at runtime, so surface views stay valid between resizes, and + dirty-rect re-emission extends the revision model additively. The canvas + element and context API land as their own focused specification amending + §5.3 with an in-place mutation path; nothing here needs to change first. + ### 4.2 Data flow ``` @@ -1317,11 +1365,13 @@ data version. All costs are bounded by registry image sizes, not by frame rate. **Streaming video ergonomics.** The registry's replace-on-id path is the -natural video-streaming API (one `setImage` per frame). Each replacement +natural video-streaming API (one `setImage` per frame; §4.5's video +producer). Each replacement costs a delete/re-transmit/re-place cycle (§9.2.2); terminals batch this within a write burst, but the protocol does not guarantee flicker-free updates. Per-frame streaming ergonomics are revisited with §16.4 -(placeholders), which would make moves pure cell diffs. +(placeholders) and §16.8 (animation frames), which would make moves pure +cell diffs and updates re-transmission-free. **Cursor safety.** `C=1` keeps the cursor at the placement origin; combined with §10.6's post-frame restore, no frame leaves the cursor somewhere the @@ -1397,10 +1447,17 @@ could be added as a directive property in a future amendment, ascii-only. ### 16.8 Animation, cropping, transitions -Multi-frame formats, source rectangles, and transitions on image geometry are -not designed here. The directive model leaves room: `image` could become a -richer descriptor and `transition` could apply to footprint axes without -breaking §7.1's shape. +Under §4.5's layering, video is a per-frame revision-bump producer and the +substrate supports it today — but per-frame full re-transmission does not +flicker-free. The Kitty protocol's own animation machinery is its designed +fast path: multi-frame images, frame composition (delta frames), and +playback controls, uploaded once and played back without re-transmission. +Landing it is a focused amendment; it is a protocol sub-feature, and §9.1 +does not verify sub-features in v1. Multi-frame source formats, cropping +source rectangles, and transitions on image geometry are also not designed +here. The directive model leaves room: `image` could become a richer +descriptor and `transition` could apply to footprint axes without breaking +§7.1's shape. ### 16.9 Synchronized-output wrapping @@ -1417,22 +1474,22 @@ approval the amendments land as their own spec-only commit._ the renderer retains between frames is the cell buffer used for diffing" to "the only state the renderer retains between frames is diffing state: the cell buffers, the image registry, and the previous frame's placements table -(as defined in the Image Specification §4.1)". Frame-snapshot independence of +(as defined in the Graphics Specification §4.1)". Frame-snapshot independence of the directive array is unchanged. **A2. Renderer-spec §7.6 (cursor carve-out).** Append to the cursor visibility/positioning section: frames containing image elements at a graphics tier MAY additionally include cursor-positioning bytes required by -the graphics protocol (Image Specification §10.6), provided the post-frame +the graphics protocol (Graphics Specification §10.6), provided the post-frame cursor visibility state defined here is unchanged and caret-declaring frames still end with the caret's cell. **A3. Renderer-spec §8.3 (directive constructors).** Add §8.3.5 `img()` -deferring normatively to the Image Specification, noting the void-directive +deferring normatively to the Graphics Specification, noting the void-directive form and that `img` values do not participate in open/close balance. **A4. Renderer-spec §7.8 (capability consumption).** Remove "Kitty graphics -emission" from the deferred list and reference the Image Specification as the +emission" from the deferred list and reference the Graphics Specification as the focused feature specification that defines it. **A5. Renderer-spec §12.1/§12.2 (current surface).** Note the `img` opcode in @@ -1444,9 +1501,9 @@ surface per §5's contract-layer boundary. Clayterm-specific error types. **A7. Renderer-spec §8.1 (createTerm options).** Add the optional -`imagePoolBytes` option (Image Specification §5.3), default 4 MiB, sizing the +`imagePoolBytes` option (Graphics Specification §5.3), default 4 MiB, sizing the carved image registry. The memory carve and capacity behavior are normative -in the Image Specification; renderer-spec §2 keeps WASM memory layout out of +in the Graphics Specification; renderer-spec §2 keeps WASM memory layout out of scope beyond such behavioral requirements. **A8. Terminfo specification.** No changes in v1. The kitty tier consumes the @@ -1461,7 +1518,7 @@ keeps the renderer's emissions from generating any. **A10. Renderer-spec §7.7 (update transaction).** Note that a focused feature specification may define immediate update-output bytes for its capability or -resize steps; the Image Specification exercises this for graphics cleanup +resize steps; the Graphics Specification exercises this for graphics cleanup (§10.5, §11.2). No foundation behavior changes. ## Open Decisions From df557df9366abc8718d7b8202ec8fdf190d4ac73 Mon Sep 17 00:00:00 2001 From: Nate Moore Date: Fri, 9 Oct 2026 13:36:44 -0500 Subject: [PATCH 3/4] spec: apply graphics amendments, add clip containment gate Apply the approved amendments to renderer-spec (retained state, cursor carve-out, img constructor, capability-consumption reference, imagePoolBytes option, update-bytes note) and terminfo-spec (createTerm restatement); the appendix becomes an applied-record, with current-surface notes (A5/A6) phased to the implementation PR. Two spec corrections found while preparing the implementation: the kitty tier gains a containment gate (a partially clipped placement would letterbox inside its box and claim cells its pixels do not fill, violating INV-I4 - falls through to ascii instead, which also resolves scroll containers), and the placements record is explicitly double-buffered like the cell buffers (stale detection diffs the two). Test file renamed to test/graphics.test.ts with the spec. --- specs/graphics-spec.md | 1339 +++++++++++++++++++--------------------- specs/renderer-spec.md | 41 +- specs/terminfo-spec.md | 7 +- 3 files changed, 691 insertions(+), 696 deletions(-) diff --git a/specs/graphics-spec.md b/specs/graphics-spec.md index f957870..790aee8 100644 --- a/specs/graphics-spec.md +++ b/specs/graphics-spec.md @@ -7,24 +7,24 @@ substrate and the image element upon adoption. ## 1. Purpose -This specification defines Clayterm's graphics substrate and its first -element. The substrate is a per-Term registry of **pixel surfaces** — -caller-supplied decoded raster buffers — plus the engine that downsamples a -surface onto the terminal through a fidelity ladder of output tiers: Kitty -graphics protocol pixels, shape-aware ASCII character art, and alt text. The -first element on the substrate is the declarative, void `img()` directive. - -The substrate is designed for more than images. Higher-level bitmap -producers — video, SVG, 3D rasterizers, a draw-canvas — are consumers of the -same surface registry and tier engine, not renderer features; §4.5 fixes -that layering as part of this specification's contract. +This specification defines Clayterm's graphics substrate and its first element. +The substrate is a per-Term registry of **pixel surfaces** — caller-supplied +decoded raster buffers — plus the engine that downsamples a surface onto the +terminal through a fidelity ladder of output tiers: Kitty graphics protocol +pixels, shape-aware ASCII character art, and alt text. The first element on the +substrate is the declarative, void `img()` directive. + +The substrate is designed for more than images. Higher-level bitmap producers — +video, SVG, 3D rasterizers, a draw-canvas — are consumers of the same surface +registry and tier engine, not renderer features; §4.5 fixes that layering as +part of this specification's contract. The image element — this specification's first consumer — fulfills the -commitment made in [Renderer Specification](renderer-spec.md) §7.8, which -defers "Kitty graphics emission" to a focused feature specification. It -consumes the terminfo foundation (PR stack #131–#133): tier selection reads -the `RuntimeCapabilities` snapshot that `term.update()` maintains, and the -kitty tier is gated on the `kitty-graphics` capability probe (query 9). +commitment made in [Renderer Specification](renderer-spec.md) §7.8, which defers +"Kitty graphics emission" to a focused feature specification. It consumes the +terminfo foundation (PR stack #131–#133): tier selection reads the +`RuntimeCapabilities` snapshot that `term.update()` maintains, and the kitty +tier is gated on the `kitty-graphics` capability probe (query 9). The division of labor follows the renderer's founding invariants: @@ -34,7 +34,7 @@ The division of labor follows the renderer's founding invariants: transcodes those pixels into whatever the current terminal can display. - **The renderer owns bytes.** Tier selection, scaling, protocol encoding, placement, cleanup, and cell-buffer integration are renderer concerns. The - caller declares *what* to show and *where*; the renderer decides *how*. + caller declares _what_ to show and _where_; the renderer decides _how_. - **The renderer owns no memory allocator.** All image state lives in regions carved once at Term creation, under the same fixed-capacity discipline as the cell buffers (Section 5). @@ -43,8 +43,8 @@ The division of labor follows the renderer's founding invariants: ### In scope (normative) -- The graphics substrate: the pixel-surface registry, the tier engine, and - the layering commitment of §4.5 +- The graphics substrate: the pixel-surface registry, the tier engine, and the + layering commitment of §4.5 - The `img()` directive: its shape, its void (self-closing) form, and its validation rules - The image registry: `term.setImage()` and `term.removeImage()` @@ -83,19 +83,19 @@ The division of labor follows the renderer's founding invariants: scroll API exists; renderer-spec §14) - Transitions on image properties - Cropping, source rectangles, and sub-image placement -- The canvas element, its 2D context, and other producers (video, svg, 3D): - §4.5 fixes their mapping onto the substrate; each lands as its own focused +- The canvas element, its 2D context, and other producers (video, svg, 3D): §4.5 + fixes their mapping onto the substrate; each lands as its own focused specification ### Out of scope (indefinitely) -- Image decoding (PNG, JPEG, WebP, …) inside the renderer. The renderer - consumes decoded RGBA pixels only (INV-I1). +- Image decoding (PNG, JPEG, WebP, …) inside the renderer. The renderer consumes + decoded RGBA pixels only (INV-I1). - File paths, URLs, or any IO-bearing reference in a directive. Pixels enter through the registry or not at all. -- Screen-reader presentation of the alt tier. The alt tier is a visual - fallback rendered as ordinary cells; accessibility-tree semantics are a - higher-level framework concern. +- Screen-reader presentation of the alt tier. The alt tier is a visual fallback + rendered as ordinary cells; accessibility-tree semantics are a higher-level + framework concern. ## 3. Terminology @@ -109,23 +109,23 @@ integer ids. The registry is a resource store, not UI tree state. **Registry id.** A caller-chosen integer in `[1, 4294967295]` identifying one image in one Term's registry. It is transmitted verbatim as the Kitty graphics -image id (`i=`), whose valid range is identical — the protocol requires -positive ids up to 4294967295 and forbids zero (§9.2.1). +image id (`i=`), whose valid range is identical — the protocol requires positive +ids up to 4294967295 and forbids zero (§9.2.1). **Image data.** Decoded raster pixels: width, height, and a `Uint8Array` of straight (non-premultiplied) RGBA8 bytes, top-to-bottom rows, exactly `width * height * 4` bytes. **Tier.** One of the output forms an image element may take: `kitty` (Kitty -graphics protocol pixels), `ascii` (shape-aware character art in the cell -grid), `alt` (alt text as plain cell text). +graphics protocol pixels), `ascii` (shape-aware character art in the cell grid), +`alt` (alt text as plain cell text). **Ladder.** The fidelity-ordered tier sequence used for automatic resolution: kitty, then ascii, then alt. -**Evidence.** A capability fact (see -[Terminfo Specification](terminfo-spec.md)) that gates a tier. The v1 gate for -the kitty tier is `RuntimeCapabilities.kittyGraphics === true`. +**Evidence.** A capability fact (see [Terminfo Specification](terminfo-spec.md)) +that gates a tier. The v1 gate for the kitty tier is +`RuntimeCapabilities.kittyGraphics === true`. **Layout box.** The element's computed bounding box in character cells, as produced by the layout engine (the `ElementInfo.bounds` value, renderer-spec @@ -137,14 +137,15 @@ for the alt tier, the whole layout box. **Half-row pixel convention.** This specification's source-pixel geometry convention: one source pixel is one cell column wide and one half-row tall, so -one cell is two pixels tall and the cell aspect ratio `A = cell_height / -cell_width = 2` (height:width 2:1). The convention assumes a typical terminal -cell; §16.3 refines it with cell-size geometry evidence when that evidence -exists. +one cell is two pixels tall and the cell aspect ratio +`A = cell_height / +cell_width = 2` (height:width 2:1). The convention assumes a +typical terminal cell; §16.3 refines it with cell-size geometry evidence when +that evidence exists. **Contain.** Scaling semantics fixed for v1: the image is rendered into the -largest rectangle within its layout box that preserves its aspect ratio, centered -on both axes (§8.2). There is no stretch mode in v1 (§16.7). +largest rectangle within its layout box that preserves its aspect ratio, +centered on both axes (§8.2). There is no stretch mode in v1 (§16.7). **Placement.** One terminal-side display of one kitty image at one cell box, identified on the wire by the pair (image id, placement id). Placements persist @@ -162,9 +163,9 @@ change. region. The mechanism is implementation detail; the described approach is overwriting front-buffer cells with the default cell value (§10.4). -**Quiet emission.** Emitting protocol bytes in a mode that provably generates -no terminal responses, so the input parser's `scan()` stream stays clean -(INV-I5; input-spec §6.2 is the consumer that must not be disturbed). +**Quiet emission.** Emitting protocol bytes in a mode that provably generates no +terminal responses, so the input parser's `scan()` stream stays clean (INV-I5; +input-spec §6.2 is the consumer that must not be disturbed). **Sampling circles.** Fixed cell-relative regions — six internal, ten external — over which both candidate characters and source-image cells are measured @@ -180,8 +181,7 @@ source image over the same circles, plus luma means over the external circles for boundary awareness (§9.3.1). **Contrast enhancement.** The pre-matching transform on sampling vectors — -directional, then global — that sharpens region boundaries in the art -(§9.3.4). +directional, then global — that sharpens region boundaries in the art (§9.3.4). ## 4. Architectural Model @@ -189,14 +189,14 @@ _This section is normative._ ### 4.1 Relationship to the frame-snapshot model -The directive array continues to fully describe each frame's UI tree. Pixels -are not part of the tree; they are resources bound to the Term through the -registry, referenced by id from `img` directives — the same relationship -dimensions and capabilities have to the Term. +The directive array continues to fully describe each frame's UI tree. Pixels are +not part of the tree; they are resources bound to the Term through the registry, +referenced by id from `img` directives — the same relationship dimensions and +capabilities have to the Term. The renderer retains the following state between render transactions, all of it -implementation detail invisible to the caller except through reduced output -size and correct cleanup: +implementation detail invisible to the caller except through reduced output size +and correct cleanup: 1. The front/back cell buffers used for diffing (pre-existing). 2. The image registry: the substrate's caller-owned pixel surfaces, keyed by @@ -204,57 +204,56 @@ size and correct cleanup: 3. The **placements table**: a fixed-capacity record of the previous frame's live kitty placements — for each, its registry id, wire image id, placement id, painted footprint box, tier, and transmission state. It exists so that - cleanup and tier switching can be computed without scanning cells and - without storing element-id strings. It is updated during each render - transaction exactly as the front buffer is (§5.4). + cleanup and tier switching can be computed without scanning cells and without + storing element-id strings. It is updated during each render transaction + exactly as the front buffer is (§5.4). Items 2 and 3 extend what renderer-spec §4.3 enumerates as retained state; the -corresponding amendment to INV-3 is listed in Appendix A. None of this state -is a UI tree: no parent pointers, no component identity, no reconciliation, no +corresponding amendment to INV-3 is listed in Appendix A. None of this state is +a UI tree: no parent pointers, no component identity, no reconciliation, no cross-frame directive memory. The placements table is derived from and bounded -by the previous frame's directives, exactly like the cell buffer. This -preserves the spirit of renderer-spec INV-3 and §10.4 (no cross-frame identity) -while admitting the resource-and-record state that graphics cleanup requires. +by the previous frame's directives, exactly like the cell buffer. This preserves +the spirit of renderer-spec INV-3 and §10.4 (no cross-frame identity) while +admitting the resource-and-record state that graphics cleanup requires. Consequently, tier selection, footprint computation, and all emission decisions -are pure functions of the directive array, the capability snapshot, the -registry state, the placements table, and the front cell buffer — no ambient -state, no clock, and no heuristic beyond the evidence table (§9.1) and the -damage rules (§10). +are pure functions of the directive array, the capability snapshot, the registry +state, the placements table, and the front cell buffer — no ambient state, no +clock, and no heuristic beyond the evidence table (§9.1) and the damage rules +(§10). ### 4.5 Substrate and consumers (layering) _This subsection is normative for the layering commitment; the per-consumer sketches are non-normative._ -The substrate is element-agnostic. Nothing in the tier engine (§9), the -memory carve (§5), the cell-buffer integration (§10), or the invalidation -rules (§11) may learn what produced a surface's pixels — not its format, not -its producer, not whether it is a still image or one frame of many. Every -consumer below is therefore a *producer of surface pixels plus ordinary -directives*; the engine stays a transcoder and never grows a drawing API. +The substrate is element-agnostic. Nothing in the tier engine (§9), the memory +carve (§5), the cell-buffer integration (§10), or the invalidation rules (§11) +may learn what produced a surface's pixels — not its format, not its producer, +not whether it is a still image or one frame of many. Every consumer below is +therefore a _producer of surface pixels plus ordinary directives_; the engine +stays a transcoder and never grows a drawing API. - **img (this specification).** A decoded raster surface plus a declarative - element with alt-text semantics — the reference consumer, specified in - §6–§8. -- **video.** A producer that re-fills a surface per frame (`setImage` per - tick). The substrate already supports it: a revision bump re-transmits and - re-places (§9.2.2; §15's streaming note) with no new API. The Kitty - protocol's animation machinery — multi-frame images, frame composition, - playback controls — is the protocol's designed fast path for this and is - deferred to a focused amendment (§16.8). -- **svg / vector / 3D.** Rasterizer producers: the caller rasterizes — with - any TypeScript-side library, including a WebGL offscreen surface read back - via `readPixels` — and fills surfaces through `setImage`. The renderer - gains nothing: no path, stroke, tessellation, or scene-graph machinery - ever enters the engine. -- **canvas (next focused specification).** A directive exposing a surface as - a layout-participating draw target, plus a user-facing 2D context that is - pure TypeScript drawing through zero-copy views into the carved pool. - INV-I8 is what makes this sound: the pool is carved once and never grows - or moves at runtime, so surface views stay valid between resizes, and - dirty-rect re-emission extends the revision model additively. The canvas - element and context API land as their own focused specification amending - §5.3 with an in-place mutation path; nothing here needs to change first. + element with alt-text semantics — the reference consumer, specified in §6–§8. +- **video.** A producer that re-fills a surface per frame (`setImage` per tick). + The substrate already supports it: a revision bump re-transmits and re-places + (§9.2.2; §15's streaming note) with no new API. The Kitty protocol's animation + machinery — multi-frame images, frame composition, playback controls — is the + protocol's designed fast path for this and is deferred to a focused amendment + (§16.8). +- **svg / vector / 3D.** Rasterizer producers: the caller rasterizes — with any + TypeScript-side library, including a WebGL offscreen surface read back via + `readPixels` — and fills surfaces through `setImage`. The renderer gains + nothing: no path, stroke, tessellation, or scene-graph machinery ever enters + the engine. +- **canvas (next focused specification).** A directive exposing a surface as a + layout-participating draw target, plus a user-facing 2D context that is pure + TypeScript drawing through zero-copy views into the carved pool. INV-I8 is + what makes this sound: the pool is carved once and never grows or moves at + runtime, so surface views stay valid between resizes, and dirty-rect + re-emission extends the revision model additively. The canvas element and + context API land as their own focused specification amending §5.3 with an + in-place mutation path; nothing here needs to change first. ### 4.2 Data flow @@ -279,28 +278,27 @@ img directives ──▶ render() ─▶ tier resolution (§9.1) replaces a live id, bumps the entry's data version (§4.3). - `removeImage()` returns cleanup bytes where the terminal state requires them and dirty-marks previously covered cells for the next render (§7.2). -- The first render transaction that places an image at the kitty tier - transmits its data. Transmission is keyed by per-entry transmission state - (§9.2.2), so unchanged, still-persisted data is never re-sent. +- The first render transaction that places an image at the kitty tier transmits + its data. Transmission is keyed by per-entry transmission state (§9.2.2), so + unchanged, still-persisted data is never re-sent. - All graphics emission happens inside the normal render transaction's byte output. There are no side-channel writes (renderer-spec INV-1, INV-2). ### 4.3 Registry lifecycle - `setImage(id, data)` inserts or replaces. Replacing a live id bumps its data - version. The pixels are copied into the carved pixel pool; the caller's - buffer is not retained (§5.3). Storage failure raises `RangeError` (§7.2). + version. The pixels are copied into the carved pixel pool; the caller's buffer + is not retained (§5.3). Storage failure raises `RangeError` (§7.2). - `removeImage(id)` removes the registry entry, returns kitty deletion bytes - when terminal-side state may exist (§7.2), and dirty-marks every cell in - every footprint that referenced the id so the next render repaints them. - Removing an unknown id is a no-op returning empty bytes. + when terminal-side state may exist (§7.2), and dirty-marks every cell in every + footprint that referenced the id so the next render repaints them. Removing an + unknown id is a no-op returning empty bytes. - **Terminal-side persistence.** Kitty images and placements persist on the terminal after the Term that placed them is discarded. Clayterm has no Term disposal callback (renderer-spec §7.4), so images that outlive their Term are - the caller's to clean up — for example by deleting placements before - teardown, or by exiting the alternate screen. This matches the existing - boundary: the renderer emits bytes, the caller owns terminal state - (renderer-spec §11.2). + the caller's to clean up — for example by deleting placements before teardown, + or by exiting the alternate screen. This matches the existing boundary: the + renderer emits bytes, the caller owns terminal state (renderer-spec §11.2). ## 5. Memory Model @@ -317,29 +315,29 @@ fixed-capacity memory plan: sized by a pre-computed size query over the requested dimensions and options (§5.2). Region carving is descriptive; the behavioral requirements are normative. -- Every image-side structure is a **fixed-capacity array or a bump-carved - extent within a pre-sized region**. There is no dynamic allocation, no +- Every image-side structure is a **fixed-capacity array or a bump-carved extent + within a pre-sized region**. There is no dynamic allocation, no free-and-reallocate of per-image memory, and no per-frame scratch region. - When a fixed-capacity structure is exhausted, the behavior is the closed - failure behavior of §7.2 and §13 — never silent truncation and never - eviction of caller-owned resources (INV-I2). + failure behavior of §7.2 and §13 — never silent truncation and never eviction + of caller-owned resources (INV-I2). - Reclamation of image pixel bytes happens only by **compaction of the pixel - pool** (§5.3): dead extents are reclaimed by copying live entries to the - pool start and resetting the bump pointer. Compaction is deterministic, + pool** (§5.3): dead extents are reclaimed by copying live entries to the pool + start and resetting the bump pointer. Compaction is deterministic, order-preserving for live entries, and triggered by `setImage` on demand. - All image state lives on the Term's C-side state struct (pointers into the - carve), never in global mutable state. Multiple Terms in one process are - fully isolated. + carve), never in global mutable state. Multiple Terms in one process are fully + isolated. - Resize (renderer-spec §7.7) re-carves the dimension-dependent regions (cell buffers, placements table) and grows linear memory if required. The registry and pixel pool are **independent of terminal dimensions** and MUST survive a resize unchanged: an image registered before a resize remains registered and - renderable after it. The placements table is discarded by resize together - with the other diff state, with the consequence documented in §10.5. -- `setImage` and `removeImage` MUST NOT grow linear memory and MUST NOT create - a new WASM module or instance. The reason is view safety: growing linear - memory detaches `Uint8Array` views over it, and output views are known-valid - only until the next `render()` or `update()` call (renderer-spec §7.3) — a + renderable after it. The placements table is discarded by resize together with + the other diff state, with the consequence documented in §10.5. +- `setImage` and `removeImage` MUST NOT grow linear memory and MUST NOT create a + new WASM module or instance. The reason is view safety: growing linear memory + detaches `Uint8Array` views over it, and output views are known-valid only + until the next `render()` or `update()` call (renderer-spec §7.3) — a registry-triggered grow would silently detach output views outside those transactions, where nothing in the contract permits it. Memory growth is a Term-creation and resize concern only (renderer-spec §7.7 discipline). @@ -356,7 +354,8 @@ linear memory at Term creation: │ Clay arena (fixed capacity; bump) │ │ image entry table (IMAGE_ENTRY_CAP × entry, §5.3) │ │ image pixel pool (imagePoolBytes, §5.3) │ -│ placements table (PLACEMENT_CAP × entry, §5.4) │ +│ placements tables (double-buffered, PLACEMENT_CAP │ +│ × entry each, §5.4) │ └────────────────────────────────────────────────────────────┘ ``` @@ -372,42 +371,43 @@ The registry is a fixed-capacity entry table plus a shared pixel pool. - `IMAGE_ENTRY_CAP` is 256 entries (normative). Each entry holds: registry id, data version, width, height, pool offset, payload byte length, transmission state (§9.2.2), and an in-use flag. -- `imagePoolBytes` is a `createTerm` option (Appendix A7) with normative - default `4194304` (4 MiB). The pixel pool is one contiguous region of that - size. +- `imagePoolBytes` is a `createTerm` option (Appendix A7) with normative default + `4194304` (4 MiB). The pixel pool is one contiguous region of that size. - `setImage(id, data)` copies the payload into the pool: the required - `width*height*4` bytes are bump-carved from the pool. If the pool's - remaining space cannot hold the payload, the renderer first **compacts** - (copies all live payloads to the pool start in offset order, resets the bump - pointer, and rewrites live entries' offsets) and retries. If the payload - still does not fit, `setImage` throws `RangeError` identifying the capacity - condition (§7.2). + `width*height*4` bytes are bump-carved from the pool. If the pool's remaining + space cannot hold the payload, the renderer first **compacts** (copies all + live payloads to the pool start in offset order, resets the bump pointer, and + rewrites live entries' offsets) and retries. If the payload still does not + fit, `setImage` throws `RangeError` identifying the capacity condition (§7.2). - Replacing a live id allocates the new payload and marks the old extent dead; - the dead extent is reclaimed at the next compaction. Until then the pool - holds both payloads. This is observable only through earlier capacity - exhaustion and is accepted behavior. -- An entry's data version is a monotonically increasing counter, incremented - on every replace. It is the deduplication key for transmission (§9.2.2) and - the memoization key for ascii art (§15). -- The renderer MUST NOT evict, expire, or garbage-collect live entries - (INV-I2). Removal is explicit via `removeImage`, or by the Term being - discarded. + the dead extent is reclaimed at the next compaction. Until then the pool holds + both payloads. This is observable only through earlier capacity exhaustion and + is accepted behavior. +- An entry's data version is a monotonically increasing counter, incremented on + every replace. It is the deduplication key for transmission (§9.2.2) and the + memoization key for ascii art (§15). +- The renderer MUST NOT evict, expire, or garbage-collect live entries (INV-I2). + Removal is explicit via `removeImage`, or by the Term being discarded. ### 5.4 Placements table _This subsection is normative for behavior._ -- `PLACEMENT_CAP` is 64 entries (normative). Each entry holds: registry id, - wire image id, placement id, the painted footprint box, the resolved tier, - and the data version last transmitted. +- `PLACEMENT_CAP` is 64 entries (normative). Each entry holds: registry id, wire + image id, placement id, the painted footprint box, the resolved tier, and the + data version last transmitted. - At the end of every render transaction, the table is rewritten to describe exactly the kitty placements that transaction emitted, in previous-frame - directive order — the same retained-state discipline as the front buffer. -- When a frame's kitty placements exceed the capacity, the excess image - elements are **degraded to the ascii tier for that frame** in directive - order, and one `"IMAGE_PLACEMENTS_EXCEEDED"` error is surfaced (§13). No - kitty placement is ever created without a table entry: cleanup correctness - (INV-I3) outranks fidelity. + directive order — the same retained-state discipline as the front buffer. Like + the cell buffers, the placements record is **double-buffered**: the previous + frame's placements (front) and the frame being built (back); stale-placement + detection diffs the two tables (§10), and the tables swap when the front + buffer does. +- When a frame's kitty placements exceed the capacity, the excess image elements + are **degraded to the ascii tier for that frame** in directive order, and one + `"IMAGE_PLACEMENTS_EXCEEDED"` error is surfaced (§13). No kitty placement is + ever created without a table entry: cleanup correctness (INV-I3) outranks + fidelity. - The table stores no element-id strings. Placement identity is the wire pair (image id, placement id) derived per §9.2.3. @@ -418,16 +418,16 @@ _This section is normative for behavior._ There is no per-frame scratch region. Every per-frame image computation is either: -- **O(1) working state** (stack locals): base64 encoder state, per-cell - sampling accumulators (six internal + ten external luma means), contrast - vector components, and the footprint arithmetic; or +- **O(1) working state** (stack locals): base64 encoder state, per-cell sampling + accumulators (six internal + ten external luma means), contrast vector + components, and the footprint arithmetic; or - **Direct output**: base64 encoding streams into the carved output buffer at the current write cursor; ascii sampling writes directly into the back cell buffer; per-cell work never buffers more than one cell. -The renderer MUST NOT build per-frame intermediate buffers for image -processing. This is what keeps the ascii pipeline inside the no-malloc -discipline: sampling, matching, and cell emission all stream. +The renderer MUST NOT build per-frame intermediate buffers for image processing. +This is what keeps the ascii pipeline inside the no-malloc discipline: sampling, +matching, and cell emission all stream. ### 5.6 Output-buffer sizing for transmission @@ -435,19 +435,20 @@ _This section is normative for behavior._ The output buffer's capacity MUST be at least the sum of: -1. the foundation's text-redraw capacity (the capacity required by an - ordinary complete redraw at the Term's dimensions), and +1. the foundation's text-redraw capacity (the capacity required by an ordinary + complete redraw at the Term's dimensions), and 2. the worst-case single-frame graphics encoding: the base64 encoding of the - largest payload the pixel pool can hold — `ceil(4/3 × imagePoolBytes)` - bytes — plus, for every 3072 payload bytes (one maximum chunk), at most 128 - bytes of control-block framing. - -Formula: `output_capacity ≥ text_capacity + ⌈4/3 · imagePoolBytes⌉ + 128 · -⌈imagePoolBytes / 3072⌉ + 64` (the final 64 bytes cover the trailing -placement and cursor bytes). If a render transaction nevertheless exceeds the -output capacity, the renderer MUST throw a `RangeError` identifying the -condition as an output-capacity overflow, in the manner of renderer-spec §9.2 -— never a raw host-level TypedArray error. + largest payload the pixel pool can hold — `ceil(4/3 × imagePoolBytes)` bytes + — plus, for every 3072 payload bytes (one maximum chunk), at most 128 bytes + of control-block framing. + +Formula: +`output_capacity ≥ text_capacity + ⌈4/3 · imagePoolBytes⌉ + 128 · +⌈imagePoolBytes / 3072⌉ + 64` +(the final 64 bytes cover the trailing placement and cursor bytes). If a render +transaction nevertheless exceeds the output capacity, the renderer MUST throw a +`RangeError` identifying the condition as an output-capacity overflow, in the +manner of renderer-spec §9.2 — never a raw host-level TypedArray error. ## 6. Core Invariants @@ -459,22 +460,22 @@ formats, fetch, or otherwise perform IO to obtain image content. The directive carries a registry id, never a path. **INV-I2. Caller-owned resources.** The registry holds exactly what the caller -stored, keyed by caller-chosen ids. The renderer MUST NOT evict, expire, -shadow, or garbage-collect live registry entries, and MUST NOT invent images. -Removal is explicit. Capacity exhaustion fails loudly per §7.2/§13. - -**INV-I3. Stateless cleanup.** All graphics cleanup — deleting stale -placements on move, resize, unmount, tier change, data replacement, or image -removal — MUST be computable from the current directive array plus the -retained diff state (cell buffers, registry, placements table). No callbacks, -no cross-frame UI tree, no imperative delete API beyond `removeImage()`. - -**INV-I4. Determinism and measurement agreement.** Tier selection and -footprint computation are pure functions of the directive, the capability -snapshot, the registry state, and the layout box. The painted footprint -computed for layout purposes MUST equal the footprint actually painted, for -every tier, on every capability state. An image MUST NOT claim cells it does -not paint, nor paint cells it does not claim. +stored, keyed by caller-chosen ids. The renderer MUST NOT evict, expire, shadow, +or garbage-collect live registry entries, and MUST NOT invent images. Removal is +explicit. Capacity exhaustion fails loudly per §7.2/§13. + +**INV-I3. Stateless cleanup.** All graphics cleanup — deleting stale placements +on move, resize, unmount, tier change, data replacement, or image removal — MUST +be computable from the current directive array plus the retained diff state +(cell buffers, registry, placements table). No callbacks, no cross-frame UI +tree, no imperative delete API beyond `removeImage()`. + +**INV-I4. Determinism and measurement agreement.** Tier selection and footprint +computation are pure functions of the directive, the capability snapshot, the +registry state, and the layout box. The painted footprint computed for layout +purposes MUST equal the footprint actually painted, for every tier, on every +capability state. An image MUST NOT claim cells it does not paint, nor paint +cells it does not claim. **INV-I5. Quiet emission and parser independence.** Every graphics-protocol control block the renderer emits MUST set the quiet flag (`q=2`), which @@ -485,34 +486,33 @@ renderer and input parser's architectural independence (renderer-spec INV-7): the renderer consumes capability facts, and nothing it emits is shaped to be parsed by the input parser. -**INV-I6. Cell-buffer honesty.** While an image covers cells at a graphics -tier, the cell buffer's content for those cells MUST be the covered-cell value -(§10.1) and MUST NOT contain other elements' content: pixel-tier coverage wins -over text (§10.3). When coverage appears, moves, or disappears, the renderer -MUST delete stale graphics state terminal-side and restore the affected cells -so that, after the caller writes the transaction's bytes, the terminal's -visible state matches the cell buffer. +**INV-I6. Cell-buffer honesty.** While an image covers cells at a graphics tier, +the cell buffer's content for those cells MUST be the covered-cell value (§10.1) +and MUST NOT contain other elements' content: pixel-tier coverage wins over text +(§10.3). When coverage appears, moves, or disappears, the renderer MUST delete +stale graphics state terminal-side and restore the affected cells so that, after +the caller writes the transaction's bytes, the terminal's visible state matches +the cell buffer. **INV-I7. Transaction boundaries preserved.** `setImage()` and `removeImage()` -are registry operations outside the render transaction. Each render -transaction remains a single TS→WASM crossing (renderer-spec INV-2). Registry -calls MAY cross the boundary individually, but they MUST NOT render, and -render MUST NOT require a preceding registry call beyond what its directives -reference. +are registry operations outside the render transaction. Each render transaction +remains a single TS→WASM crossing (renderer-spec INV-2). Registry calls MAY +cross the boundary individually, but they MUST NOT render, and render MUST NOT +require a preceding registry call beyond what its directives reference. **INV-I8. No-malloc arena discipline.** All renderer-owned image state — -registry, pixel pool, placements table, and any per-frame working memory — -MUST satisfy §5.1: fixed capacities carved at Term creation, bump-carved -extents with compaction-only reclamation, O(1) per-frame scratch, and closed -failure behavior at capacity. The renderer MUST NOT link or invoke a heap -allocator for image state. +registry, pixel pool, placements table, and any per-frame working memory — MUST +satisfy §5.1: fixed capacities carved at Term creation, bump-carved extents with +compaction-only reclamation, O(1) per-frame scratch, and closed failure behavior +at capacity. The renderer MUST NOT link or invoke a heap allocator for image +state. **INV-I9. Atomic failure.** A render transaction that throws MUST leave all retained state — cell buffers, registry, placements table — exactly as it was before the transaction, and MUST return no bytes. A terminal that has received no bytes is consistent with unchanged retained state; the caller may retry the -render after correcting the condition. The same holds for a throwing -registry call: validation MUST occur before any mutation (§7.2). +render after correcting the condition. The same holds for a throwing registry +call: validation MUST occur before any mutation (§7.2). ## 7. Public API Additions @@ -522,22 +522,22 @@ _This section is normative for the shapes shown. Adoption amends ### 7.1 `img()` ```ts -function img(id: string, props: ImgProps): Img +function img(id: string, props: ImgProps): Img; interface ImgProps { /** Registry id of the image to render. Omit for an alt-only element. */ image?: number; /** Required. Text fallback and human description. "" = decorative. */ alt: string; - width?: SizingAxis; // default fit() — see §8.3 - height?: SizingAxis; // default fit() — see §8.3 - variant?: "auto" | "kitty" | "ascii" | "alt"; // default "auto" + width?: SizingAxis; // default fit() — see §8.3 + height?: SizingAxis; // default fit() — see §8.3 + variant?: "auto" | "kitty" | "ascii" | "alt"; // default "auto" /** Element background, same value space as open({ bg }). See §8.4. */ bg?: number; } interface Img { - directive: typeof OP_IMG; // opcode value is implementation surface + directive: typeof OP_IMG; // opcode value is implementation surface id: string; image?: number; alt: string; @@ -548,17 +548,17 @@ interface Img { } ``` -The directive is **void**: one `img()` value in the array is a complete -element. It MUST NOT be paired with `close()` and MUST NOT have children. It -participates in layout as a leaf: its `width`/`height` sizing axes resolve like -any element's (§8), it is a pointer hit-test target under its element `id` -(renderer-spec §12.4), and its layout box is reported by `RenderInfo` -(renderer-spec §12.3). Its `id` MUST be unique among all element ids in the -frame, per renderer-spec §9.3 — duplicate ids are undefined behavior. +The directive is **void**: one `img()` value in the array is a complete element. +It MUST NOT be paired with `close()` and MUST NOT have children. It participates +in layout as a leaf: its `width`/`height` sizing axes resolve like any element's +(§8), it is a pointer hit-test target under its element `id` (renderer-spec +§12.4), and its layout box is reported by `RenderInfo` (renderer-spec §12.3). +Its `id` MUST be unique among all element ids in the frame, per renderer-spec +§9.3 — duplicate ids are undefined behavior. -The directive has no `caret` property. The renderer's hardware-cursor -machinery is text()-bound (renderer-spec §7.6); the only cursor bytes an image -frame may contain are the placement-positioning bytes of §9.2.3. +The directive has no `caret` property. The renderer's hardware-cursor machinery +is text()-bound (renderer-spec §7.6); the only cursor bytes an image frame may +contain are the placement-positioning bytes of §9.2.3. A frame MAY contain any number of `img` directives. Two elements MAY reference the same registry id; each receives its own placement (§9.2.4). @@ -570,8 +570,8 @@ chain (renderer-spec §9.1). ```ts interface ImageData { - width: number; // positive integer - height: number; // positive integer + width: number; // positive integer + height: number; // positive integer /** RGBA8, straight alpha, top-to-bottom rows, width*height*4 bytes */ pixels: Uint8Array; } @@ -589,29 +589,28 @@ interface Term { - `setImage` MUST throw a `RangeError` when: `id` is not an integer in `[1, 4294967295]`; `width` or `height` is not a positive integer; `pixels.length !== width * height * 4`; or storage fails — either no free - entry exists in the registry table, or the pixel pool cannot hold the - payload even after compaction (§5.3). The message SHOULD identify the - condition, mirroring the transfer-buffer error discipline of renderer-spec - §9.2. + entry exists in the registry table, or the pixel pool cannot hold the payload + even after compaction (§5.3). The message SHOULD identify the condition, + mirroring the transfer-buffer error discipline of renderer-spec §9.2. - `setImage` returns `void` and emits nothing. Transmission is lazy (§9.2.2). - `removeImage` returns a `Uint8Array` to write to the terminal immediately — - the same write-now discipline as `update()` output (terminfo-spec TINV-5). - The bytes, when the registry id was live, delete the image, all its - placements, and the terminal's stored copy of the data: one + the same write-now discipline as `update()` output (terminfo-spec TINV-5). The + bytes, when the registry id was live, delete the image, all its placements, + and the terminal's stored copy of the data: one `ESC _ G a=d,d=I,i=,q=2 ESC \` control block. The uppercase form is the protocol's data-freeing deletion; the lowercase variant deletes placements but retains stored data "so that the images can be re-displayed without needing to resend" — retention is what §9.2.5's scoped deletions - exploit, and removal is the one place it is wrong. - Deletion bytes MUST be emitted whenever the id was live in the registry — - regardless of the current `kittyGraphics` evidence — because a placement - placed under earlier evidence may still exist terminal-side after the - evidence was denied (§11.3). The result is empty when the id is unknown. + exploit, and removal is the one place it is wrong. Deletion bytes MUST be + emitted whenever the id was live in the registry — regardless of the current + `kittyGraphics` evidence — because a placement placed under earlier evidence + may still exist terminal-side after the evidence was denied (§11.3). The + result is empty when the id is unknown. - `removeImage` also dirty-marks (§10.4) every cell of every footprint that - referenced the id, so the next render repaints them; this happens even when - no deletion bytes are emitted. -- Both calls are synchronous. Neither renders, and neither performs IO - (INV-I1, INV-I7). + referenced the id, so the next render repaints them; this happens even when no + deletion bytes are emitted. +- Both calls are synchronous. Neither renders, and neither performs IO (INV-I1, + INV-I7). ### 7.3 Validation @@ -621,8 +620,8 @@ interface Term { - `image`, when present, MUST be an integer in `[1, 4294967295]`. - `variant`, when present, MUST be one of the §7.1 literals. - `width`/`height`, when present, MUST be sizing-axis values. -- `bg`, when present, MUST be a number (packed ARGB per `rgba()`, - renderer-spec §8.5). +- `bg`, when present, MUST be a number (packed ARGB per `rgba()`, renderer-spec + §8.5). - `img` directives are not opened or closed; `validate` MUST NOT treat them as contributing to open/close balance. @@ -635,36 +634,34 @@ input is unspecified. ### 8.1 Layout box The layout engine computes the element's layout box from its `width`/`height` -sizing axes like any leaf element's. The layout box is what layout, clip -regions (§8.5), pointer hit-testing, and `RenderInfo` report (renderer-spec -§12.3). +sizing axes like any leaf element's. The layout box is what layout, clip regions +(§8.5), pointer hit-testing, and `RenderInfo` report (renderer-spec §12.3). ### 8.2 Painted footprint (contain) The **painted footprint** is resolved inside the layout box, per tier: -| Tier | Painted footprint | -| ------ | ------------------------------------------------------------- | -| kitty | contain-fit region (below) | -| ascii | contain-fit region (below) | -| alt | the entire layout box | +| Tier | Painted footprint | +| ----- | -------------------------- | +| kitty | contain-fit region (below) | +| ascii | contain-fit region (below) | +| alt | the entire layout box | The contain-fit region of a `W × H` pixel source inside a `boxW × boxH` cell box, under the half-row pixel convention (`A = 2`): - `cols = min(boxW, floor(boxH · A · W / H))` - `rows = min(boxH, round(cols · H / (A · W)))` -- If `cols = 0` or `rows = 0`, the footprint is **empty** and the element - paints nothing at graphics tiers (alt still paints its full box, §9.4). +- If `cols = 0` or `rows = 0`, the footprint is **empty** and the element paints + nothing at graphics tiers (alt still paints its full box, §9.4). - Otherwise the footprint is the `cols × rows` rectangle centered horizontally and vertically in the box, top-left at `x = floor((boxW − cols)/2)`, `y = floor((boxH − rows)/2)` relative to the box origin. -The footprint is what a graphics tier paints and what the ascii tier maps -pixels onto. It is always a subset of the layout box, and it equals the layout -box whenever the box already matches the source's aspect ratio under `A = 2` -(within the rounding above) — which is the case for intrinsic `fit()` sizing -(§8.3). +The footprint is what a graphics tier paints and what the ascii tier maps pixels +onto. It is always a subset of the layout box, and it equals the layout box +whenever the box already matches the source's aspect ratio under `A = 2` (within +the rounding above) — which is the case for intrinsic `fit()` sizing (§8.3). v1 has exactly one scaling semantic. `scale: "stretch"` is not offered; see §16.7 for why (the kitty protocol letterboxes unconditionally when both @@ -673,15 +670,15 @@ tier). ### 8.3 Intrinsic size (`fit`) -`fit()` on either axis resolves to the image's **intrinsic cell size** under -the half-row pixel convention (§3). For image data of `W × H` pixels: +`fit()` on either axis resolves to the image's **intrinsic cell size** under the +half-row pixel convention (§3). For image data of `W × H` pixels: - intrinsic width = `W` cells - intrinsic height = `ceil(H / 2)` cells -This is evidence-free: the convention is an assumption about the host cell, -not a measured value. An `img` with both axes at `fit()` renders at native -terminal pixel density under the assumed cell shape. +This is evidence-free: the convention is an assumption about the host cell, not +a measured value. An `img` with both axes at `fit()` renders at native terminal +pixel density under the assumed cell shape. For an alt-only element (`image` omitted), `fit()` resolves to the alt text's wrapped size, like a `text()` element with wrapping enabled. @@ -700,17 +697,26 @@ wrapped size, like a `text()` element with wrapping enabled. ### 8.5 Clipping The effective clip region rules of renderer-spec §7.5 apply to image elements -unchanged. The element's **effective footprint** is the intersection of its -§8.2 painted footprint with its effective clip region; the effective layout -box is the intersection of its layout box with the region. +unchanged. The element's **effective footprint** is the intersection of its §8.2 +painted footprint with its effective clip region; the effective layout box is +the intersection of its layout box with the region. - When the effective footprint is empty, the element paints nothing: no placement is emitted and any previous placement is stale (§10). - The ascii and alt tiers paint only within the effective region — their - per-cell output is ordinary cells and is suppressed outside it by the - existing clip machinery. -- Placement-origin and `c=`/`r=` values are computed from the effective - footprint, so a clipped image places exactly its visible cells. + per-cell output is ordinary cells and is suppressed outside it by the existing + clip machinery. +- **Containment gate.** A pixel-tier placement paints whole cell boxes the + terminal cannot clip. When the effective footprint is smaller than the painted + footprint — the image is partially clipped — the kitty tier is unavailable for + that element in that frame and resolution falls through to the ascii tier + (§9.1), which renders the visible part as ordinary cells with no + re-measurement: kitty and ascii share the painted footprint (§8.2). This keeps + INV-I4 honest — the terminal letterboxes the transmitted image into the + `c=`/`r=` box whenever its aspect differs, so a clipped placement whose box + aspect no longer matches the image would claim cells its pixels do not fill. + The same rule resolves scroll containers: an image scrolled partially out of a + clipping ancestor renders as ascii art. ## 9. Tier Emission Contract @@ -725,8 +731,8 @@ For each `img` directive, the renderer resolves exactly one tier: evidence gate (§11.1) passes; otherwise resolution proceeds as if `variant` were `"auto"`. 3. Otherwise (`variant` omitted or `"auto"`), the ladder is walked in order: - `kitty` when its evidence gate passes, then `ascii` (always available), - then `alt`. + `kitty` when its evidence gate passes and the containment gate of §8.5 holds, + then `ascii` (always available), then `alt`. The resolution is deterministic (INV-I4): the same directive, capability snapshot, registry state, and placements table always yield the same tier and @@ -741,27 +747,26 @@ specification. Every control block the renderer emits MUST set `q=2` (INV-I5). #### 9.2.1 Wire identities -- The wire **image id** is the registry id, transmitted verbatim. The - protocol's id space — positive integers up to 4294967295, zero forbidden - (a zero or omitted id means "no id") — is exactly the registry id range of - §3, so the identity mapping keeps every registry id addressable with no - transformation to test or explain. -- The wire **placement id** is a pure function of the directive's element id - and the image's registry id, in `[1, 4294967295]`. The protocol treats - `p=0` (and omission) as "unspecified", which would create accidental - additional placements — placement ids MUST NOT be 0. - - Requirements: deterministic (same inputs → same id across frames and - across Terms' sessions with the same element ids), uniform over its range, - and cheap to compute. The specific hash is implementation-defined (§15). +- The wire **image id** is the registry id, transmitted verbatim. The protocol's + id space — positive integers up to 4294967295, zero forbidden (a zero or + omitted id means "no id") — is exactly the registry id range of §3, so the + identity mapping keeps every registry id addressable with no transformation to + test or explain. +- The wire **placement id** is a pure function of the directive's element id and + the image's registry id, in `[1, 4294967295]`. The protocol treats `p=0` (and + omission) as "unspecified", which would create accidental additional + placements — placement ids MUST NOT be 0. + - Requirements: deterministic (same inputs → same id across frames and across + Terms' sessions with the same element ids), uniform over its range, and + cheap to compute. The specific hash is implementation-defined (§15). - Collision handling: two elements whose (element id, registry id) pairs - collide in the hash would share a placement id and each other's cleanup. - The renderer MUST detect same-frame collisions and resolve them - deterministically: the affected elements resolve to the ascii tier for - that frame, emit no placements, and surface - `"IMAGE_PLACEMENT_COLLISION"` per affected element (§13) — a collision - can never leave undeletable graphics debris on the terminal. A - renderer-side id counter would reintroduce cross-frame state and is - deliberately avoided (INV-I3). + collide in the hash would share a placement id and each other's cleanup. The + renderer MUST detect same-frame collisions and resolve them + deterministically: the affected elements resolve to the ascii tier for that + frame, emit no placements, and surface `"IMAGE_PLACEMENT_COLLISION"` per + affected element (§13) — a collision can never leave undeletable graphics + debris on the terminal. A renderer-side id counter would reintroduce + cross-frame state and is deliberately avoided (INV-I3). - Every placement is identified on the wire by the pair (image id, placement id); image ids alone never address a placement this renderer created. @@ -772,32 +777,33 @@ specification. Every control block the renderer emits MUST set `q=2` (INV-I5). - Transmission is **lazy**: it occurs in the render transaction that first places the image, not at `setImage()` time. - **Transmission state.** Each registry entry tracks the data version last - transmitted (0 = never). Before a placement is emitted, the renderer - transmits the image iff `transmitted_version ≠ version`. After a successful + transmitted (0 = never). Before a placement is emitted, the renderer transmits + the image iff `transmitted_version ≠ version`. After a successful transmission, `transmitted_version = version`. -- `transmitted_version` MUST be reset to 0 — forcing re-transmission on the - next placement — when: the image's last live placement is deleted (unmount, - tier switch, collision demotion, registry removal); a `kitty-graphics` - capability denial deletes its placements (§11.2); or the Term is resized - (the placements table, and with it all assumptions about terminal-side - image persistence, is discarded). +- `transmitted_version` MUST be reset to 0 — forcing re-transmission on the next + placement — when: the image's last live placement is deleted (unmount, tier + switch, collision demotion, registry removal); a `kitty-graphics` capability + denial deletes its placements (§11.2); or the Term is resized (the placements + table, and with it all assumptions about terminal-side image persistence, is + discarded). - **Re-transmission is preceded by deletion.** Any transmission of data for a wire image id that may already exist terminal-side MUST be preceded by - `ESC _ G a=d,d=i,i=,q=2 ESC \` (delete image and all its placements). - The protocol requires this for id re-use: re-transmitted data does not - update existing placements, and the old image must be removed first to avoid + `ESC _ G a=d,d=i,i=,q=2 ESC \` (delete image and all its placements). The + protocol requires this for id re-use: re-transmitted data does not update + existing placements, and the old image must be removed first to avoid divergent behavior when ids are re-used in a session. - Payload chunking: chunks MUST be at most 4096 bytes of base64; every chunk - except the last MUST have a length that is a multiple of 4; all chunks - except the last MUST set `m=1`, the last MUST set `m=0`. The first chunk's - control block carries the full transmission keys; continuation chunks carry - only the `m` key, in the protocol's documented form. The renderer MAY choose - smaller chunks. -- The renderer MUST NOT re-transmit image data while `transmitted_version = - version` (INV-I2 discipline: no redundant uploads) — subject to the resets - above, which exist because the protocol's storage quota preferentially - deletes images without live placements, so persisted-data assumptions MUST - be dropped whenever this renderer drops a placement. + except the last MUST have a length that is a multiple of 4; all chunks except + the last MUST set `m=1`, the last MUST set `m=0`. The first chunk's control + block carries the full transmission keys; continuation chunks carry only the + `m` key, in the protocol's documented form. The renderer MAY choose smaller + chunks. +- The renderer MUST NOT re-transmit image data while + `transmitted_version = + version` (INV-I2 discipline: no redundant uploads) — + subject to the resets above, which exist because the protocol's storage quota + preferentially deletes images without live placements, so persisted-data + assumptions MUST be dropped whenever this renderer drops a placement. #### 9.2.3 Placement positioning @@ -813,32 +819,31 @@ cell-addressable placement path, and it is the one v1 uses. - The placement control block is `a=p,i=,p=,c=,r=,C=1,q=2`: the footprint's cell dimensions on `c=`/`r=`, and `C=1` to pin the cursor movement policy — the - protocol's default post-placement cursor jump by the box size would leave - the cursor at an implementation-defined position when the box exits the - screen. `C=1` does not position the placement; the preceding CUP does. -- With both `c=` and `r=` given, the protocol letterboxes to prevent - distortion. Because the footprint is the contain-fit region (§8.2), the - letterbox is a no-op and the terminal's painted pixels fill exactly the - footprint — the agreement INV-I4 requires. -- Placement ids are reused across frames for the same (element id, registry - id). Re-sending a placement with the same (image id, placement id) pair - replaces the previous graphic without flicker — the protocol's documented - move/resize path — and §10.2 uses it normatively: a footprint change of an - unchanged image is a pure re-place with no intervening deletion. + protocol's default post-placement cursor jump by the box size would leave the + cursor at an implementation-defined position when the box exits the screen. + `C=1` does not position the placement; the preceding CUP does. +- With both `c=` and `r=` given, the protocol letterboxes to prevent distortion. + Because the footprint is the contain-fit region (§8.2), the letterbox is a + no-op and the terminal's painted pixels fill exactly the footprint — the + agreement INV-I4 requires. +- Placement ids are reused across frames for the same (element id, registry id). + Re-sending a placement with the same (image id, placement id) pair replaces + the previous graphic without flicker — the protocol's documented move/resize + path — and §10.2 uses it normatively: a footprint change of an unchanged image + is a pure re-place with no intervening deletion. #### 9.2.4 Covered-cell blanking -Cells in the effective footprint are blanked as part of the frame's cell -output — spaces with the element `bg` applied (§8.4) — so no text shows -through transparent pixels and no stale text is trapped under the image -(§10.1). +Cells in the effective footprint are blanked as part of the frame's cell output +— spaces with the element `bg` applied (§8.4) — so no text shows through +transparent pixels and no stale text is trapped under the image (§10.1). #### 9.2.5 Cleanup - Before a frame's cell writes for a region whose placement is stale — the - element unmounted, the tier changed away from kitty, the element demoted by - a placement-id collision (§13), or the image removed or replaced — the - renderer MUST emit deletion of the stale placement: + element unmounted, the tier changed away from kitty, the element demoted by a + placement-id collision (§13), or the image removed or replaced — the renderer + MUST emit deletion of the stale placement: `ESC _ G a=d,d=i,i=,p=,q=2 ESC \`. The lowercase form deletes the placement while retaining the terminal's stored data, which the protocol documents as re-displayable without resending; whether the data survives is @@ -847,37 +852,37 @@ through transparent pixels and no stale text is trapped under the image region. A delete command received while an image's chunked upload is incomplete aborts the partial upload (per the protocol); §9.2.6's deletions-before-transmissions ordering makes that unobservable. -- The renderer MUST NOT rely on the terminal's text-overwrites-graphics - behavior for cleanup; it deletes explicitly (INV-I6). +- The renderer MUST NOT rely on the terminal's text-overwrites-graphics behavior + for cleanup; it deletes explicitly (INV-I6). #### 9.2.6 Frame ordering Within one render transaction's output, the renderer MUST emit, in order: -1. **Stale-placement deletions** — every placement in the placements table - that this frame does not re-create and whose image's data version did not - change, in previous-frame directive order. (Data-version deletions ride - with the transmission block below, not here.) -2. **Transmissions** — for each image requiring (re-)transmission: the - delete-id block, then its chunks, in registry id order. The delete-id - block subsumes any scoped deletions for that image's placements: the - protocol requires existing image and placements to be deleted before - re-transmission, and the same frame re-places them (§10.2). +1. **Stale-placement deletions** — every placement in the placements table that + this frame does not re-create and whose image's data version did not change, + in previous-frame directive order. (Data-version deletions ride with the + transmission block below, not here.) +2. **Transmissions** — for each image requiring (re-)transmission: the delete-id + block, then its chunks, in registry id order. The delete-id block subsumes + any scoped deletions for that image's placements: the protocol requires + existing image and placements to be deleted before re-transmission, and the + same frame re-places them (§10.2). 3. **Cell writes** — the ordinary changed-cell diff bytes, including covered cells' blanks and any repair bytes from §10. 4. **Placements** — for each kitty placement, in directive order: the positioning CUP, then the placement control block. -The two ordering constraints that matter: deletions precede the cell writes -that repaint their footprints, and blanking precedes the placements that -overwrite those blanks. Both hold in the order above. After the last -placement, the renderer MUST restore the post-frame cursor state required by -renderer-spec §7.6 (the §10.6 carve-out). +The two ordering constraints that matter: deletions precede the cell writes that +repaint their footprints, and blanking precedes the placements that overwrite +those blanks. Both hold in the order above. After the last placement, the +renderer MUST restore the post-frame cursor state required by renderer-spec §7.6 +(the §10.6 carve-out). ### 9.3 ASCII tier -The ascii tier renders the image as character art in the cell grid. It uses -only ordinary cells, so it works on every terminal and participates in normal +The ascii tier renders the image as character art in the cell grid. It uses only +ordinary cells, so it works on every terminal and participates in normal diffing. Glyph selection is **shape-aware**, adopting the approach described in @@ -902,8 +907,8 @@ consists of the single pixel at the start coordinates, clamped into the image. Three measurements come from each cell's source rectangle: -- **Color sample** — the per-channel arithmetic mean of the rectangle's - pixels. Drives foreground color (§9.3.5). +- **Color sample** — the per-channel arithmetic mean of the rectangle's pixels. + Drives foreground color (§9.3.5). - **Alpha sample** — the mean alpha of the rectangle's pixels. Drives transparency (§9.3.5). - **Glyph sampling vector** — one luma value per internal sampling circle @@ -919,8 +924,8 @@ them onto the source plane. The same luma formula, sampled over the **external** sampling circles — which reach beyond the cell into neighboring source regions — yields the cell's -**external sampling vector**. Pixels outside the source image are excluded -from an external circle's sample; a circle with no in-bounds pixels contributes +**external sampling vector**. Pixels outside the source image are excluded from +an external circle's sample; a circle with no in-bounds pixels contributes nothing to the enhancement in §9.3.4. #### 9.3.2 Sampling circles and shape vectors @@ -928,33 +933,32 @@ nothing to the enhancement in §9.3.4. Sampling circles are fixed regions of the cell, defined in cell-relative coordinates so they are independent of footprint size and source resolution: -- Six **internal sampling circles** arranged in two staggered columns — the - left column lowered, the right raised, radii enlarged — so the pair covers - the cell well while limiting mutual overlap. Staggering follows the cell's +- Six **internal sampling circles** arranged in two staggered columns — the left + column lowered, the right raised, radii enlarged — so the pair covers the cell + well while limiting mutual overlap. Staggering follows the cell's taller-than-wide shape (the `A = 2` convention of §8.2) and captures - top/bottom, left/right, and diagonal structure: `T` vs `L` vs `p` vs `q`, - `-` vs `_` vs `^`, `/`. -- Ten **external sampling circles** around the cell boundary, each reaching - into neighboring cells. They have no effect on a solitary cell, but they let - contrast enhancement see boundaries adjacent to it (§9.3.4), which removes - the staircasing that global enhancement alone produces. + top/bottom, left/right, and diagonal structure: `T` vs `L` vs `p` vs `q`, `-` + vs `_` vs `^`, `/`. +- Ten **external sampling circles** around the cell boundary, each reaching into + neighboring cells. They have no effect on a solitary cell, but they let + contrast enhancement see boundaries adjacent to it (§9.3.4), which removes the + staircasing that global enhancement alone produces. The circle geometry — centers and radii as fractions of the cell box, plus a fixed internal/external circle ordering — is a closed parameter set whose numeric values are calibration constants pinned by golden tests in the -implementation PR (Open Decision 2). What this specification -closes normatively: the counts (6 internal, 10 external), the two-column -staggered arrangement with lowered-left/raised-right and enlarged radii, the -fixed orderings used by the affecting table below, and the per-component max -normalization of shape vectors. +implementation PR (Open Decision 2). What this specification closes normatively: +the counts (6 internal, 10 external), the two-column staggered arrangement with +lowered-left/raised-right and enlarged radii, the fixed orderings used by the +affecting table below, and the per-component max normalization of shape vectors. **Shape vectors.** For each character in the candidate set (§9.3.3), a 6-component shape vector records, for each internal sampling circle, the -fraction of that circle's area covered by the character's ink. Shape vectors -are computed at build time from a rasterization of each character over the -same circle geometry and compiled into the module (§15) — no runtime font -access exists (INV-I1; the renderer has no font machinery). Every component is -then divided by the largest value of that component across all candidates +fraction of that circle's area covered by the character's ink. Shape vectors are +computed at build time from a rasterization of each character over the same +circle geometry and compiled into the module (§15) — no runtime font access +exists (INV-I1; the renderer has no font machinery). Every component is then +divided by the largest value of that component across all candidates (per-component max normalization), spreading the candidates across the full component range. @@ -970,52 +974,51 @@ The candidate set is a closed table: all 95 printable ASCII characters order. U+0020 (space) is the zero-density anchor; the alpha rule (§9.3.5) bypasses matching entirely for transparent cells. -| Range | Characters | -| ------------ | -------------------------------------------------------------- | -| U+0020 | ` ` (space) | -| U+0021–002F | `!` `"` `#` `$` `%` `&` `'` `(` `)` `*` `+` `,` `-` `.` `/` | -| U+0030–0039 | `0`–`9` | -| U+003A–0040 | `:` `;` `<` `=` `>` `?` `@` | -| U+0041–005A | `A`–`Z` | -| U+005B–0060 | `[` `\` `]` `^` `_` `` ` `` | -| U+0061–007A | `a`–`z` | -| U+007B–007E | `{` `\|` `}` `~` | +| Range | Characters | +| ----------- | ----------------------------------------------------------- | +| U+0020 | `` (space) | +| U+0021–002F | `!` `"` `#` `$` `%` `&` `'` `(` `)` `*` `+` `,` `-` `.` `/` | +| U+0030–0039 | `0`–`9` | +| U+003A–0040 | `:` `;` `<` `=` `>` `?` `@` | +| U+0041–005A | `A`–`Z` | +| U+005B–0060 | `[` `\` `]` `^` `_` `` ` `` | +| U+0061–007A | `a`–`z` | +| U+007B–007E | `{` `\|` `}` `~` | Pruning to a curated subset is Open Decision 1. The cell's glyph is the candidate whose shape vector has the smallest squared Euclidean distance to the cell's contrast-enhanced glyph sampling vector -(§9.3.4); the squared distance suffices, no square root is taken. Distance -ties resolve toward the lower code point. -Lookup acceleration — k-d trees over the 6-D vectors, or result caches keyed -by quantized sampling vectors — is implementation-defined (§15); cached and -uncached lookups MUST agree. +(§9.3.4); the squared distance suffices, no square root is taken. Distance ties +resolve toward the lower code point. Lookup acceleration — k-d trees over the +6-D vectors, or result caches keyed by quantized sampling vectors — is +implementation-defined (§15); cached and uncached lookups MUST agree. #### 9.3.4 Contrast enhancement -Before matching, the glyph sampling vector passes through two contrast steps, -in this order. Enhancement increases the separation between differently -shaded regions, so boundary cells pick characters that emphasize the boundary -(the `&` → `b` → `L` style transitions of the reference approach) instead of -mid-density compromises. It barely affects near-uniform vectors, so smooth -gradients survive. Sampling vectors are oriented bright-to-dense: brighter -source regions match denser-ink glyphs. +Before matching, the glyph sampling vector passes through two contrast steps, in +this order. Enhancement increases the separation between differently shaded +regions, so boundary cells pick characters that emphasize the boundary (the `&` +→ `b` → `L` style transitions of the reference approach) instead of mid-density +compromises. It barely affects near-uniform vectors, so smooth gradients +survive. Sampling vectors are oriented bright-to-dense: brighter source regions +match denser-ink glyphs. 1. **Directional** (per component `i`): with `m = max(v[i], max({v_ext[j] : j affects i}))`, set `v[i] = (v[i] / m)^γd · m`; when `m = 0`, the component is set to `0`. The - affecting-external sets — normative, from - the reference approach, with internal circles `0–5` and external circles - `0–9` in their fixed ordering — are: + affecting-external sets — normative, from the reference approach, with + internal circles `0–5` and external circles `0–9` in their fixed ordering — + are: | Internal | Affected by external | - | -------- | --------------------- | - | 0 | 0, 1, 2, 4 | - | 1 | 0, 1, 3, 5 | - | 2 | 2, 4, 6 | - | 3 | 3, 5, 7 | - | 4 | 4, 6, 8, 9 | - | 5 | 5, 7, 8, 9 | + | -------- | -------------------- | + | 0 | 0, 1, 2, 4 | + | 1 | 0, 1, 3, 5 | + | 2 | 2, 4, 6 | + | 3 | 3, 5, 7 | + | 4 | 4, 6, 8, 9 | + | 5 | 5, 7, 8, 9 | 2. **Global**: with `m = max(v)`, set `v[i] = (v[i] / m)^γg · m` for every component; when `m = 0`, the vector is set to all zeros. @@ -1029,13 +1032,13 @@ together with the circle geometry (Open Decision 2). (U+0020), no foreground, and the background rules of renderer-spec §8.3.3 apply (element `bg` when declared, otherwise the underlying background is preserved). -- Opaque cells (alpha sample ≥ 128) carry the color sample as their - foreground, encoded through the renderer's existing color-encoding path; when - the color-encoding capability work lands, ascii art colors follow it. The - background of opaque cells is not overridden: the glyph's ink density over - the preserved background is what carries the image's lightness, and - equalizing foreground and background would erase the very glyph texture the - shape matcher selected. +- Opaque cells (alpha sample ≥ 128) carry the color sample as their foreground, + encoded through the renderer's existing color-encoding path; when the + color-encoding capability work lands, ascii art colors follow it. The + background of opaque cells is not overridden: the glyph's ink density over the + preserved background is what carries the image's lightness, and equalizing + foreground and background would erase the very glyph texture the shape matcher + selected. - The effective footprint computed in §8 is exactly the art's cell region (INV-I4). The art is never clipped by its own tier; callers size the element or rely on clip regions as with any content. @@ -1043,10 +1046,10 @@ together with the circle geometry (Open Decision 2). ### 9.4 Alt tier The alt tier renders `alt` as plain cell text over the **entire layout box**: -wrapped at the box width, clipped at the box height (overflow is suppressed, -not laid out past the box), with the element's `bg` applied per §8.4 and no -implicit prefix or decoration. Empty `alt` renders an empty box (decorative -image). Alt text is unstyled in v1 — no foreground override, no attributes. +wrapped at the box width, clipped at the box height (overflow is suppressed, not +laid out past the box), with the element's `bg` applied per §8.4 and no implicit +prefix or decoration. Empty `alt` renders an empty box (decorative image). Alt +text is unstyled in v1 — no foreground override, no attributes. ## 10. Cell-Buffer Integration and Damage @@ -1064,74 +1067,73 @@ ordinary background cells under §8.4 and diff normally. ### 10.2 Footprint change -When an image's effective footprint changes between frames and the image's -data version is unchanged, the placement is **re-placed**: emitted at the new -origin with its same (image id, placement id) pair (§9.2.3). The protocol's -same-pair replacement removes the previous graphic without flicker and without -an intervening delete; the old box's pixels vanish with the placement record, -and the blanks previously written under them remain the terminal's text -content at those cells. The diff then repaints the old box's cells whose -front-buffer value (blank) differs from the back (the ordinary vacate case) -and emits blank bytes for newly covered cells. No deletion occurs for a mere -move or element resize, and no bytes are emitted for old-box cells whose new -content equals the blank value they already hold. +When an image's effective footprint changes between frames and the image's data +version is unchanged, the placement is **re-placed**: emitted at the new origin +with its same (image id, placement id) pair (§9.2.3). The protocol's same-pair +replacement removes the previous graphic without flicker and without an +intervening delete; the old box's pixels vanish with the placement record, and +the blanks previously written under them remain the terminal's text content at +those cells. The diff then repaints the old box's cells whose front-buffer value +(blank) differs from the back (the ordinary vacate case) and emits blank bytes +for newly covered cells. No deletion occurs for a mere move or element resize, +and no bytes are emitted for old-box cells whose new content equals the blank +value they already hold. When the data version changed, the re-transmission path of §9.2.2 applies: delete, re-transmit, then re-place every placement of that image. When the -placement is not re-created at all — unmount, tier change, collision demotion -— the stale placement is deleted (§9.2.5). +placement is not re-created at all — unmount, tier change, collision demotion — +the stale placement is deleted (§9.2.5). -A placement whose covered cells receive any cell write in the current frame — -a changed blank-state background beneath unchanged coverage, or any other -buffer change inside its footprint — MUST be re-placed with its same -(image id, placement id) pair in that frame: any write under a placement -erases the graphic's cells, so the re-place heals the whole box rather than -poking holes in it. The re-place is the repair; no deletion is emitted for it. +A placement whose covered cells receive any cell write in the current frame — a +changed blank-state background beneath unchanged coverage, or any other buffer +change inside its footprint — MUST be re-placed with its same (image id, +placement id) pair in that frame: any write under a placement erases the +graphic's cells, so the re-place heals the whole box rather than poking holes in +it. The re-place is the repair; no deletion is emitted for it. Because a deleted placement's pixels are erased by the delete itself, the renderer MUST additionally force the diff to re-emit every cell of a deleted placement's previous footprint: each such front-buffer cell is dirty-marked (§10.4) before diffing. This **deletion damage** rule is what keeps the -terminal's visible state matching the cell buffer after an unmount, tier -change, or removal (INV-I6). +terminal's visible state matching the cell buffer after an unmount, tier change, +or removal (INV-I6). ### 10.3 Overlap policy Content from other elements that lands in a live placement's footprint is **suppressed by a post-walk coverage pass**: after the render-command walk — -including border-junction resolution — image elements resolved to a pixel -tier assert their effective footprint into the back buffer, writing the -covered-cell value over whatever content the walk left there. Running the -pass after the walk, not in directive order, is what makes suppression -structural: overlapping text's bytes are never emitted, so no damage-repair -reaction is needed for text overlap, and directive order cannot defeat -coverage. Pixel-tier coverage cannot honor painter's order — the terminal -renders the placement as a layer over the whole box, and a cell buffer that -showed text under the placement would disagree with the terminal's visible -state either way the bytes were ordered (§9.2.6). v1 therefore makes the -image win at the pixel tiers, deterministically. +including border-junction resolution — image elements resolved to a pixel tier +assert their effective footprint into the back buffer, writing the covered-cell +value over whatever content the walk left there. Running the pass after the +walk, not in directive order, is what makes suppression structural: overlapping +text's bytes are never emitted, so no damage-repair reaction is needed for text +overlap, and directive order cannot defeat coverage. Pixel-tier coverage cannot +honor painter's order — the terminal renders the placement as a layer over the +whole box, and a cell buffer that showed text under the placement would disagree +with the terminal's visible state either way the bytes were ordered (§9.2.6). v1 +therefore makes the image win at the pixel tiers, deterministically. - When two pixel-tier footprints overlap in one frame, the later element (in directive order) **demotes to the ascii tier for that frame** — a ladder fallback, silent like any other fall-through (§9.1), because the demoted - element remains fully representable. The earlier placement keeps its - footprint intact; the demoted element's art cells are ordinary cells that - the coverage pass overwrites with the earlier placement's blanks. + element remains fully representable. The earlier placement keeps its footprint + intact; the demoted element's art cells are ordinary cells that the coverage + pass overwrites with the earlier placement's blanks. - Overlap between **ascii**/**alt**-tier images is ordinary painter's order: later content in directive order wins per cell, like any text. -- The placements-table capacity degradation of §5.4 applies before any - placement is emitted, so suppression never leaves an untracked placement. +- The placements-table capacity degradation of §5.4 applies before any placement + is emitted, so suppression never leaves an untracked placement. - Callers SHOULD avoid overlapping images with other content in v1; the z-ordered coexistence design is deferred (§16.6). ### 10.4 Dirty-marking primitive -Dirty-marking a cell region forces the next render's diff to emit bytes for -it. The described mechanism: overwrite the region's front-buffer cells with -the default cell value, so the next diff necessarily differs. Any mechanism -that guarantees the same emissions satisfies the contract. Dirty-marking is -used by: `removeImage` (§7.2), deletion damage (§10.2), and capability -invalidation (§11.2). +Dirty-marking a cell region forces the next render's diff to emit bytes for it. +The described mechanism: overwrite the region's front-buffer cells with the +default cell value, so the next diff necessarily differs. Any mechanism that +guarantees the same emissions satisfies the contract. Dirty-marking is used by: +`removeImage` (§7.2), deletion damage (§10.2), and capability invalidation +(§11.2). ### 10.5 Resize @@ -1144,9 +1146,9 @@ of the diff state. Before the discard, the resize transaction MUST: discipline (TINV-5): the redraw that follows cleans pixels by writing text over covered cells, but it cannot clean terminal-side records the way a deletion does. -2. Reset the affected images' transmission state (§9.2.2) — placements are - gone, so persisted-data assumptions are dropped (the protocol's storage - quota preferentially deletes images without placements). +2. Reset the affected images' transmission state (§9.2.2) — placements are gone, + so persisted-data assumptions are dropped (the protocol's storage quota + preferentially deletes images without placements). 3. Reconcile the table and the front buffer's covered cells as part of discarding diff state, so the next render's complete redraw contains no image-specific bytes beyond the new frame's own paint. @@ -1161,60 +1163,59 @@ sanctioned exception to renderer-spec §7.6's "no cursor-positioning bytes in caret-less frames": frames containing kitty-tier images MAY include the cursor-positioning bytes needed for placement, provided the post-frame cursor visibility state defined in §7.6 is unchanged, and for caret-declaring frames -the post-frame position is the caret's cell as §7.6 requires. The renderer -MUST account for the render `row` option (renderer-spec §8.2.1) when computing +the post-frame position is the caret's cell as §7.6 requires. The renderer MUST +account for the render `row` option (renderer-spec §8.2.1) when computing placement positions, exactly as it does for cell CUPs. ## 11. Capability Evidence and Invalidation ### 11.1 Evidence table -| Tier | Gate | Evidence source | -| ----- | ------------------------------------------------- | --------------------------------------------------------------------------------------- | -| kitty | `RuntimeCapabilities.kittyGraphics === true` | Probe query 9 (APC `_G` query, terminfo-spec §9.1), delivered as a `CapabilityEvent` (terminfo-spec §6.3) folded in through `term.update()` | -| ascii | none | Always available | -| alt | none | Always available | +| Tier | Gate | Evidence source | +| ----- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| kitty | `RuntimeCapabilities.kittyGraphics === true` | Probe query 9 (APC `_G` query, terminfo-spec §9.1), delivered as a `CapabilityEvent` (terminfo-spec §6.3) folded in through `term.update()` | +| ascii | none | Always available | +| alt | none | Always available | The v1 kitty gate is the existing boolean. The renderer does not verify -sub-features of the kitty graphics protocol (placeholders, animation) in v1; -it uses only transmission, placement, and deletion — the protocol's core. +sub-features of the kitty graphics protocol (placeholders, animation) in v1; it +uses only transmission, placement, and deletion — the protocol's core. ### 11.2 Capability-change invalidation -When a `CapabilityEvent` folded through `update()` changes a tier's -eligibility — a `kitty-graphics` event arriving true or false — the update -transaction's behavior is: +When a `CapabilityEvent` folded through `update()` changes a tier's eligibility +— a `kitty-graphics` event arriving true or false — the update transaction's +behavior is: - **Denial (true → false).** The transaction returns immediate deletion bytes: - one data-freeing `a=d,d=I,i=,q=2` block per registry image - with live placements (§7.2), recovered from the placements table before any - state is discarded, and resets the affected images' transmission state - (§9.2.2). The table's placement entries are then cleared and the front - buffer's covered cells are reconciled to their blank-state values, so the - next render emits no redundant image bytes. This honors the write-now - discipline (TINV-5): a caller that never renders again does not leave - graphics on the terminal. + one data-freeing `a=d,d=I,i=,q=2` block per registry image with + live placements (§7.2), recovered from the placements table before any state + is discarded, and resets the affected images' transmission state (§9.2.2). The + table's placement entries are then cleared and the front buffer's covered + cells are reconciled to their blank-state values, so the next render emits no + redundant image bytes. This honors the write-now discipline (TINV-5): a caller + that never renders again does not leave graphics on the terminal. - **Grant (false → true).** No immediate bytes: the terminal's visible state - does not change (the lower tier's art is on screen; the kitty tier has - nothing to delete). The next render's own resolution transmits and places - at the kitty tier. + does not change (the lower tier's art is on screen; the kitty tier has nothing + to delete). The next render's own resolution transmits and places at the kitty + tier. This is the focused-spec output invalidation that renderer-spec §7.7 and §7.8 reserve for feature specifications. A capability event that changes only color -encoding (e.g., `colordepth`) affects ascii art colors; the renderer MAY -re-emit affected art on the next render. It MUST NOT re-transmit unchanged -kitty data for color changes. +encoding (e.g., `colordepth`) affects ascii art colors; the renderer MAY re-emit +affected art on the next render. It MUST NOT re-transmit unchanged kitty data +for color changes. ### 11.3 Removal and capability edge cases -- `removeImage()` when `kittyGraphics` is false but the id is live still - returns deletion bytes (§7.2): placements placed under earlier evidence may - persist terminal-side. -- An image placed at the kitty tier whose capability is later denied is - cleaned by §11.2's immediate denial rule, not by `removeImage()`. -- A tier re-arm (denied → supported) re-transmits lazily at the next - placement: the denial deleted the terminal-side data, so the transmission - reset of §9.2.2 forces a fresh upload. No bytes flow at grant time. +- `removeImage()` when `kittyGraphics` is false but the id is live still returns + deletion bytes (§7.2): placements placed under earlier evidence may persist + terminal-side. +- An image placed at the kitty tier whose capability is later denied is cleaned + by §11.2's immediate denial rule, not by `removeImage()`. +- A tier re-arm (denied → supported) re-transmits lazily at the next placement: + the denial deleted the terminal-side data, so the transmission reset of §9.2.2 + forces a fresh upload. No bytes flow at grant time. ## 12. Interaction with Line Mode @@ -1229,43 +1230,43 @@ above the cap resolves to ascii, then alt, unchanged. The render result's `errors` channel (renderer-spec §12.3) gains three Clayterm-specific types alongside `"CLIP_DEPTH_EXCEEDED"`: -| `type` | Raised when | Behavior | -| ------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | -| `"IMAGE_NOT_FOUND"` | An `img` directive references a registry id that is not live (registry empty, id never set, or removed) | Renders at the alt tier (graceful degradation); reported once per affected element per render | -| `"IMAGE_PLACEMENTS_EXCEEDED"` | More kitty placements were required in one frame than the placements table holds (§5.4) | Excess elements degrade to the ascii tier for that frame, in directive order; reported once per render | -| `"IMAGE_PLACEMENT_COLLISION"` | Two or more elements in one frame derive the same placement id (§9.2.1) | The affected elements resolve to the ascii tier for that frame and emit no placements; reported once per affected element | +| `type` | Raised when | Behavior | +| ----------------------------- | ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | +| `"IMAGE_NOT_FOUND"` | An `img` directive references a registry id that is not live (registry empty, id never set, or removed) | Renders at the alt tier (graceful degradation); reported once per affected element per render | +| `"IMAGE_PLACEMENTS_EXCEEDED"` | More kitty placements were required in one frame than the placements table holds (§5.4) | Excess elements degrade to the ascii tier for that frame, in directive order; reported once per render | +| `"IMAGE_PLACEMENT_COLLISION"` | Two or more elements in one frame derive the same placement id (§9.2.1) | The affected elements resolve to the ascii tier for that frame and emit no placements; reported once per affected element | None of these abort the frame; other elements render normally. The error `message` fields SHOULD identify the affected element ids and, for -`"IMAGE_NOT_FOUND"`, the missing registry id. All three are specified as -closed fallbacks rather than undefined behavior so that no condition can -leave undeletable graphics debris or an untracked placement. +`"IMAGE_NOT_FOUND"`, the missing registry id. All three are specified as closed +fallbacks rather than undefined behavior so that no condition can leave +undeletable graphics debris or an untracked placement. Thrown conditions — descriptive `RangeError`s that abort the transaction and return no bytes (INV-I9), following the transfer-overflow discipline of renderer-spec §9.2: -| Condition | Raised by | Message SHOULD identify… | -| ------------------------------------------- | ------------ | ---------------------------------------------------------------- | -| `setImage` validation failure (§7.2) | `setImage()` | the offending field and value | -| Registry table full / pixel pool exhausted | `setImage()` | the capacity limits and the requested size (§5.3) | -| Output-capacity overflow | `render()` | the emission that overflowed (suggest a smaller image or frame) | +| Condition | Raised by | Message SHOULD identify… | +| ------------------------------------------ | ------------ | --------------------------------------------------------------- | +| `setImage` validation failure (§7.2) | `setImage()` | the offending field and value | +| Registry table full / pixel pool exhausted | `setImage()` | the capacity limits and the requested size (§5.3) | +| Output-capacity overflow | `render()` | the emission that overflowed (suggest a smaller image or frame) | After any throw, the Term remains usable: the next render behaves as a full re-render of that frame (INV-I9 keeps retained state truthful). ## 14. Testing Strategy -All tests live in `test/image.test.ts` — one spec, one test file (AGENTS.md). +All tests live in `test/graphics.test.ts` — one spec, one test file (AGENTS.md). Obligations: -1. **Tier resolution truth table.** For every combination of `variant`, - `image` presence, registry liveness, and `kittyGraphics` evidence, the - selected tier matches §9.1. +1. **Tier resolution truth table.** For every combination of `variant`, `image` + presence, registry liveness, and `kittyGraphics` evidence, the selected tier + matches §9.1. 2. **Footprint math.** Intrinsic size (§8.3), contain resolution (§8.2) — including the `ceil`/`round`/`floor` edge cases, empty footprints, and - upscale/downscale paths — agree between a pure reference and rendered - output (INV-I4). + upscale/downscale paths — agree between a pure reference and rendered output + (INV-I4). 3. **Kitty conformance.** Transmitted frames are valid APC with RGBA direct encoding (`a=t,t=d,f=32,s=,v=`), chunked at ≤ 4096 base64 bytes with multiple-of-4 non-final chunk lengths, `m=1` continuation / `m=0` final, @@ -1274,17 +1275,17 @@ Obligations: option. 4. **Transmission state.** First placement transmits; re-renders without data change do not re-transmit; a version bump emits delete-id, re-transmit, - re-place in order (§9.2.2); loss of the last placement (unmount, tier - switch, collision demotion), capability denial, and resize each force - re-transmission on the next placement (§9.2.2, §11.2). + re-place in order (§9.2.2); loss of the last placement (unmount, tier switch, + collision demotion), capability denial, and resize each force re-transmission + on the next placement (§9.2.2, §11.2). 5. **Placement and cleanup.** First placement emits no stale-delete; footprint - change of an unchanged image emits a pure same-pair re-place with no - deletion (§10.2); a version bump emits delete, re-transmit, re-place - (§9.2.2); unmount and tier switch emit the scoped lowercase - `a=d,d=i,…,p=…` form; `removeImage` emits the data-freeing `d=I` form once - and the next render emits no image bytes for the vacated footprint; - placement ids are stable across frames for the same element and distinct - across elements; ids are never 0; wire image ids equal registry ids. + change of an unchanged image emits a pure same-pair re-place with no deletion + (§10.2); a version bump emits delete, re-transmit, re-place (§9.2.2); unmount + and tier switch emit the scoped lowercase `a=d,d=i,…,p=…` form; `removeImage` + emits the data-freeing `d=I` form once and the next render emits no image + bytes for the vacated footprint; placement ids are stable across frames for + the same element and distinct across elements; ids are never 0; wire image + ids equal registry ids. 6. **Cell-buffer honesty.** Covered cells hold the covered-cell value and produce no bytes while unchanged; overlapping text is suppressed (§10.3); vacated and deletion-damaged cells repaint (§10.2). @@ -1293,15 +1294,15 @@ Obligations: shape-vector normalization, tie-breaking, the alpha threshold, and the empty-rectangle rule (§9.3). Cached and uncached glyph lookups MUST agree. Edge cases: uniform vectors pick density-appropriate glyphs with minimal - enhancement; external circles crossing the image border contribute - nothing; 1×1-cell footprints. + enhancement; external circles crossing the image border contribute nothing; + 1×1-cell footprints. 8. **Alt tier.** Wrap, clip, empty-alt, bg application, and box-width text painting (§9.4). -9. **Registry API.** `setImage` validation RangeErrors (bad id, bad - dimensions, wrong byte length, table-full, pool-exhausted-even-after- - compaction); replace bumps data version; compaction preserves live entries' - render output; `removeImage` idempotence, byte emission with - `kittyGraphics` both true and false, and dirty-marking. +9. **Registry API.** `setImage` validation RangeErrors (bad id, bad dimensions, + wrong byte length, table-full, pool-exhausted-even-after- compaction); + replace bumps data version; compaction preserves live entries' render output; + `removeImage` idempotence, byte emission with `kittyGraphics` both true and + false, and dirty-marking. 10. **Capacity behavior.** Registry and placements capacities produce §5.3's RangeError and §13's degradation respectively, deterministically. 11. **Capability invalidation.** `kitty-graphics` true after ascii renders @@ -1314,80 +1315,77 @@ Obligations: 14. **Clip interaction.** Clipped images place and paint only their effective footprint (§8.5). 15. **Resize.** Registry survives resize; the resize update returns one - data-freeing deletion per image with live placements and resets - transmission state; the placements table is discarded; the post-resize - redraw erases prior pixels; re-transmission occurs on next placement - (§10.5). + data-freeing deletion per image with live placements and resets transmission + state; the placements table is discarded; the post-resize redraw erases + prior pixels; re-transmission occurs on next placement (§10.5). 16. **Errors.** `IMAGE_NOT_FOUND` fallback and once-per-element surfacing; `IMAGE_PLACEMENT_COLLISION` demotes affected elements and emits no placements; a thrown render returns no bytes and leaves retained state unchanged, with the retry render reproducing the frame (INV-I9). -17. **Input-stream cleanliness.** Rendered output containing kitty frames, - fed to `input.scan()`, yields zero events (INV-I5). +17. **Input-stream cleanliness.** Rendered output containing kitty frames, fed + to `input.scan()`, yields zero events (INV-I5). ## 15. Implementation Notes _This section is non-normative._ -**Transfer encoding.** `img` adds one opcode to the command buffer in the -§12.1 style: an opcode word, the id string, a property mask, and fields for -the registry id, sizing axes, alt string, variant, and bg. The existing -protocol is extended, not restructured. Because the underlying layout engine -models elements as open/close pairs, the void directive expands to a balanced -open/close pair with leaf sizing at decode time; `validate` sees the void form. +**Transfer encoding.** `img` adds one opcode to the command buffer in the §12.1 +style: an opcode word, the id string, a property mask, and fields for the +registry id, sizing axes, alt string, variant, and bg. The existing protocol is +extended, not restructured. Because the underlying layout engine models elements +as open/close pairs, the void directive expands to a balanced open/close pair +with leaf sizing at decode time; `validate` sees the void form. **Placement-id derivation.** The reference implementation pins FNV-1a over the element id's UTF-8 bytes and the registry id's decimal digits — XOR-separator `0x3A` between them; a raw result of 0 maps to 1 — into `[1, 4294967295]`; golden tests lock the exact values. The derivation MUST satisfy the normative requirements of §9.2.1 (pure, deterministic, uniform); pinning is what makes -golden tests exact and cross-session reports comparable. The known limitation -is the same-frame collision case of §9.2.1, handled per §13. +golden tests exact and cross-session reports comparable. The known limitation is +the same-frame collision case of §9.2.1, handled per §13. -**Ordinary-cell art.** The ascii tier's cell pipeline (glyph + fg + bg per -cell) is the same machinery plain text uses; no new cell representation is -required for art. Kitty-covered cells are the only new cell-buffer concept, -and they are behaviorally just blank cells (§10.1). +**Ordinary-cell art.** The ascii tier's cell pipeline (glyph + fg + bg per cell) +is the same machinery plain text uses; no new cell representation is required +for art. Kitty-covered cells are the only new cell-buffer concept, and they are +behaviorally just blank cells (§10.1). **Ascii cost and shape-vector baking.** Glyph selection samples each cell's -source rectangle per circle and runs a nearest-neighbor match per cell: -O(W·H) sampling once per changed image plus O(cells × candidates) distance -evaluations. The reference approach's accelerations apply — k-d trees over -the 6-D shape vectors, or result caches keyed by quantized sampling vectors — -and are implementation-defined, with cached and uncached lookups required to -agree (§9.3.3). The renderer MAY additionally memoize per-element matched art -keyed by (registry id, data version, footprint box) in fixed-capacity carved -storage; memoized and unmemoized outputs MUST agree. Shape vectors are baked -at build time from a rasterization of the candidate characters — a small -closed table of candidates × 6 circle fractions, not a font system — so no -runtime rasterization exists (INV-I1). Kitty transmission is O(W·H) once per -data version. All costs are bounded by registry image sizes, not by frame -rate. - -**Streaming video ergonomics.** The registry's replace-on-id path is the -natural video-streaming API (one `setImage` per frame; §4.5's video -producer). Each replacement -costs a delete/re-transmit/re-place cycle (§9.2.2); terminals batch this -within a write burst, but the protocol does not guarantee flicker-free -updates. Per-frame streaming ergonomics are revisited with §16.4 -(placeholders) and §16.8 (animation frames), which would make moves pure -cell diffs and updates re-transmission-free. - -**Cursor safety.** `C=1` keeps the cursor at the placement origin; combined -with §10.6's post-frame restore, no frame leaves the cursor somewhere the -caller did not put it. +source rectangle per circle and runs a nearest-neighbor match per cell: O(W·H) +sampling once per changed image plus O(cells × candidates) distance evaluations. +The reference approach's accelerations apply — k-d trees over the 6-D shape +vectors, or result caches keyed by quantized sampling vectors — and are +implementation-defined, with cached and uncached lookups required to agree +(§9.3.3). The renderer MAY additionally memoize per-element matched art keyed by +(registry id, data version, footprint box) in fixed-capacity carved storage; +memoized and unmemoized outputs MUST agree. Shape vectors are baked at build +time from a rasterization of the candidate characters — a small closed table of +candidates × 6 circle fractions, not a font system — so no runtime rasterization +exists (INV-I1). Kitty transmission is O(W·H) once per data version. All costs +are bounded by registry image sizes, not by frame rate. + +**Streaming video ergonomics.** The registry's replace-on-id path is the natural +video-streaming API (one `setImage` per frame; §4.5's video producer). Each +replacement costs a delete/re-transmit/re-place cycle (§9.2.2); terminals batch +this within a write burst, but the protocol does not guarantee flicker-free +updates. Per-frame streaming ergonomics are revisited with §16.4 (placeholders) +and §16.8 (animation frames), which would make moves pure cell diffs and updates +re-transmission-free. + +**Cursor safety.** `C=1` keeps the cursor at the placement origin; combined with +§10.6's post-frame restore, no frame leaves the cursor somewhere the caller did +not put it. ## 16. Deferred (v1 boundaries) ### 16.1 Sixel tier The intended ladder position is kitty → **sixel** → ascii → alt. Sixel is -deferred because it cannot be specified correctly without two evidence -sources that do not exist yet: +deferred because it cannot be specified correctly without two evidence sources +that do not exist yet: 1. **Sixel support evidence.** Terminals advertise sixel in their DA1 feature - bits (feature `4`); the terminfo specification currently treats the DA1 - reply as a fence only. A follow-up must extend DA1 parsing into a new + bits (feature `4`); the terminfo specification currently treats the DA1 reply + as a fence only. A follow-up must extend DA1 parsing into a new `CapabilityEvent` key (e.g., `sixel-graphics`). 2. **Cell-size geometry evidence.** Sixel paints pixels at the cursor with no cell-footprint parameter, so both the painted footprint and the required @@ -1404,23 +1402,23 @@ kitty/ascii/alt. ### 16.2 iTerm2 inline images OSC 1337 `File=` is a third pixel protocol. Evidence (identity parsing of -DA2/XTVERSION) is a terminfo follow-up; the tier slots between kitty and -sixel by fidelity. +DA2/XTVERSION) is a terminfo follow-up; the tier slots between kitty and sixel +by fidelity. ### 16.3 Geometry-evidence scaling With cell-size evidence (`A` measured instead of assumed 2), contain becomes -exact on terminals whose cells are not 2:1, and the intrinsic-size default -can be expressed in true terminal pixels. The change is internal to §8.2 and -gated on the geometry follow-up. +exact on terminals whose cells are not 2:1, and the intrinsic-size default can +be expressed in true terminal pixels. The change is internal to §8.2 and gated +on the geometry follow-up. ### 16.4 Kitty unicode placeholders Transmitting once and placing via placeholder codepoints in cells would make -kitty-covered cells first-class diff participants (moves become pure cell -diffs, and scroll-region tracking comes nearly free). It requires evidence -that the terminal implements placeholders, which the current probe does not -capture. Deferred to a focused amendment. +kitty-covered cells first-class diff participants (moves become pure cell diffs, +and scroll-region tracking comes nearly free). It requires evidence that the +terminal implements placeholders, which the current probe does not capture. +Deferred to a focused amendment. ### 16.5 Background-aware art @@ -1432,32 +1430,31 @@ until the color-encoding work lands. ### 16.6 Z-ordering and overlap -Coexistence of graphics with overlapping text (floating overlays over images) -is unspecified beyond §10.3's policy. A future design would need kitty `z` -semantics (including negative z-index placements, which the protocol draws -under text) or placeholder-based layering. +Coexistence of graphics with overlapping text (floating overlays over images) is +unspecified beyond §10.3's policy. A future design would need kitty `z` +semantics (including negative z-index placements, which the protocol draws under +text) or placeholder-based layering. ### 16.7 Stretch scaling `scale: "stretch"` is not in v1. At the kitty tier it is unsatisfiable: with -both `c=` and `r=` specified the protocol letterboxes to prevent distortion, -so a stretched render would claim cells it does not paint (INV-I4). At the -ascii tier stretch is representable (the pixel mapping is renderer-owned) and -could be added as a directive property in a future amendment, ascii-only. +both `c=` and `r=` specified the protocol letterboxes to prevent distortion, so +a stretched render would claim cells it does not paint (INV-I4). At the ascii +tier stretch is representable (the pixel mapping is renderer-owned) and could be +added as a directive property in a future amendment, ascii-only. ### 16.8 Animation, cropping, transitions Under §4.5's layering, video is a per-frame revision-bump producer and the substrate supports it today — but per-frame full re-transmission does not -flicker-free. The Kitty protocol's own animation machinery is its designed -fast path: multi-frame images, frame composition (delta frames), and -playback controls, uploaded once and played back without re-transmission. -Landing it is a focused amendment; it is a protocol sub-feature, and §9.1 -does not verify sub-features in v1. Multi-frame source formats, cropping -source rectangles, and transitions on image geometry are also not designed -here. The directive model leaves room: `image` could become a richer -descriptor and `transition` could apply to footprint axes without breaking -§7.1's shape. +flicker-free. The Kitty protocol's own animation machinery is its designed fast +path: multi-frame images, frame composition (delta frames), and playback +controls, uploaded once and played back without re-transmission. Landing it is a +focused amendment; it is a protocol sub-feature, and §9.1 does not verify +sub-features in v1. Multi-frame source formats, cropping source rectangles, and +transitions on image geometry are also not designed here. The directive model +leaves room: `image` could become a richer descriptor and `transition` could +apply to footprint axes without breaking §7.1's shape. ### 16.9 Synchronized-output wrapping @@ -1465,121 +1462,87 @@ How graphics bytes interact with DECRQM 2026 frame wrapping is the synchronized-output follow-up's concern. This specification treats graphics bytes as ordinary frame output. -## Appendix A. Proposed Amendments to Existing Specs - -_None of the following is applied yet. This appendix is the proposal; on -approval the amendments land as their own spec-only commit._ - -**A1. Renderer-spec §4.3 / §6 INV-3 (retained state).** Amend "the only state -the renderer retains between frames is the cell buffer used for diffing" to -"the only state the renderer retains between frames is diffing state: the -cell buffers, the image registry, and the previous frame's placements table -(as defined in the Graphics Specification §4.1)". Frame-snapshot independence of -the directive array is unchanged. - -**A2. Renderer-spec §7.6 (cursor carve-out).** Append to the cursor -visibility/positioning section: frames containing image elements at a -graphics tier MAY additionally include cursor-positioning bytes required by -the graphics protocol (Graphics Specification §10.6), provided the post-frame -cursor visibility state defined here is unchanged and caret-declaring frames -still end with the caret's cell. - -**A3. Renderer-spec §8.3 (directive constructors).** Add §8.3.5 `img()` -deferring normatively to the Graphics Specification, noting the void-directive -form and that `img` values do not participate in open/close balance. - -**A4. Renderer-spec §7.8 (capability consumption).** Remove "Kitty graphics -emission" from the deferred list and reference the Graphics Specification as the -focused feature specification that defines it. - -**A5. Renderer-spec §12.1/§12.2 (current surface).** Note the `img` opcode in -the transfer encoding and the `ImgProps` surface, as current implementation -surface per §5's contract-layer boundary. - -**A6. Renderer-spec §12.3 (errors).** Add `"IMAGE_NOT_FOUND"`, -`"IMAGE_PLACEMENTS_EXCEEDED"`, and `"IMAGE_PLACEMENT_COLLISION"` to the -Clayterm-specific error types. - -**A7. Renderer-spec §8.1 (createTerm options).** Add the optional -`imagePoolBytes` option (Graphics Specification §5.3), default 4 MiB, sizing the -carved image registry. The memory carve and capacity behavior are normative -in the Graphics Specification; renderer-spec §2 keeps WASM memory layout out of -scope beyond such behavioral requirements. - -**A8. Terminfo specification.** No changes in v1. The kitty tier consumes the -existing `kitty-graphics` evidence; sixel and geometry evidence are future -amendments (§16.1, §16.3). (The `createTerm` restatement in terminfo-spec -§10.2 lists the same options as renderer-spec §8.1; when A7 lands, that -restatement is updated in the same commit to avoid drift.) - -**A9. Input specification.** No changes. The input parser already recognizes -and silently consumes kitty graphics APC replies (input-spec §6.2); INV-I5 -keeps the renderer's emissions from generating any. - -**A10. Renderer-spec §7.7 (update transaction).** Note that a focused feature -specification may define immediate update-output bytes for its capability or -resize steps; the Graphics Specification exercises this for graphics cleanup -(§10.5, §11.2). No foundation behavior changes. +## Appendix A. Amendments Applied + +The draft 0.1 appendix proposed ten amendments to existing specifications; they +were approved with this specification and are applied as follows: + +- **With this specification (contract items).** A1 — renderer-spec §4.3's + retained-state sentence now names diffing state (cell buffers, image registry, + placements record). A2 — renderer-spec §7.6 gained the graphics-protocol + cursor carve-out. A3 — renderer-spec §8.3.5 `img()`. A4 — renderer-spec §7.8 + references this specification instead of deferring Kitty graphics emission. A7 + — renderer-spec §8.1's `createTerm` gained the `imagePoolBytes` option. A8 — + terminfo-spec §10.2's `createTerm` restatement gained the same option. A10 — + renderer-spec §7.7's capability semantics note that this specification defines + immediate update bytes for graphics cleanup. +- **With the implementation PR (current-surface notes).** A5 — renderer-spec + §12.1/§12.2 notes for the `img` opcode and `ImgProps` surface. A6 — + renderer-spec §12.3's error-type list gains `"IMAGE_NOT_FOUND"`, + `"IMAGE_PLACEMENTS_EXCEEDED"`, and `"IMAGE_PLACEMENT_COLLISION"`. These + describe the current implementation surface (§5) and are applied when the + surface exists. +- **No changes.** A9 — the input specification is untouched: the parser already + recognizes kitty APC replies (input-spec §6.2), and INV-I5 keeps the + renderer's emissions from generating any. ## Open Decisions The following are open. They are deliberately not resolved by this draft. 1. **The ascii candidate set.** §9.3.3 proposes all 95 ASCII printable - characters. The alternative is a curated subset favoring shape-stable - glyphs (`.`, `:`, `-`, `/`, `T`, `O`, …), trading matching precision for - robustness against font variation between the baked rasterization and the - host terminal. The set is a closed normative table; changing it later is a - versioned spec change, not a tweak. + characters. The alternative is a curated subset favoring shape-stable glyphs + (`.`, `:`, `-`, `/`, `T`, `O`, …), trading matching precision for robustness + against font variation between the baked rasterization and the host terminal. + The set is a closed normative table; changing it later is a versioned spec + change, not a tweak. 2. **Sampling-circle geometry and contrast constants.** §9.3.2 and §9.3.4 fix the structure (six staggered internal circles, ten external circles, the - affecting-external table, directional then global enhancement, - per-component max normalization) but defer exact centers, radii, orderings, - and exponents to calibration against the reference implementation, locked - by golden tests in the implementation PR. + affecting-external table, directional then global enhancement, per-component + max normalization) but defer exact centers, radii, orderings, and exponents + to calibration against the reference implementation, locked by golden tests + in the implementation PR. 3. **Alpha in glyph sampling.** §9.3.1 ignores alpha when sampling glyph lightness; transparency is decided per cell by the alpha sample. The alternative — alpha-weighting the luma samples or compositing semi-alpha color over the element background — would render semi-transparent regions blended, at the cost of coupling glyph choice to a channel the color path otherwise ignores. -4. **`removeImage()` return value.** §7.2 returns delete bytes immediately. - The alternative — defer all cleanup to the next render — keeps - `removeImage` void but leaves graphics on screen if the caller never - renders again. -5. **Quiet mode strictness.** INV-I5 fixes `q=2` (all responses suppressed), - so protocol failures are invisible to v1. A debug-mode `q=1` (suppress OK - only) would surface failures — as APC replies the input parser would need - to route somewhere safe — at the cost of violating the silence guarantee - INV-I5 exists for. -6. **Image-id cohabitation and APC passthrough.** Registry ids map directly - onto wire image ids (§9.2.1). A Term sharing its screen with another - program that also uses kitty image ids could collide. Reserving a base - offset or negotiating ids (the protocol's `I=` numbering) is possible; v1 - declares the single-owner screen assumption instead. The probe's own - graphics query is harmless to this assumption: it is a query action - (`a=q`), which per the protocol "will not replace an existing image with - the same id, nor will it store the image" — re-probes leave no image - behind and collide with nothing, so no id reservation exists in v1. - Multiplexers are the related hazard: a mux that swallows APC bytes - silently degrades the ladder to ascii (no probe response arrives), which - is correct behavior; a passthrough mux that mishandles graphics itself is - not detectable in v1. +4. **`removeImage()` return value.** §7.2 returns delete bytes immediately. The + alternative — defer all cleanup to the next render — keeps `removeImage` void + but leaves graphics on screen if the caller never renders again. +5. **Quiet mode strictness.** INV-I5 fixes `q=2` (all responses suppressed), so + protocol failures are invisible to v1. A debug-mode `q=1` (suppress OK only) + would surface failures — as APC replies the input parser would need to route + somewhere safe — at the cost of violating the silence guarantee INV-I5 exists + for. +6. **Image-id cohabitation and APC passthrough.** Registry ids map directly onto + wire image ids (§9.2.1). A Term sharing its screen with another program that + also uses kitty image ids could collide. Reserving a base offset or + negotiating ids (the protocol's `I=` numbering) is possible; v1 declares the + single-owner screen assumption instead. The probe's own graphics query is + harmless to this assumption: it is a query action (`a=q`), which per the + protocol "will not replace an existing image with the same id, nor will it + store the image" — re-probes leave no image behind and collide with nothing, + so no id reservation exists in v1. Multiplexers are the related hazard: a mux + that swallows APC bytes silently degrades the ladder to ascii (no probe + response arrives), which is correct behavior; a passthrough mux that + mishandles graphics itself is not detectable in v1. 7. **Pixel-pool capacity policy.** §5.3 makes the pool a `createTerm` option (default 4 MiB) with compaction-only reclamation. Alternatives: a fixed normative constant (no option, less API surface, more waste for non-image apps), or size-classed blocks instead of contiguous extents (no compaction cost, more access complexity). 8. **Term teardown.** There is no disposal callback; graphics placed by a - discarded Term are the caller's cleanup (§4.3). A `term.dispose()` - returning deletion bytes is the natural completion of the lifecycle and is - deliberately out of scope until the foundation defines Term disposal. + discarded Term are the caller's cleanup (§4.3). A `term.dispose()` returning + deletion bytes is the natural completion of the lifecycle and is deliberately + out of scope until the foundation defines Term disposal. 9. **Terminal-state desync.** Caller-owned terminal operations clear images behind the renderer's wire state: a reset (RIS) clears all images, an alternate-screen switch clears the screen's images (mode 1049), and the clear-screen escape (`ED 2J`) is specified to clear images — all - caller-managed operations the wire state does not observe. v1 documents - the ownership boundary (§4.3) and does not detect resets. Candidate - future mitigations: a post-reset companion routine, or re-probe-driven - re-sync. Undecided because resets are caller-owned terminal state, the - same class of problem as alt-screen ownership. + caller-managed operations the wire state does not observe. v1 documents the + ownership boundary (§4.3) and does not detect resets. Candidate future + mitigations: a post-reset companion routine, or re-probe-driven re-sync. + Undecided because resets are caller-owned terminal state, the same class of + problem as alt-screen ownership. diff --git a/specs/renderer-spec.md b/specs/renderer-spec.md index 0e1d6ff..f013aec 100644 --- a/specs/renderer-spec.md +++ b/specs/renderer-spec.md @@ -149,9 +149,11 @@ yields, or partial results. Each render transaction operates on a complete, self-contained snapshot of the UI. The renderer MUST NOT maintain an internal component tree or UI state across -frames. The only state the renderer retains between frames is the cell buffer -used for diffing, which is an implementation detail of output minimization and -not observable to the caller except through reduced output size. +frames. The only state the renderer retains between frames is diffing state — +the cell buffers, the image registry, and the previous frame's placements +record (as defined in the [Graphics Specification](graphics-spec.md) §4.1) — +which is an implementation detail of output minimization and correct graphics +cleanup, and not observable to the caller except through reduced output size. ### 4.4 Double-buffered diffing @@ -412,6 +414,12 @@ multi-cursor extension without breaking this contract. This responsibility is limited to the hardware cursor's position and visibility. Cursor shape and blink rate remain caller-managed. +Frames containing image elements at a graphics tier MAY additionally include +cursor-positioning bytes required by the graphics protocol ([Graphics +Specification](graphics-spec.md) §10.6), provided the post-frame cursor state +defined here is unchanged and caret-declaring frames still end with the +caret's cell. + ### 7.7 Update transaction The update transaction changes a Term instance's dimensions or capability state @@ -451,7 +459,9 @@ be a no-op for that step. `value` into the Term's private `RuntimeCapabilities`. The foundation update transaction does not emit bytes or alter renderer output as a consequence of a capability event. A focused capability specification MUST define any output -invalidation or immediate bytes required by its consumer. +invalidation or immediate bytes required by its consumer; the [Graphics +Specification](graphics-spec.md) defines such bytes for graphics cleanup on +capability denial and resize (§11.2 and §10.5 there). **Return value.** `update()` returns a `Uint8Array` of bytes to write to the terminal immediately. An empty array is valid when the update changes no @@ -476,8 +486,9 @@ own renderer section, capability evidence, invalidation rules, and tests in a focused feature specification. In particular, color encoding, synchronized-output wrapping, pointer-shape -output, Kitty keyboard mode setup, and Kitty graphics emission are deferred to -their respective follow-up PRs. The renderer continues to emit its existing +output, and Kitty keyboard mode setup are deferred to their respective +follow-up PRs; Kitty graphics emission is specified by the [Graphics +Specification](graphics-spec.md). The renderer continues to emit its existing hardcoded ANSI output until one of those specifications is adopted. --- @@ -494,6 +505,7 @@ createTerm(options: { width: number; height: number; terminfo?: TerminalInfo; + imagePoolBytes?: number; }): Promise ``` @@ -506,6 +518,9 @@ The optional `terminfo` value (from `detectTerminal()`; see `RuntimeCapabilities` from the static `Capabilities` it carries. When omitted, the Term uses the §7.1 baseline. +The optional `imagePoolBytes` sizes the Term's carved image pixel pool +([Graphics Specification](graphics-spec.md) §5.3; default 4 MiB). + ### 8.2 Render invocation ``` @@ -635,6 +650,20 @@ Snapshots enable higher-level frameworks to implement dirty tracking: a component whose inputs have not changed can reuse a previously created snapshot, avoiding the cost of re-packing its subtree each frame. +#### 8.3.5 img + +``` +img(id: string, props: ImgProps): Img +``` + +Creates an image directive: a void, self-closing element that participates in +layout as a leaf and does not participate in open/close balance — it MUST NOT +be paired with `close()` and has no children. The `id` provides element +identity as for `open()`. Its properties (`ImgProps`), the tier ladder, and +the Term-level pixel-surface registry it references are specified normatively +in the [Graphics Specification](graphics-spec.md); this section defines only +its existence, its plain-object form, and its void semantics. + ### 8.4 Sizing helpers These functions produce sizing-axis values for use in element layout diff --git a/specs/terminfo-spec.md b/specs/terminfo-spec.md index d800909..b2846f0 100644 --- a/specs/terminfo-spec.md +++ b/specs/terminfo-spec.md @@ -486,11 +486,14 @@ function createTerm(options: { width: number; height: number; terminfo?: TerminalInfo; + imagePoolBytes?: number; }): Promise; ``` -When `terminfo` is provided, the renderer initializes its private -`RuntimeCapabilities` from `terminfo.capabilities` and `RuntimeCapabilities` +The optional `imagePoolBytes` sizes the Term's carved image pixel pool +([Graphics Specification](graphics-spec.md) §5.3; default 4 MiB). + + dynamic fields at their baseline values. When omitted, the renderer uses the §7.1 baseline for all fields. From 5b92fb320e781e14059cb0100b9a12dd3f6efb1b Mon Sep 17 00:00:00 2001 From: Nate Moore Date: Fri, 9 Oct 2026 15:16:41 -0500 Subject: [PATCH 4/4] chore: deno fmt spec files --- specs/renderer-spec.md | 42 +++++++++++++++++++++--------------------- specs/terminfo-spec.md | 1 - 2 files changed, 21 insertions(+), 22 deletions(-) diff --git a/specs/renderer-spec.md b/specs/renderer-spec.md index f013aec..e159935 100644 --- a/specs/renderer-spec.md +++ b/specs/renderer-spec.md @@ -150,10 +150,10 @@ yields, or partial results. Each render transaction operates on a complete, self-contained snapshot of the UI. The renderer MUST NOT maintain an internal component tree or UI state across frames. The only state the renderer retains between frames is diffing state — -the cell buffers, the image registry, and the previous frame's placements -record (as defined in the [Graphics Specification](graphics-spec.md) §4.1) — -which is an implementation detail of output minimization and correct graphics -cleanup, and not observable to the caller except through reduced output size. +the cell buffers, the image registry, and the previous frame's placements record +(as defined in the [Graphics Specification](graphics-spec.md) §4.1) — which is +an implementation detail of output minimization and correct graphics cleanup, +and not observable to the caller except through reduced output size. ### 4.4 Double-buffered diffing @@ -415,10 +415,10 @@ This responsibility is limited to the hardware cursor's position and visibility. Cursor shape and blink rate remain caller-managed. Frames containing image elements at a graphics tier MAY additionally include -cursor-positioning bytes required by the graphics protocol ([Graphics -Specification](graphics-spec.md) §10.6), provided the post-frame cursor state -defined here is unchanged and caret-declaring frames still end with the -caret's cell. +cursor-positioning bytes required by the graphics protocol +([Graphics Specification](graphics-spec.md) §10.6), provided the post-frame +cursor state defined here is unchanged and caret-declaring frames still end with +the caret's cell. ### 7.7 Update transaction @@ -459,9 +459,9 @@ be a no-op for that step. `value` into the Term's private `RuntimeCapabilities`. The foundation update transaction does not emit bytes or alter renderer output as a consequence of a capability event. A focused capability specification MUST define any output -invalidation or immediate bytes required by its consumer; the [Graphics -Specification](graphics-spec.md) defines such bytes for graphics cleanup on -capability denial and resize (§11.2 and §10.5 there). +invalidation or immediate bytes required by its consumer; the +[Graphics Specification](graphics-spec.md) defines such bytes for graphics +cleanup on capability denial and resize (§11.2 and §10.5 there). **Return value.** `update()` returns a `Uint8Array` of bytes to write to the terminal immediately. An empty array is valid when the update changes no @@ -486,10 +486,10 @@ own renderer section, capability evidence, invalidation rules, and tests in a focused feature specification. In particular, color encoding, synchronized-output wrapping, pointer-shape -output, and Kitty keyboard mode setup are deferred to their respective -follow-up PRs; Kitty graphics emission is specified by the [Graphics -Specification](graphics-spec.md). The renderer continues to emit its existing -hardcoded ANSI output until one of those specifications is adopted. +output, and Kitty keyboard mode setup are deferred to their respective follow-up +PRs; Kitty graphics emission is specified by the +[Graphics Specification](graphics-spec.md). The renderer continues to emit its +existing hardcoded ANSI output until one of those specifications is adopted. --- @@ -657,12 +657,12 @@ img(id: string, props: ImgProps): Img ``` Creates an image directive: a void, self-closing element that participates in -layout as a leaf and does not participate in open/close balance — it MUST NOT -be paired with `close()` and has no children. The `id` provides element -identity as for `open()`. Its properties (`ImgProps`), the tier ladder, and -the Term-level pixel-surface registry it references are specified normatively -in the [Graphics Specification](graphics-spec.md); this section defines only -its existence, its plain-object form, and its void semantics. +layout as a leaf and does not participate in open/close balance — it MUST NOT be +paired with `close()` and has no children. The `id` provides element identity as +for `open()`. Its properties (`ImgProps`), the tier ladder, and the Term-level +pixel-surface registry it references are specified normatively in the +[Graphics Specification](graphics-spec.md); this section defines only its +existence, its plain-object form, and its void semantics. ### 8.4 Sizing helpers diff --git a/specs/terminfo-spec.md b/specs/terminfo-spec.md index b2846f0..df2e13b 100644 --- a/specs/terminfo-spec.md +++ b/specs/terminfo-spec.md @@ -493,7 +493,6 @@ function createTerm(options: { The optional `imagePoolBytes` sizes the Term's carved image pixel pool ([Graphics Specification](graphics-spec.md) §5.3; default 4 MiB). - dynamic fields at their baseline values. When omitted, the renderer uses the §7.1 baseline for all fields.