Contents

Reelwright architecture

Status: accepted 2026-09-29. Milestones M0–M3 and M5 are done, and M4, M6, M7, M8 and M10 are under way (roadmap). Toolchain: Godot 4.7.2-stable, ffmpeg 6.1 or later, Node 24, Rhubarb Lip Sync 1.14.0 (built from source); tools/toolchain.json pins them.

Reelwright lets one person run an animated series by talking to an AI showrunner. The showrunner never draws. It writes specs: small, versioned, schema-validated JSON (and Fountain) documents that a person can read, edit and diff. Deterministic code turns specs into assets and assets into frames. Everything the AI proposes arrives as a reviewable change set, and nothing it made is ever silently replaced.

This document is the map. Decisions with real alternatives live in adr/; the milestone plan and the changes proposed to it live in ROADMAP.md.

1. Principles

1. Specs, not pixels. The AI (and every generator) emits documents. Final frames come only from deterministic builders and the renderer. Tier 2 generative output (M9) is source material: normalized, frozen by content hash, then used like any other asset.
2. Decide upstream, draw downstream. Every choice (random or AI) is made in the core and recorded in a spec. The engine is a pure function from resolved specs to pixels: it makes no decisions, reads no clock, draws no unseeded random number.
3. Reproducible by construction. Same specs + same seeds + same builder versions ⇒ same assets and the same frames (defined precisely in §6).
4. Everything is a proposal. AI output becomes a change set with a diff and previews. The user approves, tweaks, locks or regenerates at any granularity, and can always undo.
5. Canon is sacred. Locked documents and locked subtrees never change without an explicit approved change set, even when generators or builders improve.
6. Film grammar is code. Rules like the 180-degree rule are validators and solver constraints over real geometry, not advice in a prompt.
7. Offline first. Tier 1 (procedural, local) must make a watchable show with no network and no API key. Paid providers make it better, never make it work.

2. System overview

A diagram, as its Mermaid source.

flowchart LR
  user([User]) -- chat --> panel[Chat panel<br/>Godot editor plugin]
  mcp([Claude Code / any MCP client]) -- MCP --> tools
  panel -- JSON-RPC --> sr[Showrunner agent]
  sr --> tools[Tool registry]
  tools --> cs[Change sets<br/>diff + previews]
  cs -- approve --> store[(Show Pack<br/>specs in git)]
  store --> val[Validators<br/>schema + semantic + film grammar]
  store --> gen[Samplers<br/>pattern + seed → instance]
  store --> comp[Compiler<br/>Episode IR → Timeline]
  gen --> store
  comp --> job[Render job]
  job --> eng[Godot engine<br/>builders · player · frame server]
  eng -- frames --> ff[ffmpeg<br/>mix · encode · deliver]
  adapters[[Adapters: LLM · TTS · lip sync · Tier 2]] -.-> sr & comp & gen

Two programs, one contract (ADR-0003):

People use Reelwright through Reelwright Studio (a standalone Godot app), the Godot editor plugin, the CLI, or any MCP client. Studio and the plugin host the same UI components (ADR-0011).

They talk two ways:

• Jobs (batch). The core writes a job directory (.reelwright/jobs/<hash>/job.json + inputs) and runs godot --path engine -- rw <command> --job <dir>. The engine writes frames or images and a result.json (frame hashes, timings, diagnostics). Used by the CLI, CI and render boxes.
• Session (interactive). The editor plugin connects to the core daemon (reelwright serve) over WebSocket JSON-RPC 2.0. It streams chat, shows proposals, and asks the in-editor engine for previews. Without an editor, the core starts a headless- capable engine worker for previews instead.

Studio's languages. Studio says every word through RwLang (RwLang.t, RwLang.n for a count), which looks it up in the chosen language's gettext catalog (engine/translations/<code>.po, Godot's TranslationServer); Settings › Appearance › Language chooses it (the computer's or the browser's by default), and it takes effect at Studio's next start. node tools/i18n/extract.ts writes the template (studio.pot) from every call and brings each catalog up to it; engine/translations/glossary.md keeps the words. The core stays in English, for Claude, the CLI and the logs, and sends nothing new: its messages and hints are in the template too, under the context core, and Studio shows what the core said through RwLang.core, which finds the message a line was made from and fills in its translation. What the core composes from pieces as it goes (the reading's staged lines, the timeline's business, a scene's opening, the directions and shot lines it reads, a preview's caption) it says with say() (core/src/said.ts): the English as before, and beside it (session protocol 1.8) the template and what fills each slot, which RwLang.said translates, agreeing with a character's gender from their pronouns (docs/design/translate-core.md). A show's own words (titles, names, scripts) are never translated; a show has its own language (show.json).

3. The Show Pack (data model)

A Show Pack is a directory, meant to be its own git repository. Documents are found by convention, <folder>/<slug>.<kind>.json, so tools, humans and editors agree on where things are and VS Code can validate by glob.

my-show/
  show.json                                 ShowManifest: format, active theme+style, providers
  theme/<slug>.theme.json                   ThemeSpec
  styles/<slug>.style.json                  StyleSpec (several; show.json picks the active one)
  patterns/{characters,sets,props,fx,graphics}/<slug>.<type>-pattern.json
  cast/<slug>.character.json                CharacterInstance
  sets/<slug>.set.json                      SetInstance
  props/<slug>.prop.json                    PropInstance
  fx/<slug>.fx.json   graphics/<slug>.graphic.json
  bible/bible.json                          lore, relationships, running gags, tone rules
  bible/continuity/<episode>.continuity.json  facts established per episode (M10)
  bible/memory.json                         ShowMemory: what the show remembers of every episode (M10)
  episodes/<slug>/<slug>.fountain           screenplay
  episodes/<slug>/<slug>.episode.json       EpisodeIR (the director's cut)
  episodes/<slug>/writers-room/…            premise, logline, beat outline (M5)
  audio/takes/<sha256>.flac                 approved voice takes, content-addressed
  audio/sfx/<slug>.flac or .sound.json      a sound effect: recorded, or a SoundRecipe
  assets/<sha256>.<ext>                     frozen Tier 2 assets (git LFS), content-addressed
  assets/<sha256>.json                      each one's provenance: source, license, steps, QA
  showrunner/NOTES.md                       the showrunner's durable notes on your preferences
  comments/<slug>.thread.json               CommentThread: people's comments on the work (M13)
  .reelwright/                              caches, jobs, renders, sessions (gitignored)

Document kinds (JSON Schemas in schemas/v1/):

Two definition-only schemas hold shared vocabulary: recipe.schema.json (the parametric geometry recipe used by kit pieces and props: flat nodes such as box, extrude, lathe, panel with openings, decal, array and subtract, driven by params through a restricted expression grammar) and pattern-rules.schema.json (pointer-addressed sampling rules shared by the prop, FX and graphics patterns).

Sound ids map to files by convention: sfx.<name> → audio/sfx/<name>.flac and music.<name> → audio/music/<name>.flac, both relative to the show root. Either may instead be a recipe, audio/sfx/<name>.sound.json or audio/music/<name>.sound.json, synthesized deterministically into .reelwright/cache/sfx/ (docs/design/m5-writers-room.md, "Sound recipes").

Conventions (ADR-0005):

• JSON Schema 2020-12, and every schema is also valid as a Claude strict tool or structured-output schema: closed objects, no recursion, anyOf + const discriminators, no map-shaped objects. Hierarchies are flat lists linked by parent ids. A CI lint enforces this.
• Every document starts with kind, schemaVersion, id, then a human name/title, then meta (status, locks, provenance, derivation).
• Ids are prefixed by kind (char.otto, set.marigold_bridge, ep.lighthouse_cold_open), so a reference says what it points at and a regex can check it. Local ids inside a document are short slugs (helm, sh040).
• Units live in field names: heightCm, positionM, yawDeg, durationSec, weightPx1080 (pixels at a 1080-line reference, scaled to the output resolution).
• Addresses inside documents are JSON Pointers. Patterns, locks, variations ("vary only /head/hair/style") and diffs all use the same addressing.
• One writer. Only the core writes specs, in canonical form: schema key order, 2-space indent, shortest round-trip numbers, trailing newline, so diffs are minimal and stable. Content hashes use RFC 8785 (JCS) canonical bytes + SHA-256.
• schemaVersion is MAJOR.MINOR. Minor versions only add optional fields. A major bump ships a reelwright migrate step and golden tests for it.

4. World-building: Theme → Style → Pattern → Instance

• ThemeSpec: what exists. Premise, genres, tone (with a never list of guardrails), setting, world rules (hard ones are enforced by the writers'-room validators), in-world language, and a catalog of set types, costume/prop/FX/graphics families and archetypes. Catalog entries point at the pattern that can generate them; entries without one are what the missing-asset pass offers to create.
• StyleSpec: how it looks and moves. Every field maps to a shader uniform, material parameter, animation parameter or solver weight. Swapping the active style re-renders the show without touching story data. Two parts of a style reach past rendering and are treated as design changes: design transforms (e.g. bigger eyes for an anime restyle) and timing.preferredFps. Applying those to locked characters produces a change set with before/after model sheets.
• Patterns are seedable distributions. A CharacterPattern is a list of rules addressed by JSON Pointer into the instance (range, choice, const), plus archetypes, conditional rules for correlated traits, costume layer rules, a palette plan (pick / ramp / derive-from-slot, solved in OKLab, then nudged to satisfy the style's color rules) and ensemble constraints for generating a cast together (minimum silhouette and palette distance, height spread, unique hair).
• Instances are fully resolved: no ranges, no randomness. They record meta.derivation (pattern id, pattern hash, seed, sampler version), so the core can reproduce them, tell when a pattern has drifted, and re-derive on approval.

Locking. meta.status: locked freezes the document; meta.locks[] freezes subtrees ("regenerate the costume, keep the face": lock /head). Samplers treat locked pointers as fixed inputs.

Dependency graph. Nodes are documents and derived artifacts (built rigs, model sheets, timelines, renders); edges come from references and derivations. Each derived artifact stores the hashes of its inputs. reelwright status lists what is stale; reelwright impact style.saturday_morning answers "what changes if I edit this?" before you edit it.

5. Characters, sets and the stage

Stage space: 3D stage, 2D actors (ADR-0004)

Sets are built in 3D stage space (meters, +Y up). Characters are 2D cutout rigs drawn into camera-facing cards standing on floor marks. Consequences:

• Real cameras: lens in mm, dolly, crane, orbit and parallax come from actual geometry; cuts between set angles are just different camera positions.
• Real occlusion: the Chief crouching behind a console is hidden by the console's geometry. No hand-sorted layers.
• Film grammar is computable: the axis of action is a line on the floor, and "which side is this camera on" is a sign test.
• The look stays 2D: flat or cel shading, ink lines on silhouettes, painted textures and atmosphere all come from the StyleSpec. Distant scenery is backdrop cards at real depths.
• Path to 3D (M10): replace a card with a 3D character; nothing else in staging changes. Done for rig template biped_3d.v1: a figure of shaded primitives on the card's own skeleton and pose solver (docs/design/m10-3d-characters.md).

Characters

CharacterInstance → character builder (versioned, in the engine) → rig:

• Rig template (biped_toon.v1): a fixed bone hierarchy with standard names (hips, spine, chest, neck, head, upper_arm_l, forearm_l, hand_l, thigh_l, shin_l, foot_l, …) and named part slots. Built as Skeleton2D + Bone2D, with Polygon2D parts skinned to bones (bendy limbs) or rigidly parented (jointed). Because names are standard, gestures, walk cycles and posture changes retarget to every character on the template.
• Part library (toon.v1): parametric vector parts selected by the instance's enums (head shapes, eye styles, garment cuts…), tessellated to polygons with style-driven ink strokes (weight, taper, boil). Vector all the way down, so any resolution is crisp.
• Turning. The head is a 2.5D rig: facial features sit at angles on an implicit head volume and are re-projected for any yaw and pitch, so heads turn smoothly. The body swaps between front, three-quarter, profile and back views, mirrored for the other side. The card shows the character at character facing minus camera direction, so the correct view falls out of the staging.
• Faces. Expressions are a small FACS-like parameter set (brow raise and angle, eye openness, squint, mouth curve and openness, cheek raise, head tilt…) blended continuously. Mouth shapes follow Rhubarb's A–H + X vocabulary as parameters (open, width, round, teeth), so lip sync blends with a smile instead of pasting over it.
• Model sheet. The engine renders turnaround, expression grid, mouth chart, color callouts and a height line-up against the cast. It is the approval artifact and the reference for canon-drift checks.

Sets

A SetInstance places kit pieces (prim.box, prim.extrude, prim.panel, …, or kit.<kit>.<piece> recipes from a SetPattern) in stage space, and adds everything a director needs: marks (position, facing, posture, seat), entrances with doors, walk paths, named camera setups (the set's standard angles, including which are good establishing shots) and lighting presets (key, fill, ambient, rim, practicals). Kit recipes are data (small CSG-style node lists with parameter expressions), so "build an alien marketplace" can mean new pieces without new engine code.

6. Determinism

Four levels, each with its own test (ADR-0002):

Rules that make it hold:

• Seeds by name, not by draw order. seed(child) = u32(sha256(le32(parent) ‖ path)) with a stable path such as char.otto/head/hair/style. Adding a new attribute never reshuffles existing ones. Samplers draw from PCG32 streams (core only). Both languages implement deriveSeed, checked against shared test vectors.
• The engine uses stateless randomness. Line boil, grain, flicker and starfields use pcg_hash (a 32-bit integer hash, also usable in shaders) of (seed, frame, index), so any frame renders alone.
• Playback is a pure function of the frame number. evaluate(frame) sets every property: no accumulated state, no physics, no GPU particles without replay. Any stateful effect (springs for overshoot, secondary motion) is replayed deterministically from the shot start. That is what makes single-frame previews, timeline scrubbing, chunked and parallel rendering, and golden tests work.
• Versioned builders (ADR-0007). Any change that alters a builder's output ships as a new major version, and golden tests fail otherwise. Instances pin the builder major. Locked instances keep their major until an approved migration shows before/after model sheets.
• No hidden inputs. Engine code may not call randomize(), randi(), randf() or Time.*; resolvers may not call Math.random() or Date.now(). A CI grep enforces it.

7. The episode pipeline

premise → logline → beats → scenes → screenplay.fountain        (writers' room, M5)
        → missing-asset pass: characters/sets/props the script needs   (M5)
screenplay → EpisodeIR: performance + coverage                   (director, M4)
EpisodeIR + voices → Timeline (frame-exact)                      (compiler, M1)
Timeline → frames → mix + encode → deliverables                  (engine + ffmpeg, M1)
chat edits → IR patches → diff → apply → re-compile → re-render  (re-direction, M7)

Episode IR: performance and coverage (ADR-0006)

Each scene has two independent parts, the way live action is shot and cut:

• cues: one continuous performance. Lines (with emotion, intensity and delivery notes), moves, looks, gestures, expressions, posture changes, prop business, sound effects, FX, music cues and beats. Timing is relative: by default a cue follows the previous one; after chains, with overlaps (optionally anchored to a word of a line, e.g. point on the word "kid"). Audio lengths are unknown until voices exist, so absolute times are compiled, never authored.
• shots: coverage over that performance. Each shot has a cut point anchored to a cue (with snap: silence so cuts never land mid-word) and a camera: either one of the set's named setups or a framing intent (MCU on char.otto, eye height, optional over-the-shoulder) that the framing solver turns into a camera on the legal side of the axis.

Re-cutting never changes the performance. Re-timing a line automatically moves the cuts anchored to it. J- and L-cuts come for free because picture and sound are separate.

Compiler: EpisodeIR → Timeline

Deterministic, in the core:

1. Voices. Pick a take per line (approved recording → cached TTS → generate) and measure its exact length.
2. Timing. Resolve the cue graph (topological; cycles are errors), in seconds, then to frames at the show's fps. Cut points snap to frames or to the nearest silence.
3. Blocking. Walk paths between marks with the style's gait, look-at targets, posture transitions, and occlusion hints.
4. Framing solver. Framing intents → camera position, orientation and lens from subject positions and body proportions. Closed-form and deterministic.
5. Performance layers. Visemes from lip sync, expression curves, gesture clips, procedural idle (breathing phase, seeded blinks avoiding mid-line, eye darts).
6. Film-grammar validation (errors and warnings, each with a JSON Pointer and a fix hint that goes back to the director for repair): 180-degree rule, 30-degree rule and jump cuts, screen direction on exits and entrances, eyeline match in shot/reverse-shot, establishing shot at a new location, shot length within the style's cutting bounds, speaker visibility, and cuts on silence.
7. Emit the Timeline (engine input) and the audio plan (every clip at an exact sample offset with gain, fades and ducking).

8. Performance: voices and lip sync

Provider chains, first available wins, with per-show order in show.json (ADR-0009):

• TTS: recorded (approved takes) → cloud (M6+: ElevenLabs, Cartesia, Azure, etc. behind one interface) → kokoro (local, Apache-2.0, word timestamps). Every take is cached by hash(text, voice binding, delivery, provider version) and stored as FLAC.
• Lip sync: rhubarb (MIT, v1.14.0, built from source for arm64; given the line text via -d, extended shapes G/H/X) → later a phoneme-alignment provider (TTS phoneme timings, or forced alignment) mapped to the same A–H/X shapes. Results are cached by audio hash and text.
• Mix: the audio plan compiles to one ffmpeg filtergraph (adelay in samples, amix with normalize=0, gains, fades, sidechain ducking under dialogue), then two-pass loudnorm to the show's target (−16 LUFS web, −23 LUFS broadcast). The mix is sample-exact and byte-reproducible.

9. Rendering (ADR-0008)

• Frame server (primary). The engine renders the stage into a SubViewport at the exact output size. For each frame it evaluates the Timeline, then calls RenderingServer.force_draw() and reads the texture back. It runs with the render loop disabled and a fixed step. This one path serves full renders, single stills, storyboards, model sheets, golden tests and chunked parallel renders. It keeps working when the macOS window is minimized.
• Movie Maker (supported alternative). --write-movie with stretch=viewport at the output size, physics_jitter_fix=0, and a pre-created, emptied output folder, which works around three confirmed 4.7.2 bugs. A CI parity test renders the same frames both ways.
• Renderer. Forward+ is canonical (it has 2D MSAA, which Compatibility lacks): Metal on macOS, Vulkan elsewhere. Linux boxes and CI have no GPU, so they use xvfb-run with Mesa's software Vulkan (lavapipe); --headless cannot capture frames. Golden frames are exact for CI's canonical renderer; other renderers are compared for information (mean difference under 0.1 levels).
• Encode. From the PNG frames, ffmpeg makes H.264 (BT.709, limited range, correct color tags, CRF 16), and optionally ProRes 422 HQ or 4444 masters, stems, WebVTT/SRT captions (free, because every line's timing is known), and an OpenTimelineIO export of the edit for Resolve, Premiere or Kdenlive.

10. The showrunner

The main interface. It is an agent with tools; every tool that changes anything returns a proposed change set, never a direct write (ADR-0003 explains why it lives in the core).

• Tools.
  • Read: get, list, search_bible, impact, estimate_cost, render_preview.
  • Propose: create_theme, create_style, generate_characters (count, archetypes, ensemble constraints), vary (N options over chosen pointers), revise (id + pointer edits), build_set, generate_props, restyle_show, write_episode, direct_episode, revise_shot, render.
  • Decide (the user, via UI buttons or explicit confirmation): approve, reject, lock, unlock, undo, redo.
• Previews in chat. Model sheets, set thumbnails from each camera setup, style swatches and storyboard frames, all rendered by the engine's frame server. "The second one, but taller" works because variations are numbered proposals and "taller" is a pointer edit on /body/heightCm.
• The repair loop. Tool inputs are strict JSON Schemas. Outputs pass schema validation, then semantic validation (references exist, marks are on the set, words exist for word-anchored cues), then film grammar. Diagnostics go back to the model with pointers and hints until the proposal is clean or a retry budget is spent.
• Context. The bible, theme, style summary and cast summaries form a stable, cached prompt prefix; scene-specific canon is added by id. Long context plus prompt caching replaces a vector database until a bible outgrows it.
• Memory. showrunner/NOTES.md (committed) holds durable preferences ("keep cold opens under 40 seconds"); transcripts stay in .reelwright/sessions/.
• Provider. One small interface, implemented first with the official Anthropic SDK. Defaults: claude-opus-5-5 with adaptive thinking; effort and model are set per stage in show.json; refusal fallbacks are on; forced tool_choice is not used (the current models reject it; tools are strict and the prompt names the tool). The key comes from the environment, then a key file only its owner can read, then the macOS Keychain, then the SDK's own credential chain. It is never written to disk or logs.
• Cost and latency. Cached prefixes, per-stage models and effort, token estimates before expensive stages, budgets in show.json that stop spending, the Batches API for bulk offline generation, and a --dry-run that shows the plan and the estimate.
• Tests without a key. The provider records request/response pairs keyed by the hash of the canonical request. CI only replays. Refreshing fixtures is an explicit command.
• MCP. reelwright mcp exposes the same tool registry over MCP. From M3, Claude Code or Claude Desktop can act as the showrunner before the in-editor panel is finished.

11. Change sets, undo and history

A ChangeSet has a title, a rationale, its author (user or showrunner), per-file operations (add, replace, remove) with base hashes (so a stale proposal cannot apply), RFC 6902 patches for display, previews, and the impact list from the dependency graph.

• Approve: the core applies it atomically and, if the Show Pack is a git repo, commits it with the change set's id in a trailer.
• Undo: apply the inverse as a new change set. History is never rewritten.
• Conflict: an out-of-band edit (a person editing JSON in their editor) changes the base hash; the proposal is marked stale and regenerated against the new base.

12. Tier 2 generation and asset QA (M9)

Adapters for image, texture, 3D, voice, music and SFX sit behind narrow interfaces chosen at build time, not hard-coded. One API used for this project shut down in September 2026, which is the reason for the rule. Every generated asset runs one pipeline:

1. Generate.
2. Segment into rig parts on transparent backgrounds.
3. Vectorize or keep raster.
4. Snap to the palette, then line and shading passes to match the StyleSpec.
5. Attach to the rig template.
6. QA: rig validation, part naming, palette compliance (ΔE in OKLab), silhouette readability at small sizes, consistency against the approved model sheet.
7. Freeze by content hash with full provenance: generator, model, prompt, seed, date, license.

A failure loops back with the QA report. Frozen assets make Tier 2 content reproducible even though the providers are not.

Art the person brings in runs the same pipeline from step 4, and is the first source built (reelwright asset import, import_asset; design). Its first use is painted backdrops: a set backdrop that draws a frozen equirectangular image instead of a procedural generator. A frozen asset is stored before any document names it, as kept voice takes are, since change sets carry JSON and text only; validation checks that each one a document names is pinned by its hash, still holds those bytes, and has a licensed provenance.

13. Quality gates

Tooling choices are in ADR-0010.

14. Repository layout

docs/                ARCHITECTURE.md, ROADMAP.md, adr/
schemas/v1/          JSON Schemas (source of truth; TS types and GDScript classes are generated)
core/                TypeScript: @reelwright/core, `reelwright` CLI, MCP server, showrunner
engine/              Godot 4.7 project: addons/reelwright/{runtime,editor}, cli/, tests/, goldens/
examples/shows/      demo Show Packs (original content, CC BY 4.0)
tools/               toolchain.json (pinned versions + checksums), install/build scripts
.github/workflows/   CI

15. Toolchain (pinned in tools/toolchain.json)

16. Risks and open questions

• Name. The project keeps "Reelwright"; its home is reelwright.diy. An unrelated early-access AI-video product uses reelwright.ai. There is no US trademark filing for it, but consider filing our own before a public launch.
• "Showrunner". Fable Studio's Showrunner is a well-known product in exactly this niche. Use "showrunner" only as a lowercase description of the agent, never as a product name.
• Patents. Google holds US 9,786,032 and US 10,089,715 on parametric, scalable vector 2D characters with animation templates (active to 2036). Get a freedom-to-operate review before any commercial use of the character system.
• Rhubarb is dormant (last release April 2025) and English-centric. It is behind an adapter with an alignment-based successor planned.
• Software rendering speed. Lavapipe renders 1080p slowly. CI renders goldens at 640×360 and the demo at 720p; real render boxes should have GPUs.
• Tier 2 provenance. Some permissive voice and image models were trained on unclear data. Provenance travels with every asset, so a show can audit and replace them.