Contents

M6 Performance

M6 makes characters act: hands that hold a pose, gestures and faces chosen from what is said and done, listeners who react, and motion with the snap of the style. It ships in two parts.

• Offline (this design): a larger gesture library with held poses, a director that performs, idle life, and the motion style fields the engine reserves today.
• Later: cloud TTS adapters with emotion control and alignment, and alignment-based lip sync. These need provider keys, so they wait for them.

Read with docs/design/m1-engine.md (pose solver, sampler), docs/design/m1-compiler.md (performance channels) and docs/design/m4-director.md (the heuristic director).

Where things stand

• Gestures. The library (biped_toon.v1.gestures.json) has 8 gestures: point, shrug, nod, head_shake, wave, fist_pump, facepalm, wrench_wave. Each runs for its library duration. The engine envelope rises to the stroke and falls back over the last 20%, and nothing can hold it. "Arms folded" in a screenplay cannot be performed.
• Director. The heuristic director emits postures and looks from action verbs, and the line's emotion from its parenthetical. It emits no gestures and no expressions, so only the model director gestures.
• Idle.
  • In core: blinks and breathing, seeded per actor.
  • In the engine: fidget, eye darts and arcs, stateless by RwHash.
  • Reserved but unused until Part 2: fastActionStep, and smears (it needs a per-frame speed the actor state does not have).
• Bug. motion.gestureIntensity is applied twice. The core uses it as a cue's default intensity, and the engine scales by it again, so the dial acts quadratically.

Part 1: held gestures, a larger library, a director that performs

Held gestures.

• A library gesture may be holdable. Its keys run from rest to the held pose at strokeSec and back to rest at durationSec.
• GestureCue gains an optional hold: {until: <cue id>} or {sec}. A held gesture's Timeline span is in, then hold, then out.
  • The core places it: the out phase ends at the until cue's start, or sec after the stroke.
  • The engine maps the phase: in over strokeSec, the stroke pose held, out over the remaining time.
• A move or a posture change releases a hold (the director ends it there). Two held gestures on one hand are a diagnostic.

Library. These join the 8 existing gestures, all on biped_toon.v1 with mirrored either-hand tracks:

(implementation) A gesture whose verb the action writes in capitals ("She TRIPS") is done at intensity 0.9; any other at the cue's default. That word is emphasis, not a noise: it asks for no sound unless the show has one by that name.

Each needs a model-sheet-style review still at its stroke, and a test that it moves the rig there and returns to rest. Holdable ones also need a test that the held pose is stable.

Director. The heuristic director performs. All rules are deterministic, scored, and spaced so nobody gestures more than once every 2 s:

• Action verbs become gestures: "folds his arms" becomes arms_folded held until the character's next move or the scene's end; "shrugs", "waves", "points at X", "scratches his head" and "claps" map directly. (implementation: every gesture a sentence writes, in its order, each after the one before: "points at the Chief and shrugs" points, then shrugs. Each is made at whoever its words name: "waves at Otto". A gesture the script writes is never dropped by the spacing: it waits for the one before it to finish, and lets go of a held pose. lean_back keeps no hand, so "leans back and folds his arms" holds both. The script editor's read-along found all three misses.)
• Lines become expressions and beat gestures:
  • an expression cue at the line, from its emotion and intensity;
  • for intense lines, a beat gesture by emotion: excited is fist_pump or thumbs_up; angry is a pointing jab; confused is head_scratch or palms_up; dismissive is head_shake; proud is hand_to_chest; (implementation: played as strongly as the line, so a 0.9 line's thumbs-up comes up to the chest, and on the word the line leans on)
  • (implementation, the third product audit's finding 6) any other on-screen line of four words or more gets a small hand beat on the word it leans on (director/stress.ts: a word the parenthetical quotes to stress, else one in capitals, else the last word of the first full sentence that carries meaning, a name of address after a comma left out: "something BIG", "a LIGHTHOUSE", "away from THINGS, kid"), by its feeling: beat (an open hand's downstroke) for neutral, happy, excited, surprised or annoyed lines, beat_fist for angry and determined ones, beat_open (the palm turned up to offer the point) for curious, confused, amused, smug and happy ones. Afraid, worried, sad, tender, disgusted, contemptuous, embarrassed, tired, bored and deadpan lines keep the hands still. It plays as strongly as brings the hand up in front of the chest, whatever the style's gesture size: 0.85 of the library's beat, and 0.15 more for an intense line, as the engine plays it (the cue's strength is that over the style's 0.6 + 0.8 × gestureIntensity, at most 1). A beat lifts the arm from where it hangs, so a weaker one is a lower one: at 0.57, as it was, the hand reached only the waist, under the foot of the medium shots and close ones a speaker is cut in, and the gesture pass dropped every beat of the fourth product audit's new show (gesture-framing.test.ts, "a show made from words"). Where the hands cannot beat, the body does (beat_lean: the shoulders lift and drop, the chest leans in and the head dips on the word): the hands full or held in a pose (the Chief, arms folded, said "held together with tape and optimism" stock still for 2.17 s), and where the gesture pass finds no shot shows the hands (Lunch Rush's Jada, in an MCU at the cafeteria table, talked 2.75 s still); a beat goes only when not even that is seen. One beat is not enough for a long line: more beats fill it, on the words that carry meaning, so that no stretch of it runs 1.4 s unmoved as it is estimated to play (beatFill, at most three more, 1.2 s apart at least), after an intense line's own gesture too. On the words, they go with the line wherever it falls: the idle layer's motion runs on the episode's clock, and Nell's "It looks like a muffin" measured still 1.5 s or 2.25 s by where its line fell. A line whose feeling keeps the hands still (afraid, sad, tender, deadpan…) beats quietly with the body. One hand holding something leaves the other to gesture with (Lunch Rush's Kowalski, a megaphone in hand, had none of his three); a thing held in both hands, or a gesture that takes both, goes to the body. A person's body only: other plans have no beat. The pick is seeded by the speaker and the line, so the same script stages the same beats;
  • "yes, aye, right" lines nod, and "no" lines head_shake.
• Listeners. In long turns, listeners react with nods and expressions. They are seeded per actor and placed on the speaker's pauses, never over a cut.

Fix. gestureIntensity applies once, in the engine. A cue without its own intensity gets 0.5 from the core. The Lighthouse goldens may move. Review them, and pin the reason in CHANGELOG.

Part 2: motion style

• Velocity. The sampler computes the actor state at f and f−1, which keeps it a pure function of f. The pose solver then derives hand and foot speeds. RwActorState gains limb_speed per end effector.
• smears. Above smears.threshold, a fast limb draws a stretched copy along its motion: one frame, toon-shaded, in the style's ink.
• fastActionStep. Characters animate on characterStep (twos), and a fast action (a gesture's stroke, a limb over the threshold) samples on ones, as cel animation does.

As built.

• Speed needs the rig's bones, which the sampler does not have, so the state carries the drawing before it instead of a speed. While a gesture or a gait plays, RwActorSampler.sample(f) sets state.previous, the drawing one step back, and state.step. The pose solver measures speed from the two (RwPoseSolver.limb_speed, relative_speed). Each hand and foot's middle moves some body heights per second; 5 heights per second is a relative speed of 1. Measured on Otto at a cue's default intensity (0.5), a wave, fist pump, facepalm, clap or head scratch peaks at 4 to 5.5 heights per second, a point, shrug or thumbs-up at 1.9 to 2.8, and hands on hips at 0.4. So Saturday Morning's threshold of 0.7 (3.5 heights per second) catches the fast beats, at their peak; a point joins them at full intensity (5.4).
• Fast actions: the sampler asks the card's probe (RwCharacterCard.motion_probe, the solver's relative_speed) how fast the limbs move between f and f−1. At or over the threshold, that frame is drawn on fastActionStep. The probe runs only while the actor acts, and only when fastActionStep is finer than characterStep. Bright Modern draws everything on ones, so it never pays for it.
• Smears: RwPoseSolver.smears_between lists every limb end at or over the threshold (RwSmear: bone, from, to, speed). RwCharacterRig draws each one as a short teardrop of the limb's own fill with an ink edge, just under the limb, plus three ink speed lines trailing back toward where it was. It is opaque, because the card cuts alpha at one half, and is recomputed every drawing, so it lasts as long as the speed does.

Part 3: a show's own gestures and walks (product audit #17)

Performance was a closed list: the gestures loaded only from the engine's install, the walks were the show's one style, and an unknown gesture was skipped without a word. A conductor's baton or a signature swagger could not be made.

• A show's gesture libraries.
  • gestures/<slug>.gestures.json is a GestureLibrary (1.3, with an id, a name and meta) for one rig template.
  • It is laid over Reelwright's by gesture id: one of the same id is played instead of Reelwright's, any other is added (overlayLibrary).
  • The compiler times them (loadGestureLibraries(characters, show)), the director reads them, and the render job carries them to the engine (documents.gestures, RenderJob 1.4). The engine lays them over its own (RwGestureLibrary.use_show), for each template on their skeleton, so the 3D figure plays them too.
  • The validator checks the bones are the template's, one id each, the stroke inside the gesture and the keys in order.
• The pose editor.
  • Character studio, Poses, New (or Edit on one of Reelwright's, which starts the show's own version of it) opens RwGestureEditor.
  • A gesture is keys at times through it: rest at the start and the end, the pose at the stroke, and any between. At each key a bone of the template is turned (out and forward on a limb, a turn and a nod on the trunk and head) or moved: the pose solver's own channels.
  • Each change is posed live on the character, held at the key, before anything is kept (RwGestureLibrary.set_draft). Play runs it.
  • Save keeps it as the person's own change (save_gesture); remove_gesture takes one out, and Undo takes either back.
• The script reads them by name. "Otto salutes" plays the show's salute; "the Chief tips her hat" plays tip_hat (showGestureEntry: the id's first word in the third person, the rest after it). Without the show's library, the words are no gesture.
• An unknown gesture is said. It is gesture.unknown in the compile ("biped_toon.v1 has no gesture "salute"; char.culpepper plays nothing there"), with the fix: pose it in the character studio and save it, or use one the library has. Studio's timeline counts it among its compiler notes, and the note's tooltip says it.
• A character's own walk (CharacterInstance 1.6 walk): a style of the show's walks or one of its own (swagger: a pelvis twist and a lean back; stomp; tiptoe, up on the toes), and a pace that scales the gait speed the compiler blocks with. Without one, the character walks as the show does.

Part 4: the pose maker, and poses held until released (product audit #17)

Part 3 keyed a gesture with sliders, one bone and one number at a time. A pose is made by taking hold of the character: a hand pulled up to the brow, a foot planted wider, the hips dropped into a crouch, the head turned to look.

• Handles on the stage. While the pose editor is open, the character studio draws a handle on each hand and foot (a dot), on the head and the chest (a ring) and on the hips (a square), on the card or the 3D figure as it is drawn, at any angle of the turntable (RwPoseHandles).
• Dragging one solves the rig (RwPoser):
  • A hand or a foot reaches for the point it is dragged to with the two limb bones above it: the upper arm and forearm, the thigh and shin. A paw reaches the same way.
  • The head and the chest turn toward the drag.
  • The hips move with it (their offsetCm). The legs stay planted, since the pose solver's own leg IK bends the knees under them, so a drag down is a crouch with the feet on the floor.
  • The solve is damped least squares over the chain's channels. Its Jacobian is measured by turning each channel a little and solving the character as it is drawn: its state, its angle, the style's squash and stretch, card or figure alike. So the hand lands where it is dragged on any character.
  • What the screen cannot say (how far toward the camera a hand is, on a flat card) is settled as a person reaches: the arm forward and a little out, the elbow and knee soft. A drag that starts from rest starts from that pose, so an arm comes up in front of the body, not behind it. Turn the character and drag again to place it in depth.
  • Every value stays within the limits the rig template gives each bone's channel (limits in its rig.json). A reach out of range stops at its nearest; an elbow never bends backwards.
• Keyed on a strip (RwGestureStrip): the gesture's time with a diamond at each key and the stroke marked. Click a key to pose it, drag it along, double-click to add one, Delete to take one out. Each key's ease is drawn as the curve it moves on by. The playhead scrubs the pose live, Play runs it, and Save keeps it in the show's library (save_gesture).
• The fingers take a preset grip while the gesture is up: relaxed, a pointing finger, an open palm, a fist, a grip, a thumb up (GestureLibrary 1.4 handShape).
• As posed. The studio poses and shows a gesture at full intensity, and a cue that names none plays Reelwright's at 0.5. So a gesture says its own default (GestureLibrary 1.4 intensity), and the pose maker saves 1: the wave posed above the head is above the head in the episode. A cue's own intensity still scales it.
• A pose to hold. New pose… starts one that is holdable, with both hands: arms folded, hands on hips, or anything the show needs held. The validator holds a show's gestures to the rig's limits (gesture.beyond_limits), as the pose maker does, and Reelwright's own keep to them too.
• Held until released (EpisodeIR 1.56 hold: {"untilReleased": true}).
  • A holdable gesture's pose is held from its stroke with no end of its own. It lets go:
    • as their next gesture with an arm it holds starts (so the hand goes straight from one to the other; a nod takes no arm and plays over it);
    • for prop business on those arms, a move or a change of posture;
    • or at the scene's end, whichever comes first.
  • The timeline inspector's Holds chooses Plays once, For a time (the length under it), Until released, or shows the cue a hold waits for. A beat that cannot be held says so.
  • set_gesture takes the hold, and the model director is told of it.
  • The heuristic director holds a show's own pose as it holds arms folded, when a script names it ("Otto salutes" holds the salute when it can be held).

Part 5: poses keyed by hand (product audit 2026-10d, finding 10)

Nothing could be corrected at the drawing level: an elbow through a table or a hand that should rest on a shoulder could only be fixed by changing what drove it. A pose is now keyed by hand on a character at a moment of a scene, over what the director made, as an animator's correction layer goes over generated motion.

• What it is (EpisodeIR 1.63 Scene.poses, not a cue): whose it is, which of their poses (a gesture of their rig's library held as at its stroke, the show's own from the pose maker or Reelwright's), the hand a one-handed one is made with, and the frames it comes in over before its moment, is held for after it, and goes out over (6, 12 and 6 unless the person says).
• Anchored and kept by what plays there (core/src/compile/poses.ts): keyed at a moment, it is anchored on the latest word of a line, or line or beat start, at or before it (the first after it otherwise), as a cut is, and keeps what that was: the word, the line's words or the beat's action. A new take of the line carries it to the same word. Where the scene no longer says it there, the compiler does not play it and says so (pose.words_changed), and directing the episode again drops it, saying so (core/src/director/kept-poses.ts); while it does, the new direction keeps it.
• Played over the director's body (RwPoseSolver._key_poses): the sampler weighs each pose in over its in frames, holds it and weighs it out (RwActorSampler.pose_weight). The solver solves the pose on its own, at the actor's posture, as the pose maker shows it, and blends the bones it poses toward it by that weight, after everything else: an arm it moves goes whole (as a gesture takes the arm from the posture), the legs follow when it moves the hips, the hands take its shape once it is more than half in, and the rest of the body is as directed. A body plan takes one the same way. RwActorState.pose_weight is the strongest pose's weight at the frame, for the idle layer to yield to.
• On the timeline: each character's lanes have a Poses row under them, folded away until the arrow before their name opens it (remembered per character), so no row moves. Right-click it, or the lane, for Pose keyed by hand: the character's poses, the pose maker's first. The inspector sets the pose, the hand and the three lengths in frames; a drag keys it at another moment; Delete takes it away. Each is set_pose_key, so Undo takes it back.

Watch it

Direct The Salvage Claim heuristically and render it. The Chief folds his arms on the bridge and holds it until he trudges to the hatch. Otto fist-pumps on "Our salvage!" and scratches her head at "Was that supposed to happen?". The listener nods. A fast wave smears.