This guide uses Lumen Review Desk, an original fictional dashboard with a filter, a detail drawer, an activity list, and a review action. You can download its recording, frames, implementation, and tests, then use the same process with your own interface. The goal is a runnable local reconstruction with a short behavior specification: what the user does and what should happen next. You need basic familiarity with files, a terminal, and a browser; the guide does not assume you already know how to turn a recording into requirements. Connecting the frontend to a real service requires its API and data rules.
Choose the scope before sending the recording to an agent
First decide whether you need a visual prototype, a working frontend with local data, or an integration into an existing product. These require different inputs. If the brief only says "copy this video," the agent has to invent the missing behavior and choose an architecture without knowing which decisions matter.
| Your material | Useful approach | What to settle first |
|---|---|---|
| An interface you own, with source code | Give the agent the recording and relevant components. Ask it to reuse the existing controls, styles, and data layer. | Which behavior is changing, and which existing behavior must remain? |
| A permitted recording without source | Provide selected frames, exact visible text, a transition timeline, and local sample data. Build a frontend prototype. | Which hidden behaviors are unknown, and who will choose them? |
| One screenshot | Reconstruct that visible state and write proposed interactions separately. | A screenshot supplies no evidence for the next screen or how to reach it. |
| A desktop recording with no mobile view | Match the observed desktop layout and design a separately specified narrow layout. | Which elements stack, disappear, scroll, or become full width? |
| A dashboard connected to a real service | Start with local sample records, then connect the documented API. | Request and response shapes, authentication, permissions, validation, and failed requests cannot be recovered from pixels. |
Prepare a recording you are allowed to use. Remove private names, account details, internal URLs, tokens, and sensitive values before sharing frames. Supply approved images and fonts separately when available. If assets are missing, name the replacements in the brief; a guessed logo or compressed image extracted from a video should not silently become a final asset.
For a new prototype, plain HTML, CSS, and JavaScript are sufficient for this example. Inside an existing application, keep its framework and component system unless the task requires changing them. Specify the route or entry point and the files the agent may edit. You should be able to start the result locally without connecting production accounts.
Try the working example
Download the Screen Recording UI Rebuild Kit v1.1. It contains the original recording, six reference frames, state contract, deliberately incomplete mock, rebuilt interface, browser test, dependency lockfile, saved results, visual diffs, and review checklist.
Extract the ZIP and open a terminal in its top-level directory, where package.json and release-gate.py are visible. Start a local server bound to your computer:
python3 -m http.server 8766 --bind 127.0.0.1
Leave that terminal running. Open http://127.0.0.1:8766/rebuilt/ and follow this path: Ready, Homepage launch review (LN-104), Show activity, Mark reviewed, then Escape. The Ready count should fall from two to one. LN-104 should leave that filter, LN-111 should remain, and focus should return to the Ready filter button. Reloading resets the example data.
To execute the browser checks, you need Python 3 for the server, Node.js 22 or later, npm, and an installed Google Chrome. In a second terminal in the same directory, run:
npm ci
npm test
python3 release-gate.py
The test uses Playwright's chrome channel. If Chrome is unavailable, install Chrome or provide the path to an existing Chrome executable through CHROME_PATH. A server on another local port can be selected through LUMEN_BASE_URL. The Playwright browser documentation explains installed browser channels and their platform requirements.
npm test launches a browser and writes new results. python3 release-gate.py reads the saved files and checks their structure and reported outcomes. Running the Python command alone cannot verify your modified code. Keep a copy of the original evidence/ directory before a rerun if you need to compare results. A successful browser run writes evidence/fresh-run.json; a failed run exits with an error and leaves that completion marker absent.
Extract frames that explain the interaction

Watch the recording once at normal speed to identify the task. Watch it again slowly and note every action that changes selection, visibility, content, status, counts, position, or navigation. Keep the settled frame after each change. For a moving drawer, also retain its start and an intermediate frame if reproducing motion is part of the brief.
A short clip can be inspected manually. For a longer one, FFmpeg can generate candidates. These commands assume FFmpeg and FFprobe are installed and the input is named recording.mp4; change the filename for another format.
# Inspect duration and encoded frame dimensions.
ffprobe -v error -select_streams v:0 \
-show_entries stream=width,height:format=duration \
-of json recording.mp4
# Extract two candidates per second.
mkdir -p frames candidates
ffmpeg -i recording.mp4 -vf fps=2 frames/frame-%04d.png
# Extract candidates at larger visual changes.
ffmpeg -i recording.mp4 \
-vf "select='gt(scene,0.35)',showinfo" -fps_mode vfr \
candidates/scene-%03d.png
# Extract one frame near a moment you have inspected.
ffmpeg -i recording.mp4 -ss 00:00:02.000 -frames:v 1 inspected-moment.png
The FFmpeg selection filter supports scene scores; its documentation suggests roughly 0.3 to 0.5 as a useful starting range. A changed label or focus outline may produce little pixel change, while a decorative animation may produce many candidates. Review the frames against the video. Two frames per second can miss a brief tooltip or menu.
The supplied recording is 5.76 seconds long. In our local command check, fixed-rate sampling produced 12 candidates and scene selection produced two. The contract contains six meaningful states, so neither extraction result supplies the final inventory on its own.
Frame filenames are sequence numbers, not timestamps. Keep the source time beside each selected frame; the scene-selection command prints timing information through showinfo. Name the retained files by state, such as S2-detail-open.png, so the agent can connect an image to a requirement.
Encoded video dimensions also need interpretation. A cropped, scaled, or high-density recording does not establish the page's CSS viewport size. Ask for the original viewport and zoom when possible. Otherwise, record the video dimensions and mark CSS measurements as estimates. Check relative placement first: the drawer meets the right edge, the rail stays separate from the content, and the cards share a grid. Blurry text should become a transcription question, not invented UI copy.
When the recording is long, incomplete, or hard to read
Split a long recording by user task, such as filtering a queue or submitting a form. Finish one path before adding another. Include the screen immediately before the first action, each meaningful result, and a short note about what changed outside the crop. Keep shared labels, IDs, and layout rules in one file so separate coding requests do not produce conflicting versions of the same screen.
If the next screen appears without a visible action, mark the trigger as unknown. Ask for a slower recording or a written description when possible. Otherwise, choose a provisional interaction and label it as your decision. When text is unreadable, ask for its exact wording or use an explicitly named placeholder. Adding more nearly identical frames will not recover letters that the recording never captured clearly.
Keep a full-viewport image for layout and add a close crop only where small text or a control needs explanation. If your tool limits image attachments or context, send the frames for one path with their filenames and timeline, rather than compressing every screen until the labels become unreadable. State which images belong to the same interface and which show a different viewport.
Turn the selected frames into a state map
A state is a recognizable condition of the interface, such as a selected filter or an open drawer. A transition is the action that moves it to the next condition. The Lumen recording follows one successful path. These six states describe what a reader can reproduce in the example. They do not cover every state a real review application would need.
| State | Action that reaches it | Expected visible result | Decision requiring more than the frame |
|---|---|---|---|
S0 Initial | Load or reset | Four brief cards; Ready count is two | Where the records come from and whether changes persist |
S1 Ready filter | Activate Ready | LN-104 and LN-111 remain; Ready is selected | Whether selection changes the URL or browser history |
S2 Detail open | Open LN-104 | A detail drawer meets the right edge over a dimmed page | Whether it is modal, its initial focus, and background interaction |
S3 Activity open | Activate Show activity | Three events appear; the control becomes Hide activity | Whether expansion survives closing and reopening |
S4 Reviewed | Activate Mark reviewed | Status changes; Ready becomes one; the action is disabled; feedback appears | What happens if saving fails or the action is repeated |
S5 Closed | Close the drawer | LN-111 remains in the Ready queue | Where focus goes after LN-104's opener disappears |
Write each uncertainty as an explicit decision. For this local prototype, filters do not change the URL, updates stay in memory, closing resets the activity section, and the active filter receives focus when its reviewed card has disappeared. Those are implementation choices. A recording of the successful path cannot establish whether a real product wants the same behavior.
Use one state record per distinct outcome. Add loading and failed-save states when implementing a server-backed action. Avoid treating every animation frame as a new application state; the drawer can be opening while the selected record and the intended destination remain the same.
Write a contract the coding agent can implement
The state contract is a written list of those states, actions, and expected results. Separate what you observed from what you chose. "A dimmed page behind a right-edge surface" is visual evidence. "Modal dialog with contained keyboard focus" is a behavior requirement you must supply. In this example the desktop drawer is 430 CSS pixels wide at a 1280-pixel viewport; do not reuse that width merely because your recording also contains a drawer.
A useful requirement names its starting state, trigger, visible outcome, and a check. This example adds the focus decision missing from the recorded path:
{
"id": "S5-closed-after-review",
"precondition": "Ready filter; LN-104 marked reviewed; drawer open",
"trigger": "press Escape",
"observed": [
"the drawer disappears",
"LN-111 remains in the Ready queue"
],
"chosen_behavior": [
"Ready filter remains selected",
"focus moves to the Ready filter button",
"background controls become available"
],
"decision_origin": "chosen for this local prototype"
}
Prepare a small input directory with the recording, selected frames, state contract, exact text and sample records, and approved assets. If the agent accepts images, attach the selected frames explicitly or confirm it can open the files in the workspace. A path mentioned in a prompt is useful only when the agent can read that path. If your chosen tool accepts video, retain the frame names and timeline so a reviewer can still locate each requirement.
Use a brief like this after replacing the example's scope and filenames:
Build a local interactive frontend for the supplied recording.
Read reference/keyframes/, state-contract.json, and the supplied text/data.
Treat the frames as visual evidence. Treat chosen_behavior entries as
prototype requirements. List unresolved behavior before implementing it.
For this prototype:
- Use plain HTML, CSS, and JavaScript under rebuilt/.
- Implement S0 through S5 with the supplied records and exact labels.
- Derive list contents and counts from one data model using stable IDs.
- Make the detail drawer a named modal dialog. Contain keyboard focus,
support Escape, and restore focus using the contract's fallback.
- Match the observed 1280 x 720 layout. At 390 x 844, use the separately
specified full-width drawer and single-column queue.
- Honor reduced motion. Keep content and controls usable without animation.
- Keep updates in memory. Do not add an API, login, analytics, or persistence.
First return the file plan and open decisions. Then build the interaction
path and tests. After behavior passes, compare the named visual states.
Return startup commands, test commands, changed files, and remaining gaps.
For an existing React, Vue, or other application, replace the new-prototype instruction with the project's actual framework, entry point, components to reuse, and allowed files. Keep the same state requirements. Ask the agent to fit those requirements into the current architecture before introducing dependencies.
Use the brief with your own recording
- Create a separate project directory. Add your selected frames, a timeline, exact visible text, sample records, and the behavior specification. Use filenames in the timeline so every action points to the right image.
- Replace the Lumen names, six-state assumption, viewport sizes, and sample-data rules in the brief. Keep only the requirements your own interface needs. Supply missing decisions, such as whether closing saves edits, before the agent implements them.
- Attach the files or place them in the agent's readable workspace. Ask it to summarize the visible sequence and list unreadable text or unknown actions. Correct that summary before asking for code.
- After implementation, use the startup command the agent provides and replay the path in a browser. For each step, compare the visible result with the matching frame and the expected behavior with your specification. Give corrections by state name.
- Ask for tests that use your labels, records, and transitions. The kit's
tests/rebuild.test.mjsexpects Lumen's IDs, names, counts, and screenshots; it will not validate an unrelated interface unchanged. Keep your reference frames, regenerate the new app's captures, and inspect the comparisons before accepting them.
For an existing application, also ask the agent to explain how it will reuse the current components, routes, and data calls before it edits them. Review the changed files after each interaction works. If the local page fails to start, give the agent the actual command and error output; another screenshot of the target interface will not explain a missing dependency or build error.
Build the state changes before tuning the styling

The comparison mock has the palette, typography, and dimmed background, but its modal is centered instead of attached to the right edge. It has no activity history or working review transition. We constructed it deliberately as a review exercise. It is not an unedited model response.
Start the rebuild with real buttons, a selected filter, record IDs, and a selected-record state. Complete the path from opening an item through reviewing and closing it. Keep the selected record's ID separate from its position in the filtered array; after a status change, that array can shrink or reorder.
Counts and cards should read the same underlying data. For the Lumen example, the two Ready records become one after LN-104 is reviewed:
const readyItems = briefs.filter(brief => brief.state === 'ready');
const visibleItems = activeFilter === 'all'
? briefs
: briefs.filter(brief => brief.state === activeFilter);
readyCount.textContent = String(readyItems.length);
Do not update only the badge text and leave the record unchanged. Check the status inside the drawer, the count, and the filtered cards after the action. If a rerender replaces DOM nodes, find the new card by its stable ID before returning focus. Keeping a reference to the original button does not help once that node is detached.
The sample records are fixed fictional content. When adapting the prototype to external text, create text nodes or use framework escaping instead of inserting untrusted strings through innerHTML. A visual reconstruction is not a reason to bypass the existing application's data-handling rules.
Handle the modal as a keyboard interaction
The W3C modal dialog pattern describes focus moving into the dialog, Tab and Shift+Tab remaining inside it, Escape closing it, and a suitable focus destination afterward. An aria-modal="true" attribute alone does not implement those behaviors.
In kit v1.1, opening the drawer prevents the background workspace from receiving clicks or keyboard focus, using the HTML inert property. It then focuses Close brief. Tab wraps through the enabled drawer controls. Reviewing a record disables the review button and moves focus to Close brief if that button held focus. Closing looks for the current card with the original ID; if the card has left the filter, it focuses the active filter button. The browser checks test these destinations explicitly.
Test both closing before an update and closing after it. Also review the last Ready item: the queue should show its empty message, and focus should still land on a usable control. These cases expose mistakes that the initial open-drawer screenshot cannot reveal.
Specify layouts and motion the recording did not show
At the recorded desktop size, compare the page's major regions before adjusting decoration: navigation width, content bounds, card columns, drawer position, and drawer width. Then check text wrapping and vertical spacing. A wrong font changes line lengths and card heights, so confirm the font or an approved substitute before compensating with arbitrary widths.
For the example's narrow layout, the navigation simplifies, summary cards stack, the queue becomes one column, and the drawer fills the viewport width. This is a designed adaptation. A desktop clip supplies no evidence that its original product uses the same mobile layout.
Set separate acceptance checks for long titles, several activity entries, increased text size, and a short viewport. An action pinned to the bottom of a drawer can overlap long content; choose either a scrolling content region with a separate action area or an action that follows the content. Check that the user can reach the action without losing the relevant fields. The example's mobile check with its original text does not establish that every long-content case works.
For motion, record the property that changes: translation, opacity, height, or a combination. Estimate duration from the interval between the visible start and settled state. A video's frame rate does not establish animation duration, and a few intermediate frames do not establish an exact easing curve. Begin with an approximation, compare the same transition, and tune one property at a time.
When reduced motion is requested, remove nonessential travel while preserving the final state and usable controls. The W3C reduced-motion technique documents using the preference query. The kit includes a reduced-motion path; its screenshots use that setting for stable comparison. Matching live transition timing is a separate review.
Add the missing branches for your kind of interface
The following branches are requirements to decide and verify when adapting the workflow. They are not additional capabilities demonstrated by the local Lumen example.
| Situation | Decision to give the agent | Concrete check |
|---|---|---|
| A save uses an API | Choose when the UI updates, how pending status appears, and whether a failed request rolls back. | Simulate a failed response. Confirm that the UI does not report success and that retry uses the intended record. |
| Repeated clicks or slow requests | Choose how to prevent duplicate actions and what happens if the user closes the drawer while saving. | Activate the action twice under delay. Inspect requests and the final count; one visible click path should not silently create duplicate work. |
| A form validates input | Specify field rules, error text, error placement, and what happens to existing values. | Submit empty and invalid values, correct them, and submit again. Check keyboard focus and server-side rejection separately. |
| Filtering, sorting, or pagination changes the list | Define stable item IDs, selection after updates, empty results, and whether filters affect the URL. | Update or remove the selected record. Close its detail view and use Back if routing is part of the contract. |
| Hover reveals a menu or tooltip | Provide a keyboard trigger, dismissal rule, and touch behavior. | Reach the control without a pointer; open and dismiss the extra content without losing access to it. |
| Records or permissions change elsewhere | Define what happens when an open record is stale, deleted, or no longer editable. | Return that condition from the test data layer. Confirm that the drawer explains the outcome and blocks an invalid update. |
| Drag, scroll, canvas, or chart interaction | Capture start and end positions, coordinate space, scroll container, and an alternative control where needed. | Replay the action at the target size and after scrolling. A matching static chart does not establish working hit targets. |
| Long content or another language | Supply representative strings, wrapping rules, number/date formats, and any right-to-left requirement. | Use long labels and the target language in the actual layout. Check clipping, action visibility, and reading order. |
Choose how a real save behaves
A recording that shows a successful save does not reveal when the server accepted it. One implementation can wait for confirmation before changing the status and count. Another can update the screen immediately and restore the previous values if the request fails. Specify which approach you want; otherwise the agent may build a success message that appears before anything has been saved.
Test a rejected save as a complete sequence. Begin with two Ready items, open LN-104, attempt the update, and return a failure from the test data layer. Under a wait-for-confirmation design, the count should remain two and the item should remain Ready. Under an immediate-update design, the count and status must return to those values after failure. Keep enough context for a retry, and check that retry updates the same ID. This is a proposed test for a server-backed adaptation; the downloadable example has no save request.
Also decide what closing during a pending request does. If the request continues, the result needs a visible destination after the drawer closes. If users can submit again, define how the service prevents duplicate work. Disabling a button is only a frontend behavior; it does not establish what the server does with two requests.
For destructive actions, add the product's confirmation or recovery behavior before implementing them. For authentication or a real backend, keep the reconstruction local until the actual integration requirements are available. If form behavior is your next task, the web form repair guide develops labels, validation feedback, and focus checks in more detail.
Verify behavior and visual similarity separately
On October 15, 2026, kit v1.1 passed 40 automated browser checks in installed Google Chrome. The run covered the six-state path, derived counts, a named dialog, initial focus, forward and reverse Tab wrapping, focus after a disabled action, both focus-return cases, an empty Ready queue, data reset on reload, reduced-motion preference, two viewport sizes, visual comparisons, and browser errors.
| Check | Result in this local run | What remains outside it |
|---|---|---|
| Automated browser checks | 40 of 40 passed; no console or page errors | Real APIs, authentication, server failures, and production integration |
| Layout | No horizontal overflow at 1280 × 720 or simulated 390 × 844; mobile drawer was 390 pixels wide | Physical devices, other engines, zoom, and arbitrary content lengths |
| Four stable reference states | About 0.18% to 0.62% of pixels differed | Animation timing, semantic correctness, and usability |
| Deliberate geometry error | Moving the drawer 32 pixels left produced a 6.59% difference and exceeded the 3% maximum | Detection of every meaningful visual defect |
| Offline Python gate | Saved evidence and expected package files were consistent | A fresh browser execution or approval to release |
The screenshot comparison uses a 0.12 per-pixel threshold and permits at most a 0.03 differing-pixel ratio. Those numbers control different things. The first affects whether two individual pixels count as different; the second limits how much of the whole image may differ. The 3% limit was checked against one deliberately introduced geometry error. It is not a universal quality threshold.
Keep browser, operating environment, viewport, fonts, data, scroll position, and motion setting steady when comparing frames. Wait for the UI's settled state and fonts before capture. Playwright's visual comparison guide explains baseline handling and environmental differences. A screenshot recorded on another platform may need investigation before it can serve as a reliable baseline.
When a comparison fails, inspect the actual and diff images before accepting a new baseline. A small ratio can hide a missing label or wrong status in a small region. Check those labels and values directly in the browser tests. A large ratio can come from a font or capture-size mismatch, so compare dimensions and environment before rewriting the interface.
Fix failures by symptom
| What you see | Inspect first | Useful next change |
|---|---|---|
| The interface looks plausible but buttons do nothing | State changes and event handlers; look for decorative controls with no action | Implement one contract transition and assert its changed content before further styling. |
| The count changes but the wrong card disappears | Array-index selection, duplicate IDs, and separate count/list data | Keep selection by stable ID and derive both count and cards from the updated records. |
| Focus disappears after closing | Whether the opener was removed or replaced during rendering | Find its current DOM replacement by ID, then use the documented filter or next-item fallback. |
| A modal looks dimmed but Tab reaches the page behind it | Background inertness and the first/last enabled controls | Implement modal behavior and test both Tab directions, including after disabling an action. |
| Text and cards differ throughout the screenshot | Font availability, viewport dimensions, zoom, and exact copy | Correct those inputs before changing individual margins. |
| A mobile action is off-screen or covered | Fixed heights, fixed positioning, long content, and the scroll container | Define which region scrolls and make the action reachable at the chosen small viewport. |
npm test cannot connect | Server terminal, directory, URL, and port | Confirm /rebuilt/ opens in a browser; match the server port through LUMEN_BASE_URL. |
| The test cannot launch Chrome | Installed browser and executable path | Install Chrome or set CHROME_PATH to its executable; do not treat a launch failure as an interface result. |
| The Python gate passes after you changed code | Result timestamps and whether a new browser run completed | Run npm test successfully before inspecting saved results. The Python gate alone cannot detect a stale passing report. |
For a visual correction, give the agent one state, the reference frame, the current capture, and the specific difference. For example: "In S2 at 1280 × 720, attach the drawer to the right viewport edge and preserve the filter and focus behavior. Change only drawer geometry, then rerun the transition tests." A bounded request makes it easier to see whether the correction broke something that already worked.
Prepare the reconstruction for a real project
Keep the recording, named frames, chosen behavior, local data, startup commands, and latest test results with the implementation. Record which mobile and failure states were designed separately so the next developer can distinguish observed behavior from decisions made during reconstruction.
The example still needs screen-reader testing and a production release review. Its automated keyboard checks do not establish screen-reader behavior or accessibility conformance. Before integrating a real service, add the actual data contract and failure tests; before releasing, check the browsers, devices, content lengths, languages, zoom levels, and security requirements your product supports. Include the remaining decisions, who will resolve them, and the commands needed to run the current version.
