# Evaluating frontend design experiments

The frontend cohort studies default design preferences and usable interfaces across three tasks: an editorial magazine, event discovery, and a team inbox. Each model receives two independent vanilla attempts per brief. An additional Forma frontend condition reuses the original activity facts and photographs without supplied fonts, and uses the same frontend-only flows. This document is a review companion, not part of the generation prompt. The original full-stack Forma experiment keeps its original rubric and backend checks.

## Three separate perspectives

### 1. Descriptive taste profile

Record choices before judging them. Tags are observations, not points; use multiple tags or `not observed` where appropriate. Add one concrete example for each meaningful observation.

| Aspect     | Useful observations                                                                                                     |
| ---------- | ----------------------------------------------------------------------------------------------------------------------- |
| Typography | Serif/sans/monospace; family choices; display scale; contrast between headings and body; editorial or compact treatment |
| Palette    | Light/dark/mixed surfaces; muted/saturated colors; warm/cool relationships; number and purpose of accents               |
| Density    | Spacious/compact/mixed; information visible without scrolling; reading line length                                      |
| Structure  | Symmetry/asymmetry; columns; navigation placement; repeated rows/cards; relationship between browsing and detail        |
| Imagery    | None/photos/illustration; chosen subjects and sources; crop and proportion; image as information or atmosphere          |
| Ornament   | Borders, shadows, texture, gradients, corner treatment, decorative marks; consistency across states                     |
| Motion     | None/subtle/prominent; which change it explains; hover, transition, and reduced-motion behavior                         |

Record chosen asset sources and whether the model used fallback assets, downloaded assets, system fonts, or code-drawn graphics. Observe the full page and an interaction state, not just the hero. A repeated choice across briefs and samples is evidence of a default preference; it does not identify specific training data or separate pretraining from later tuning and agent instructions.

### 2. Human design quality

Judge how well the choices serve the specific interface. Distinctive design can be useful or distracting; familiar design can be appropriate or unconsidered.

| Dimension                    | Weight | Evidence to inspect                                                                                                  |
| ---------------------------- | -----: | -------------------------------------------------------------------------------------------------------------------- |
| Identity and relevance       |    20% | Coherent choices suited to the product and its content; a recognizable identity without obstructing the task         |
| Hierarchy and comprehension  |    20% | Clear orientation, understandable controls, scannable facts, and an evident next action                              |
| Composition and typography   |    20% | Alignment, spacing, line lengths, type relationships, density, and image treatment across the entire interface       |
| Interaction and feedback     |    20% | Successful core flow, visible state changes, reversible actions, recoverable empty states, and a usable detail view  |
| Responsive and inclusive use |    15% | Deliberate small-screen adaptation, keyboard access, focus, labels, readable text, and appropriate motion            |
| Craft and restraint          |     5% | Consistent details; no accidental clipping, dead controls, unresolved overlaps, or decoration that competes with use |

Use the same 1–5 anchors as Forma: **1 broken**, **2 weak**, **3 competent**, **4 strong**, **5 exceptional**. A 1 prevents understanding/completion or has pervasive incoherence; 2 has recurring substantial problems; 3 is usable with ordinary execution; 4 is deliberate and polished with minor issues; 5 is exceptionally resolved across the inspected states. Record evidence with every score. Use **not assessed** when evidence is missing. Do not fill in missing ratings automatically.

A weighted score is optional and requires all dimensions: `sum(weight × score / 5)` with weights totaling 100. Keep the dimension profile visible; do not replace it with an overall leaderboard. Quick preference ratings are separate from this rubric.

### 3. Automated frontend usability

Checks should verify observable behavior without rating aesthetic preferences:

- Rendering, visible fixture content, JavaScript errors, horizontal overflow, and local resource loading at desktop 1440 × 1000 and mobile 390 × 844.
- Title search, category/status filtering, combined filter behavior, empty results, and recovery.
- Opening and leaving details; complete content; usable keyboard controls; Escape and returned focus for modal dialogs.
- Magazine/event saving and unsaving, a saved-only view, and state retained while navigating within the page session.
- Task status changes, visible feedback, and updated status filtering.
- Automated accessibility findings on the inspected states, with their scope recorded.

No backend, persistence across server restarts, booking, pricing, or API checks apply. Browser reload persistence is optional. A check count diagnoses the tested flows; it is not a design score or a complete accessibility audit. Report harness/provider failures separately from generated-interface defects. If a review-only repair is needed, retain the original and label the repaired preview.

## Review procedure and interpretation

1. Compare the same brief version, sample, viewport, and interaction state. Review the raw output before reading automated results.
2. Hide model names and randomize left/right order where possible. Record side-by-side preference with **tie** and **insufficient evidence** available.
3. Browse the full desktop interface, complete its core flow, and repeat on mobile with keyboard checks. Capture a representative detail and changed-state view.
4. Record descriptive tags, dimension evidence/scores, preference, and confidence separately. Then inspect automated failures for additional evidence.
5. Compare the two samples to distinguish a recurring choice from a one-off result. Compare briefs to see whether preferences adapt to context. Two samples are initial repeatability evidence, not enough to declare statistical winners.

The briefs fix product purposes, fictional content, a plain frontend stack, and a few native controls/test hooks. Those constraints still shape outputs. For the three new product briefs, every model has the same optional seven-family/eight-image fallback pool. Forma frontend instead supplies only the six original photos, with no bundled fonts. Every frontend condition permits downloading additional usable assets during generation. **Equal asset access, not identical asset selection, is intentional**: selection is part of the taste being observed. Additional assets must be stored locally with a source/license record; remote runtime dependencies are excluded for reproducibility. Network availability and search/tool limitations can still affect what a model finds.

Keep provider model mapping, reasoning level, time/turn limits, tool access, input hashes, selected assets, and failures alongside results. Original full-stack Forma outcomes are a different task condition and should not be pooled with frontend results as if their scores were interchangeable. Forma frontend is also a separate asset/content condition from the other three briefs; shared subjects allow qualitative comparison, not a causal claim that removing the backend alone produced a change.

## Compact review record

```text
Brief / version / sample / run ID:
Reviewer / date / inspected states:
Taste observations and evidence:
Asset choices and provenance:
Six dimension scores or not assessed, each with evidence:
Strongest decision / most consequential weakness:
Pairwise preference: A / B / tie / insufficient evidence
Confidence and reason:
Automated failures and suspected source:
Repeated preferences across samples or briefs:
```

## Lessons from the first frontend iterations

A persistent search or category choice is not automatically a usability defect. An early evaluator reloaded the page between scenarios but left session storage intact, so one scenario contaminated the next. Final captures clear browser state between independent checks while keeping state during each complete user flow. Earlier reports are retained separately.

Counting rows can conceal incorrect results. The revised evaluator compares visible fixture identities, combines search with filtering, checks the full article/detail body, and waits for initial fixture rendering before taking screenshots. It also retains manifest-verified input data so later brief revisions cannot silently change the meaning of an older run's checks.

A missing test hook is an integration-contract failure, not proof that the corresponding visible control cannot work. Inspect the artifact before translating a selector failure into a human interaction score. Conversely, a polished opening screen cannot demonstrate that saving, removing, leaving a detail view, or updating status works. Review those transitions directly.

Visible text includes CSS casing changes; matching fixture facts should tolerate uppercase styling while still requiring complete content. Removing the last saved item may legitimately close its detail view automatically. Evaluator version 4 handles both cases; the [native recovery audit](native-recovery.md) records affected runs and archived evaluations. Corrections to test assumptions must preserve generated artifacts and genuine interface failures.

Dialog visibility and focus restoration need not settle in the same browser task. Version 5 waits briefly for stable focus on the opener, resolving it again if the interface re-renders. A bounded wait prevents timing races from masquerading as defects while still failing persistent focus loss.