We trained a reward model on the components developers choose on 21st and used it to train Qwen3.8-27B to generate UI aligned with developers' preferences.
What we found
- We trained a reward model: a model that scores a screenshot of a UI component by what developers choose on 21st. In 65% of pairs it had not seen, it gives the higher score to the component developers preferred. A coin gives 50%.
- Qwen, trained to get a higher score from that reward model, builds a product around the component. Asked for one hero or one button, the trained Qwen adds a site navigation bar, a logo with the brand name or stat cards. After training 16% of its components have such additions, up from 1%.
- The trained Qwen fills the screen and adds colour. Its components cover 37% of the screenshot, up from 23%, and 65% of them have a coloured glow in the background, up from 20%.
1 The reward model
We want a model that looks at a component and says how likely developers are to choose it over another component made for the same need.
The reward model is Qwen3.8-27B with a small trained adapter (LoRA) and one output number. It gets the search text and one screenshot, and returns a score.
1.1 Data
On 21st a developer searches or opens a category page, sees a grid of component previews and copies the one they want into their project. Many developers pay to copy, so a copy is a deliberate choice. We turned these copies into two datasets.
Clean pairs
A pair is clean when both components were shown for the same search many times and one of them, the winner, has a clearly higher ratio of copies to expected copies than the other, the loser. Expected copies are what an average component gets at the same places in the results. The winner's ratio is at least twice the loser's, and the difference is too large to be chance. A pair stays only if the two components are interchangeable: a developer who needs a pricing table with three plans could use either one. A model removed the pairs that are not interchangeable, for example a login form vs a full login page with a photo. Then we looked at every remaining pair ourselves and removed the ones the model had missed.
Noisy comparisons
1,386 clean pairs are too few to train a 27B model on, so we also built a much larger and rougher set. When a developer opens several components from one search and copies one, the copied one wins a comparison against each of the others. In the example below, 5 opened and 1 copied give 4 comparisons.
One developer's choice can be chance, so these comparisons are noisy. They cover five times as many components as the clean pairs, so the model learns from them first.
1.2 Training
Training settings
| Adapter | LoRA, rank 16 |
| Noisy comparisons | 1,210 steps, learning rate 1e-4, about 46 minutes |
| Clean pairs | 87 steps, learning rate 2e-5, about 4 minutes |
The model learns one rule: in every pair, the winner must get a higher score than the loser. It is trained first on the 19,355 noisy comparisons and then on the 1,386 clean pairs. The diagram shows one training example from start to finish.
turns the score difference into a number between 0 and 1. The loss is small when the winner scores higher (0.40 here) and large when it scores lower (1.11 if these two scores were swapped).
The loss is the standard pairwise (Bradley–Terry) loss for reward models. We trained with ms-swift, one pass over each dataset.
Test runs. Before training the final model, we checked that the method works. We trained five test models. Each one never saw a different fifth of the clean pairs: their components were removed from both the clean pairs and the noisy comparisons. Then each test model picked the winner in the pairs it had never seen. Guessing gives 50%. These five models are used only for this measurement and are then thrown away.
Final model. The final reward model is trained the same way on all the data, so there are no unseen pairs left to test it on; this is its training loss.
2 The generator
Qwen already generates React components that build and render. Nothing in its training told it which components developers choose. The goal is a model that generates components developers would choose more often. Developers cannot rate every component during training, so the reward model stands in for them.
2.1 Training setup
We train the generator with GRPO: at each step it generates several components for the same brief, each gets a reward, and the model is pushed towards the ones that scored above the group's average. The model we train to generate UI is Qwen3.8-27B, the same model the reward model is built on, with a separate adapter. Qwen gets a brief and generates one React component. It trained on 480 briefs. We compare it before and after training on 30 other briefs it never saw.
System prompt for the generator (Qwen)
You are an expert frontend engineer. Write ONE self-contained React component (TSX). Rules: - Style with Tailwind CSS utility classes. - Allowed imports only: 'react', 'framer-motion', 'lucide-react'. No other packages. - No external images, fonts or network requests; draw visuals with CSS, SVG or gradients. - Use realistic placeholder text, not lorem ipsum. - export default the component; it takes no props. - It is rendered alone on a page 1080px wide, so it must look complete in that viewport. - Use exactly the theme (light or dark) the request asks for. Output only the code, in a single ```tsx block.
Example brief for the generator (Qwen)
Build a single content card whose main feature is a glowing light effect, such as a glowing border or a glow following the cursor, not a grid of cards and not a landing page. It is for "Quantum Notes", a note-taking app. Required: exactly one card with an image area, a title, a short description and a button, with the glow clearly visible and animating. Visual style is up to you. Show only this component on the page: no headline, hero, navigation or other sections around it. Use a light theme.
At each step it generates 8 components for a brief. Every component is built, opened in Chromium and screenshotted, and gets a reward:
| What happened | Reward |
|---|---|
| The code did not build, crashed or showed a blank page | −3 |
| Claude Haiku 5.5 says the screenshot breaks the brief | −2 |
| Otherwise | The reward model's score |
Brief check prompt for Claude Haiku 5.5
You check whether a screenshot of a rendered UI component fulfills a design brief. You do NOT judge visual quality or style at all.
Answer with JSON only: {"component_ok": bool, "theme_ok": bool, "elements_ok": bool, "scope_ok": bool, "reason": "<max 15 words>"}
- component_ok: the screenshot shows the kind of component the brief asks for (same job, subtype and rough item count; small deviations in counts are fine).
- theme_ok: the overall page is predominantly light when the brief says light theme, predominantly dark when it says dark theme.
- elements_ok: false only if MOST of the required elements are missing or the screen is an error/blank/placeholder.
- scope_ok: if the brief says "Show only this component", false when the screenshot adds a substantial invented page around it (a hero headline section, a navbar, an extra editor or window, a full landing page). Small captions are fine. If the brief does not say this, true.
Be lenient on details and strict only on clear violations.
Brief:
{brief}
[screenshot]2.2 Training
Training settings
| Adapter | LoRA, rank 32 |
| Learning rate | 1e-4 |
| Temperature | 1.1 |
| Components per brief | 8 |
Two charts show how training went: how many generated components failed at each step, and how the reward model scored the components for 30 unseen briefs.
Qwen reasons before it generates code, and the reasoning and the code share one limit of 12,000 tokens. From step 4 the reasoning grew from about 2,100 characters to 14,900, and the code no longer fit. A cut-off answer gets −3, so training pushed the reasoning back down. The second dip is a different failure: the answers end without a finished code block and without reaching the limit.
On the 30 unseen briefs the reward model's score went from 0.32 before training to 0.66 after it. At steps 10 and 20 many components failed, so those two points average only the ones that rendered.
2.3 Results
For each of the 30 test briefs, we compare 4 components generated before training with 4 generated after.
The gain does not come from fewer broken components. Everything we measured on the same components:
| Before training | After training | |
|---|---|---|
| Components that rendered | 93% | 94% |
| Added extra sections the brief did not ask for | 1 of 112 | 18 of 113 |
| Required elements missing | 6 of 112 | 0 of 113 |
| Share of the screenshot covered | 23% | 37% |
| Length of the answer | 3,818 tokens | 3,791 tokens |
Below are the 30 briefs in three groups, by how the reward model score changed.
Reward model score went up (21 briefs)
Build a single animated wave effect that fills the viewport, not a static divider and not a chart. It is for "Tidal", a surf forecast app. Required: exactly one continuous wave animation made of layered flowing lines, shapes or a shader, looping smoothly. Visual style is up to you. Show only this component on the page: no headline, hero, navigation or other sections around it. Use a light theme.
After training, 3 of the 4 components add extra sections the brief did not ask for: the Tidal logo in a corner and a card or bar with the day's wave height, wind and water temperature on top of the waves. None of the 4 components generated before training have them.
3 more briefs where the score went up
Time picker, 0.38 → 1.28, is the figure at the top of the post.
Build a single looping decorative animation built from shapes, not a page with animated sections and not a collection of different effects. It is for "Nimbus Weather", a hyperlocal forecast app. Required: one self-contained animated composition of at least 5 moving elements, a seamless infinite loop, and smooth easing. Visual style is up to you. Show only this component on the page: no headline, hero, navigation or other sections around it. Use a light theme.
Build a single hero section whose main feature is an image gallery, not a full landing page and not a standalone lightbox. It is for "Casa Alba", a boutique hotel in Lisbon. Required: a headline, a short subtext, 1 call-to-action button, and exactly 5 gallery images (use placeholder image URLs). Visual style is up to you. Use a light theme.
Build a single hero section with an animated aurora-borealis style background, not a full landing page and not a bare background without content. It is for "Northlight Analytics", a product analytics platform. Required: the aurora-style animated background, exactly 1 headline, 1 supporting paragraph and 2 call-to-action buttons. Visual style is up to you. Use a dark theme.
Reward model score barely changed (5 briefs)
Build a single glassmorphism card with a frosted translucent surface and background blur, not a grid of cards and not a full page. It is for "Lumen Bank", a mobile banking app. Required: one card containing a small icon, a title, a short description line, a balance figure and one action button, placed over a visible backdrop so the frosted blur reads clearly. Visual style is up to you. Show only this component on the page: no headline, hero, navigation or other sections around it. Use a dark theme.
2 more briefs where the score barely changed
Build a single hero section for a gaming venue, not a full multi-page website and not a game library grid. It is for "Pixel Pit", an arcade bar with retro cabinets. Required: 1 headline, 1 supporting paragraph, exactly 2 call-to-action buttons, exactly 3 small stat items (such as machines, tournaments, opening hours), and one gaming-themed visual element. Visual style is up to you. Use a light theme.
Build a single gradient call-to-action button, not a button group and not a form. It is for "Orbit Pay", a payments app for freelancers. Required: exactly one button with a text label and a small arrow icon, and a vivid multi-stop gradient fill that is clearly visible in its resting state. Visual style is up to you. Show only this component on the page: no headline, hero, navigation or other sections around it. Use a dark theme.
Reward model score went down (4 briefs)
Build a single chat interface where a user talks to an AI coding assistant and replies include code, not a landing page and not a plain messaging app. It is for "Syntaxly", an AI pair-programming tool. Required: exactly 4 messages alternating user and assistant, at least 1 syntax-highlighted code block with a copy button, and a message input bar with a send button. Visual style is up to you. Use a light theme.
All 4 components generated after training put the chat inside an app with a sidebar of past chats and a user profile, and one adds a file tree. None of the 3 components that rendered before training have a sidebar.
2 more briefs where the score went down
Build a single date range picker with an open calendar popover, not a single-date picker and not a full booking page. It is for "Stayloop", a vacation rental booking site. Required: a trigger field showing the selected range, a calendar showing exactly 2 months side by side with a highlighted start, end and in-between days, 4 preset shortcuts, and Cancel and Apply buttons. Visual style is up to you. Show only this component on the page: no headline, hero, navigation or other sections around it. Use a dark theme.
After training, one component shows a blank screen and in another the calendar dates overlap. Before training, one component shows a large yellow block instead of the calendar.
Build a single full-viewport decorative background pattern, not a gallery of pattern samples and not a page with content. It is for "Tessellate", a graphic design studio. Required: exactly 1 seamless repeating pattern built from one repeating motif that fills the whole viewport. Visual style is up to you. Show only this component on the page: no headline, hero, navigation or other sections around it. Use a dark theme.
3 Next steps
- Show Qwen's components to developers on 21st. We put components generated before and after training next to the ones made by people and count which ones developers choose. That is the test of whether training made the components better.
- Feed those choices back into the reward model. So far the reward model has learned only from components made by people. Choices between generated components are the kind of data it is later asked to score.
- More data for the reward model. It gives the higher score to the component developers preferred in 65% of pairs. We collect comparisons over a longer period and reduce them to one answer per pair.
- Show the reward model more than a screenshot. It sees one static image. A developer also sees the animation, the hover states and the code.