Multi-model Compare

• A playground or eval tool where someone sends one prompt to several models and decides which reply is better.
• Choosing a default model: speed, length and cost side by side, measured on the same prompt.
• Collecting preference data: the pick is a callback, ready to store as a label.

• One model's reply in a chat: use Conversation with Response.
• Comparing runs that call tools step by step: use Agent Run Console.
• Comparing aggregate scores across many prompts: a table of eval results says more than two replies.

import { MultiModelCompare, type CompareModel } from "@/components/eased/multi-model-compare/multi-model-compare";export function Playground({ prompt, models, save }: { prompt: string; models: CompareModel[]; save: (id: string | null) => void }) {  return <MultiModelCompare className="h-dvh" prompt={prompt} inputTokens={182} models={models} onWinnerChange={save} />;}// One model. Append each chunk as it lands, stamped with ms since the request was sent.const kestrel: CompareModel = {  id: "kestrel-2",  name: "Kestrel 2",  provider: "Fieldline",  chunks: [    { at: 1120, text: "Check these ", tokens: 3 },    { at: 1161, text: "in order:", tokens: 2 },  ],  pricing: { inputPerMTok: 2.5, outputPerMTok: 10 },};// When the stream closes: { ...kestrel, endedAt: 2840 }.

Every number comes from the chunks: the first chunk's at is the time to first token, tokens per second counts from it to the latest chunk, and cost adds the prompt's inputTokens to the output at each model's pricing. A model is waiting until its first chunk, streaming after it, done once it has an endedAt and failed with an error; set status: "stopped" after Stop.

MultiModelCompare

CompareModel

compareStatus(model) returns the derived status, for your own controls.

• Each card is a section named by its model; its numbers are a description list, so a cell reads as "Tokens/s, 70". An empty number reads as "None yet".
• Prefer is a toggle button (aria-pressed) whose name includes the model: "Prefer Kestrel 2". It stays in place but inert while the replies stream, so focus never lands on it early.
• One live region speaks for the screen: "All 2 replies finished. Pick the one you prefer.", then "Kestrel 2 preferred". The replies themselves are not live, so streaming text is never read out word by word.
• The chart is a slider with the crosshair's value as its text ("At 2.75s: Atlas 3 Pro 41, Kestrel 2 114") and every point in a hidden table. Status is text as well as colour.

On mount the prompt, the cards and the chart rise 4px in turn (stagger.line, once). While the replies stream each new word sharpens out of a soft blur, the numbers roll on the snappy spring, the status line swaps with y + blur, and the chart grows a step at a time, its lines and scale morphing on smooth. Scrubbing glides one crosshair across every line and rewinds each reply to the same moment, with no animation on the text. Picking a reply lifts its card 2px on the snappy spring while a raised shadow fades in behind it and its border firms; the other cards recede to 60% over 200ms. Reduced motion: fades only. Nothing rises, slides or lifts, words appear without the blur, and the pick is told by the border, shadow and fade alone.

Pick this when one prompt goes to two or three models and a person decides between the replies. Feed it the raw stream: append each chunk with its at (ms since the request) and tokens, set endedAt when the stream closes, and pass pricing per model; the block derives time to first token, speed, totals, cost and the chart from those. Store the pick from onWinnerChange.

Do not compute the stats yourself or add another chart, timer or legend beside it: the cards are the legend, and the chart is already scrubbable. Do not wrap the cards in your own Card, and keep it to three models; past that, compare scores in a table.

Two models share a row from 40rem and three from 56rem; narrower, the cards scroll sideways with snap points, the next one peeking in, and a new pick scrolls its card into view. The hint above the cards moves under the prompt. A card's four numbers sit in a row from 26rem and two by two below it. The chart spans the full width under the cards. When the block is shorter than its content, it scrolls as a page. Nothing scrolls sideways at 390px except the row of cards.