About this lab + first run
What is this lab?
This comparison page was added by A Rough Idea for classroom experiments. Chuanqi Sun’s original Spaceship uses decision models within the flight
deck; it did not include this separate lab.
A decision model judges supplied text against a question and returns a structured score or probability. Here you can inspect those judgments
directly, before choosing a model for Spaceship’s relevance, influence, and realization controls.
Try a comparison
- Sign in with an invited email. Live evaluations use the shared demo allowance set by the owner.
- Start with the sample passage and suspense question below. The context is the text to judge; the question says what to judge.
- Choose a model and select evaluate. Keep the context, question, and rubric unchanged, switch models, and evaluate again.
-
Compare the judgments, response time, and reported cost. Then change one phrase in the passage or question and see how sensitive the results
are.
Model agreement is not proof of correctness. Treat these judgments as material for discussion, especially for subjective qualities such as
suspense.
Choosing an answer type and reading results
Rubric score: write one level per line, ordered from lowest to highest. Levels are numbered from 0. The sample’s four levels
therefore run from 0 (no suspense) to 3 (strong suspense). A fractional score falls between levels; it is not a percentage.
Yes / no probability: ask a yes/no question, such as “Does this passage create suspense?” The result field noul runs
from 0 (no) to 1 (yes). For example, 0.8 expresses the model’s estimated probability of yes, not a guarantee that its judgment is correct. The
rubric is not used in this mode.
Results include the submitted context and question so you can see what changed. In the structured answer, probabilities, when
supplied, gives the model’s distribution over rubric levels. confidence is a provider-reported measure, not independently measured
accuracy, and may differ in meaning across models.
Response time includes the server’s provider round trip. Cost is the provider-reported charge for that run; a pending cost means it was not
returned. Model-list prices are per million input tokens, not per evaluation.
Availability does not guarantee scoring compatibility. Failed requests are shown explicitly.