Skip to Content
Product GuideVerify

Verify

Verify is where you check Morse’s judge on inferred outcomes, one random spot-check at a time. Morse draws the items for you — you cannot pick or reorder them — and the judge’s answer stays hidden until after you submit, so it cannot sway your answer.

What you see

Queue

One item at a time, in the order Morse drew them for the agent. A condensed trace sits beside the question and rubric (read-only). Keyboard shortcuts: Y / N / U (unsure).

Progress

Per agent: Runs checked toward 30, Agreed “no” toward 8, and Agreement (at least) toward 70%, plus Waiting, how many checks in each group were skipped and how many older checks were unblinded. The copy says what is needed for evaluation — not a promise that the agent will be promoted.

The status line states the result in words, followed by the figures:

  • Your checks agree with Morse at least 70% of the time — the check is met.
  • Not yet shown: your checks agree with Morse at least 70% of the time — not met yet.
  • Not enough checks yet.

The figures read, for example, “Agrees 88%, at least 76% of the time, needs 70%”.

How we know. Agreement (at least) is the low end of a 95% confidence range (the Wilson score lower bound): your checks agree with Morse at least this often, so a lucky streak can’t pass the check. It doesn’t say the judge is calibrated. Groups are the judge’s answers sorted into likely yes, likely no and borderline; a skipped check is one you skipped or marked unsure (nonresponse). See the glossary.

Blind labeling

Before you submit, Morse never sends the judge’s label, probability or rationale to your screen. After you submit, Verify shows them with whether you agreed, and adds disagreements to the agent list.

Disagreements

Items where your label disagreed with the judge, including any note you left.

Editing the question or rubric asks for confirmation when existing checks would stop counting toward evaluation. Without confirmation the edit is refused.

How to use it

  • Open Verify from the nav (badge shows open items) or from an agent page link.
  • Filter to one agent with ?agent= when you want a single queue.
  • Prefer finishing one agent’s batch before switching agents — evaluation runs when a batch is complete.
  • Opt in to a daily digest under notification preferences when you want a reminder of waiting items.

Before you have data

Verify stays empty until Morse has drawn runs to check for a confirmed question. Confirm the agent’s questions and let finished runs accumulate first.

Last updated on
Score SettingsOverview