Skip to Content
Getting StartedConceptsOutcomes that fill themselves

Outcomes that fill themselves

A run finishing is not the same as the work holding up. Most teams never label most runs, so Morse can fill in the label from sources that already know what happened. This page covers the three sources, which one wins when they disagree, and what the labels do not mean.

Where to see it: Settings > Outcomes (/settings/outcomes) shows, per agent, how many runs are reported, rule-labeled, inferred or unlabeled, and how many of the labeled ones are still provisional. Connecting GitHub and turning on a rule are both optional steps on that page.

The three sources

GitHub merged and reverted pull requests

For a coding agent, a merged pull request is the outcome. With the GitHub connector connected, a run that carries a record key github_pr = owner/repo#number gets a provisional success when that pull request merges. If a verified revert merges inside the agent’s window (7 days by default), the success is replaced by a failure, and the history keeps both events with the failure linked to the success it corrects.

These labels have the webhook provenance: they come from a system that reports on the run’s work, not from a model’s guess.

Success rules

A rule is a small declarative check that turns facts you already send into a label. Two templates ship: coding (verified pull request merged, no qualifying revert) and support (ticket resolved and not reopened for seven days). The rule writes a label only if you map its result to an outcome. See Outcome rules.

Rule labels have the rule provenance and are shown with the rule’s criterion and version.

Inferred outcomes

If you have turned on inferred outcomes, Morse can judge a run from its own trace and write an inferred label. This is an estimate. It never overrides a label from any source above it, and it is not counted as a confirmed success.

Which label wins

Every run holds one winning label. When two sources disagree, the higher one stays:

  1. developer (set in your code, or by a person in the dashboard)
  2. classifier-confirmed
  3. webhook (GitHub merged or reverted)
  4. rule
  5. record (sent to POST /outcomes/by-key)
  6. inferred
  7. eval (a sampled quality check)
  8. run status only (the run finished and nothing reported whether it worked; shown as implicit in the API)

Two details matter more than the order:

  • A rule never replaces a record label. If you sent a record outcome for a run and a rule disagrees, the record label stays and the rule’s result is kept as a visible, non-winning candidate (Blocked by a record label). A record outcome that arrives later does replace a rule label. Rule results therefore never remove a run from the confirmed-success count.
  • Rule and inferred labels are not human-confirmed. In the coverage breakdown, Reported counts developer, classifier-confirmed, webhook and record labels. Rule and Inferred are counted separately and described as estimates.

Within one source, business time orders the events, not arrival time. A revert that is delivered late still lands after the merge it reverts, and a re-delivered merge cannot bring a reverted run back.

Provisional and mature

A label can carry a maturity date. A merged pull request is a provisional success until its merge time plus the agent’s window; after that it is mature. Coverage reports both counts. Maturity belongs to the label, so it does not depend on whether the run is still open.

Morse does not predict when a label will arrive. A run with no label shows as unlabeled with no date, and the next step on the page is something you can do now: connect GitHub, add a record key, or complete sampled quality checks.

What this does not do

  • Merged is not code quality. A merged pull request that is never reverted is a weak signal that the change was kept, not that it was correct, safe or well written.
  • GitHub is the only connector. Support tickets and other records come in through the record outcomes API, not through a provider connection.
  • No prediction. Morse does not estimate when outcomes will arrive, and an empty window shows “no data”, not a zero rate.
  • A revert it cannot prove stays unresolved. Manual, cherry-picked and partial reverts do not produce a failure on their own. An authorized admin can confirm one.
  • Rules and inference are estimates. They are not promoted to confirmed outcomes by repetition.

Tracing does not depend on any of this. Runs without labels are still traced and priced; they are reported as unlabeled.

Last updated on
Honest NumbersLifecycle