Skip to Content
Getting StartedCommon tasksWhat does a good answer cost?

What does a good answer cost?

Open Cost Intelligence and go to the Workflows tab. The figure there is cost per confirmed outcome: all the spend of the runs in scope, divided by the number of outcomes that something other than a guess has confirmed as successful. It is a harder number than cost per run, because failed retries and runs with no outcome still add to the cost and add nothing to the count.

Steps

  1. Open the figure. Go to Cost Intelligence, then Workflows. Each workflow row shows its outcomes, cost and Per confirmed outcome, with an org-wide line under the list. A workflow is whatever you set in set_context(workflow="refunds"); runs without one are grouped under their agent.
  2. Read the evidence line. Under the list, the org line (for example “Org (30 days)”) states what the figure is counted in and what it rests on: the unit, how many records or runs are confirmed out of the total, how many are matched to your records, how many are too recent to count, and how many unconfirmed successes were left out.
  3. Check the unit. If runs carry a record key, Morse counts business records: a ticket handled across five runs is one outcome. Without keys it counts runs. A mix is labeled mixed and flagged low confidence.
  4. Look at what is left out. Records whose latest run is too recent to count are excluded from both spend and successes until the finality window passes (24 hours by default). Unconfirmed successes (inferred, rule-based or judged) are listed but not counted.
  5. Act on the next step. When the figure is a dash, the line under it says what to do, usually to record more confirmed outcomes or widen the window.

What the numbers mean

  • A dash is not zero. With fewer than 5 confirmed outcomes, Morse shows a dash and the next step. It never guesses and never shows $0.
  • Low confidence appears when fewer than 20 confirmed outcomes back the figure, or when the unit is mixed. Treat the number as a rough guide.
  • Matched to your records is the share of runs carrying a record key. If it moved by more than 15 points since the previous period, Morse flags the figure, because part of the change is how you instrument, not how the agent behaves.
  • Which outcomes count. Only outcomes set by your code, a person, a webhook (such as a merged pull request) or an outcome you sent for the record. A success that Morse inferred is shown but not counted.
  • It is a description, not a saving. The figure says what confirmed outcomes cost. It does not say a change saved money or caused anything.

When there is not enough data

  • No confirmed outcomes yet: record some. See Set up outcomes.
  • Outcomes arrive days later (refunds, tickets)? Give the agent a longer finality window, from 1 to 168 hours. The call is described under per-agent outcome finality.
  • Spend that Morse could not price is left out of totals, never counted as $0. The Coverage & Rates tab shows how much of your spend is priced and where each rate came from.
Last updated on
Why is my agent failing?Can I trust Morse's labels?