Glossary
Every term on a Morse screen that a newcomer could not guess, in plain words. Each entry says what the term is, sometimes why it matters, and where to see it. Method names (the statistics behind a number) are kept here and on the pages they link to, so the screens themselves can stay short.
Entries are alphabetical. Each term is a heading, so you can link straight to one, for example
/start/glossary#agent-run.
A
Action needed
The old name of needs attention on Dashboard cards. Morse now uses one set of verdict words everywhere: needs attention, worth watching, healthy, baseline forming and not enough labeled outcomes.
See: Dashboard.
Agent
A named AI workflow whose runs you send to Morse. Morse creates the agent the first time it sees a run with that name, so there is nothing to register.
See: Agents, Quickstart.
Agent run
One execution of an agent from start to finish. Morse stores it as a trace. Most screens say “run”; the technical record is the trace.
See: Traces.
Agent version
Your own label for the deployed agent, such as a release name or build number, up to 128 characters. A new agent version tells Morse a change happened.
See: Record keys and versions.
Agreed “no”
On Verify, a spot-check where Morse’s judge said “no” and your answer agreed. Morse needs 8 of them (at least 2 from the borderline “no” group) before an inferred outcome can be evaluated, because agreeing on easy “yes” answers says little about whether the judge catches failures.
See: Verify.
Agreement
How often your spot-check answers matched Morse’s judge. 90% agreement on 40 checks means you agreed on 36 of them. Because a small number of checks can flatter the judge, Morse leads with agreement (at least).
See: Verify.
Agreement (at least)
The low end of the agreement figure: with this many checks, your checks agree with Morse at least this often (95% confidence). Morse requires at least 70%, so a lucky streak on a few checks cannot pass; Verify reads “Your checks agree with Morse at least 70% of the time” when it does. It doesn’t say the judge is calibrated. Earlier screens called it “agreement (worst case)”. Statisticians call it the Wilson 95% lower bound.
Why it matters: it keeps small samples honest. 9 agreements out of 10 checks is 90% agreement but only about 60% at least.
See: Verify, Agent detail.
Alert
A rule that watches one measure (failure rate, latency, cost, eval score, a service target’s early warning and more) and tells you when it crosses a line you set, or, in anomaly mode, when it is clearly worse than that agent’s own usual. Two more types watch a budget.
See: Reliability, Alert detail.
API key
The secret that lets your code send runs to Morse and ties them to an environment. Each environment has exactly one active key. Rotating a key keeps the old one working for 24 hours so a rollout does not break. Personal API keys are separate: they let you read or write through the API as yourself, with either full or read-only access. An environment key is shown once, at creation.
See: Workspace settings, Environments, Profile.
Ask Morse
The chat panel available on every dashboard page. You ask in plain language and it answers from your own Morse data, with links to the evidence. It reads; it does not change anything.
See: Ask Morse.
Audit log
The record of who changed what in your organization: settings, members, API keys, webhooks, workspaces and environments. Owners and admins only. Rows show the action in words, such as “Personal API key created”, plus who did it and where from.
See: Workspace settings.
B
Baseline
An agent’s own recent history, used as the yardstick for “better or worse than usual”. Morse judges Dashboard cards against the agent’s baseline, never against a fixed number, because a good cost for one team is a bad one for another.
Baseline forming
A neutral status: there is not enough history yet to say whether this is normal, so Morse does not guess. An anomaly alert, for example, needs 24 hours of activity before it can say what “usual” looks like. Nothing is wrong; the number simply is not ready.
As an agent verdict, it means the agent has at least 20 labeled runs in the window, but they fall on too few days to compare against its own history: Morse needs labeled runs on at least four different days. Below 20 labeled runs the verdict is not enough labeled outcomes instead.
See: Honest numbers, Reliability.
Benjamini-Hochberg
See multiple-testing correction.
Blind check
See spot-check.
Borderline
See group (stratum).
Budget
A dollar amount for a day, week or month, for your whole organization or one agent. Morse shows spend against it, forecasts where the period will end, and alerts you. A budget never blocks, slows or drops anything; it only changes what Morse reports.
See: Budgets and Forecast.
C
Cache hit rate
See prompt cache.
Change
A moment when an agent’s setup changed: a new agent version, a new git revision, or a different model, system prompt or tool description. You can also add one by hand. Morse tracks changes so it can compare the runs before and after.
See: Change Verdicts.
Change verdict
After a change, Morse compares the runs before and after, one measure at a time (cost per run, two latency measures, error rate, success rate), and says improved, worsened, unchanged or too early. It shows movement, not cause, and it never combines the measures into one score.
See: Change Verdicts.
Confidence
How much weight a number can bear. Morse labels a figure “low confidence” when it rests on too few runs or on rough inputs: cost per confirmed outcome with fewer than 20 confirmed outcomes, or a context breakdown whose estimated sizes are far from the billed token counts. An investigation also shows a confidence rating (high, medium or low), but that is the model’s own self-assessment, not a measured probability.
See: Cost, Investigations.
Configuration fingerprint
A short hash of what a run used: its models, system prompts, tool descriptions, prompt references and agent version. Two runs with the same fingerprint ran the same setup. Morse uses it to notice that a setup changed. The hash itself is not shown on screen.
See: Record keys and versions, Change Verdicts.
Confirmed outcome
An outcome that something other than a guess has vouched for: your code, a person in the dashboard, a webhook such as a merged pull request, or an outcome you sent for the record. Inferred, rule-based and eval-judged labels are shown but are not counted as confirmed.
See: Outcomes that fill themselves, Cost.
Context window utilization
How much of the model’s context window a call filled. Score Settings can queue runs for review when utilization is above 85%. A call that fills most of the window is a candidate for trimming.
See: Score Settings, Context.
Correction
A display-only adjustment to an agent’s inferred success rate, using your spot-checks to account for the judge’s mistakes. It is shown only when the checks are enough to bound the result to a useful range, and otherwise Morse says what is missing instead. It never changes confirmed counts, cost per confirmed outcome, service targets or alerts. Statisticians call it judge-error correction.
See: Agent detail.
Cost coverage
How much of your spend Morse could price. Two separate numbers share the name: trace cost coverage (how much of one trace’s model spend has a price) and catalog pricing coverage (how many of your models have a known rate). Unpriced spend is never counted as $0.
See: Cost, Honest numbers.
Cost per confirmed outcome (CPSO)
All the spend of the runs in scope, divided by the number of confirmed successful outcomes. Spend on failed retries and on runs with no recorded outcome is included, which is why it is a harder number than cost per run. With fewer than 5 confirmed outcomes it shows a dash and what to do next. It describes what confirmed outcomes cost; it does not claim savings or causes.
See: Cost, Cost per good answer.
Coverage
How much of something Morse could measure, always with its denominator. Four kinds are named separately and never combined:
- Outcome coverage — runs that have an outcome from any source, out of all runs. It says nothing about whether answers were checked.
- Scoring coverage — runs that were scored by a check, out of all runs. It says nothing about whether the work succeeded for the business.
- Pricing coverage — model calls that have a known price, out of all model calls. Spend on unpriced calls is not in the total.
- Customer attribution coverage — runs attributed to a customer, out of all runs. Unattributed runs are left out of per-customer figures.
Each reads like “Outcome coverage: 840 of 1,200 runs have an outcome (70%).”
See: Honest numbers.
D
Demo workspace
A read-only workspace with sample data, so you can see what a full account looks like. It follows the same rules as a real account, only with more data. Switch it on or off in the workspace switcher; whether it appears there is a toggle in Workspace settings.
See: How demo mode works, Lifecycle, Workspace settings.
Deploy marker
A dated tick on a chart showing when Morse detected a new version or revision of an agent, so you can read a dip or a jump against what shipped. Appearing next to a change is timing, not proof of cause.
E
Environment
See workspace and environment.
Error group
Failed runs bundled because they fail the same way: same kind of step, same exception type and the same message once ids, numbers and other changing parts are stripped out. Groups are ranked by what the failures cost. Also called a failure group or repeated failure. A group is a pattern, not a root cause.
See: Reliability.
Estimated rate
A model price taken from a public third-party price list (refreshed weekly) rather than the provider’s own price page or the provider’s billed amount. It is a good approximation, and Morse labels it as an estimate so you can tell it from provider-billed cost. The Cost screen used to call it an aggregator rate.
See: Cost.
Eval case
See test case.
Eval cases, test cases and “cases” all mean the same thing on screen.
Explicit outcome
See reported by you.
F
False-alarm rate
The chance a comparison reports a difference that is really just noise. Regression comparisons in the Regression tab use 5%. Change verdicts are stricter: each of the five measures uses 1%, so the chance of any false verdict on one change stays at or under 5%. Statisticians write this as alpha.
See: Regression, Change Verdicts.
Finality window
How long Morse waits after a record’s latest run before counting it in cost per confirmed outcome. Outcomes often arrive late (a refund, a reopened ticket), so a record whose last run is inside the window is shown as “too recent to count” and left out of both spend and confirmed successes. The default is 24 hours; you can set 1 to 168 hours per agent.
See: Cost.
Finding
A candidate to review that Morse found in your first traces, such as a repeated tool call, a large share of input spend in one part of the prompt, or a missed prompt-caching opportunity. Findings are grouped by how the number was measured. They are starting points, not verdicts.
See: Cost findings.
Forecast
Where Morse projects a budget period will end, using the weekday pattern of your recent spend. It is plain arithmetic you can recompute, not a model, and it needs at least 14 days of spend history before it appears. Daily budgets have no forecast. The range around it is a typical range, not a promised percentage.
See: Budgets and Forecast.
G
Group (stratum)
Verify spot-checks are drawn from groups defined by the judge’s own answer: likely yes, likely no, and borderline yes or borderline no (where the judge was close to its undecided line, within 15 points of confidence). Drawing from every group means the answers the judge was least sure of are always checked, not only the easy ones. Statisticians call the groups strata.
See: Verify.
H
Health status
An agent’s verdict, shown beside its name on the Agents list and the agent page. The same verdict appears on Home and in the weekly brief.
See: Agents, Agent detail.
Healthy
The verdict when nothing needs action: no alert is firing, no incident is open, every service target is met, no recent change made a measure worse, and success is steady against the agent’s own baseline. On a Dashboard card it means the capability is stable against its baseline.
I
Implicit outcome
Incident
A record of something that broke, with a severity, a status (open, investigating or resolved) and a timeline. You can file one by hand or from an alert.
See: Reliability, Incident detail.
Include in quality verdict
A setting on each rule check. When it is on, the rule’s results feed the quality verdict on the Quality page. It is off by default, so a new rule cannot swing your verdict before you trust it.
See: Score Settings.
Inferred from run status
The outcome source for a run that finished but where nothing told Morse whether it worked. It is the weakest label: it is never counted as a confirmed success. The opposite is reported by you. (The internal name is an implicit outcome.)
See: Outcomes that fill themselves.
Inferred outcome
A success or failure that Morse’s judge decided from the run itself, for agents where you have turned inferred outcomes on. It is an estimate and stays separate from confirmed outcomes. It starts as a shadow label and is only used once your spot-checks show the judge is reliable.
See: Agent detail, Verify.
Investigation
The automatic first look Morse takes when an alert fires. It gathers traces, a baseline comparison, recent deploys and related alerts, and returns a hypothesis with its evidence. It is a starting point for your own check, not a verified cause.
See: Investigations.
J
Judge
An AI model that reads a run (or one part of it) and answers a question about it, such as “did the agent handle the request correctly?”. Morse uses judges for inferred outcomes and for LLM-judge scorers. Judges can be wrong, which is why Morse asks you to spot-check them. Also called an LLM judge.
See: Verify, Score Settings.
L
Judged run
A run with a recorded success or failure. Success rate is computed over judged runs only, so it reads “312 of 420 judged runs succeeded · last 7 days”. A run with an outcome label but no success or failure is labeled yet not judged.
See: Agents.
Labeled run
A run that has an outcome label or a recorded success or failure from a real source (not the automatic default). Outcome coverage is labeled runs divided by all runs.
See: Outcomes that fill themselves.
See: Outcomes that fill themselves.
List price
The provider’s published price for a model, used to price a call when no better source applies. Estimates such as prompt-caching savings are at list price, so your contract price may differ.
See: Cost, Cost findings.
M
Matched to your records
The share of runs that carry a record key. The Cost screen shows it so you know whether cost per confirmed outcome counts business records (tickets, orders, pull requests) or individual runs. A shift of more than 15 points since the previous period is flagged, because it can change the figure on its own.
See: Cost, Record keys and versions.
Maturing
See too recent to count.
Multiple-testing correction
When you compare many scorers at once, a few will look different by luck. Morse corrects for that across all scorers in one comparison, with the Benjamini-Hochberg method at a 5% false-alarm rate, so testing more scorers does not manufacture regressions.
See: Regression.
N
Needs attention
The red verdict: an alert is firing, a critical or high incident is open, a service target is breached, or success is clearly worse than the agent’s own baseline. On Home’s Needs attention list, items with this label come first.
Not enough labeled outcomes
The verdict when Morse cannot judge yet because there are too few judged runs in the window: fewer than 20 labeled runs, or too small a share of runs judged for the success rate to be shown. Morse shows coverage instead of guessing. With enough labeled runs but too few days of history, the verdict is baseline forming.
See: Agents.
No recent data
A Dashboard card status: there are no runs at all in the window, so there is not even a partial history. It is different from baseline forming, where some data exists but not enough to judge.
See: Dashboard.
Nonresponse
See skipped check.
O
Outcome
What happened to a run’s work: it succeeded or it failed. Outcomes can come from your code, a person, a webhook, a record you report on, a rule, or Morse’s judge, and Morse remembers which. A run with no outcome is unlabeled, not failed.
See: Outcomes that fill themselves, Set up outcomes.
P
Performance brief
Morse’s ranked summary of how your agents are doing: for each agent, its success rate against its own history, its spend, and what needs attention first. The verdict is built from labeled production outcomes, spend and trends. Sampled quality checks are shown beside it, never blended in. Also called the verdict.
See: Dashboard.
Prompt cache
A provider feature that reuses an unchanged start of your prompt at a lower price. The cache hit rate is the share of input tokens that were read from the cache. Morse shows the hit rate, the estimated savings and the write cost, and flags agents that repeat a long identical prefix without caching.
Prompt version
A numbered version of a named prompt, attached to a run when your code declares which prompt it used. Morse compares versions by cost per confirmed outcome.
See: Prompts, Record keys and versions.
Provider-billed
A cost that the model provider reported on the call itself, so it is as exact as it gets. It is tried first, before any catalog rate.
See: Cost.
Provisional outcome
An outcome that can still change. A merged pull request is a provisional success until its merge time plus the agent’s window (7 days by default), so a revert inside the window can turn it into a failure. After that it is mature. Outcome coverage shows both counts.
See: Outcomes that fill themselves.
Q
Quality verdict
The Quality page’s one-line read on quality: healthy, at risk or blocked, with the weakest scorer and the sample sizes behind it. It comes from sampled scorers and never mixes with cost.
See: Quality.
R
Record key
A tag you put on a run naming the business record it handled, for example a support ticket, an order or a pull request. With a record key, an outcome you report later (reopened, refunded, merged) finds every run that handled that record. It is also what lets cost per confirmed outcome count records instead of runs.
See: Record keys and versions, Record outcomes API.
Regression Lab
The screen where you replay saved test cases and compare two replay runs with real statistics. A replay re-scores a frozen copy of the recorded input and output with the current scorers. It does not re-run your agent.
See: Regression.
Replay run
One pass of Regression: it re-scores a dataset (or one case) with the current scorers so you can compare it with another run.
See: Regression.
Reported by you
An outcome that came from your side: your code called set_outcome(), or a person set it in the
dashboard. It is the strongest label. (The internal name is an explicit or developer outcome.)
See: Outcomes that fill themselves.
Rule check
A simple deterministic test on a run, such as “the output matches this JSON shape” or “the refund is under the cap”. Rule checks run on every in-scope run, are not sampled, and are not blended with judge scores. A separate kind, a success rule, turns facts you already send into an outcome label.
See: Score Settings, Outcome rules.
S
Scorer
A check that gives a run a score or a pass/fail: a built-in one (hallucination, tool-call validity, and so on), a custom judge you write, or a rule check. Scorers run on a sample of traffic, and a run with no applicable scorer simply gets no score.
See: Score Settings, Quality.
Service target
A reliability target someone on your team set for an agent, such as “99% success over 30 days” or “95% of responses finished within 3.0 s”. Engineers call it an SLO. Each one shows its status (healthy, at risk, breached or not enough data) with the measured fact behind it, the failure allowance left (how much of its error budget remains in the window), and the early warning alert watching it, if any.
See: Reliability.
Shadow label
An inferred outcome that Morse records but does not yet use. When an owner or admin turns on inferred outcomes, Morse answers two questions on each finished run (“Handled correctly” and “Need met”). Each answer stays in shadow, away from verdicts, alerts and cost per confirmed outcome, until your spot-checks show the judge agrees with you often enough and you choose to count it. A counted label is always marked inferred.
See: Agent detail.
Skipped check
A spot-check you skipped or marked unsure (“couldn’t tell”). Skips are tracked per group because they can bias the result: if most borderline checks get skipped, the ones you did answer are not representative. Statisticians call this nonresponse. More than 15% skipped in a borderline group blocks evaluation.
See: Verify.
SLO
See service target.
Spot-check
One question put to you on Verify: here is a run and what Morse’s judge decided; do you agree? You cannot pick or reorder them: Morse draws them at random from each group, and hides the judge’s answer until you submit so it cannot sway you. That is why they are called blind, random checks.
See: Verify.
Span
One step inside a run: a model call, a tool call, a database query, an HTTP request, and so on. Every span has a type, and the type decides how it is priced and where it shows up. The canonical types are agent, llm, tool, http, db and custom, plus memory, retriever, guardrail, handoff and queue.
See: Data model, Traces.
Run status
Whether a run finished operationally: completed or failed. It is not the same as success. A run can complete and still fail its task.
See: Traces.
Success rate
Successful runs divided by judged runs, for a time window. Runs with no outcome are left out, not counted as failures, so the basis is always shown (“312 of 420 judged runs succeeded · last 7 days”). When some labels were inferred, the split is shown too. Verdicts and service targets use reported outcomes only. Screens label it Success rate (runs with a reported outcome), or “…reported or inferred outcome” when inferred labels are included.
See: Agents, Is my agent getting better?.
System prompt segment
The part of a call’s input that is the system prompt. Context Economics shows its share of input spend, named by kind with the version hash after it, for example “System prompt (a9b84bbb)”, so you can tell which prompt version a number belongs to.
See: Context Economics.
T
Test case
A real production run saved with one click (“Save as test case”) so you can replay it and gate on it later. Test cases are grouped into datasets. Also called an eval case or just a case.
See: Cases.
Too recent to count
A record whose latest run is still inside its finality window, so its outcome may not have arrived. It is left out of cost per confirmed outcome (both spend and successes) until the window passes. The internal name is “maturing”.
See: Cost.
Trace
The full record of one agent run: its steps, timings, cost, tokens and outcome. A thread groups the traces of one conversation.
See: Traces, Data model.
U
Unconfirmed success
A run that looks successful but that nothing has vouched for: an inferred, rule-based or eval-judged success. Cost per confirmed outcome lists these but does not count them.
See: Cost.
Unlabeled run
A run with no outcome yet. It counts toward activity and cost but not toward success or failure. Unlabeled never means failed.
See: Set up outcomes.
Unpriced
A model call Morse has no applicable rate for. It is left out of totals and counted in coverage; it is never shown as $0 and never given a guessed price. Other price sources are provider-billed, list price, an estimated rate (a rate from a public third-party price list, refreshed weekly) and a user-configured rate (one you set yourself).
See: Cost, Honest numbers.
User-configured rate
A model price you set yourself in Morse’s price catalog, for example for a self-hosted model or a negotiated rate. Morse uses it as given and labels the cost as a user-configured rate.
See: Cost.
V
Verdict
Morse’s one judgement of an agent, the same on Home, the Agents list, the agent page and the weekly brief: needs attention, worth watching, healthy, baseline forming or not enough labeled outcomes. The last two withhold a verdict rather than give one. It takes the worst of: a firing alert, an open incident, a service-target breach, success against the agent’s own baseline over reported outcomes, and a recent change that made a measure worse. Each verdict names its reasons and its window (“last 7 days”).
See: Agents, Weekly brief.
Other verdicts use the same rule of answering with evidence: the performance brief, change verdict and quality verdict. Each says “too early” or “not enough labeled outcomes” instead of guessing.
Verify
The screen where you answer spot-checks. One item at a time, with the judge’s answer hidden until you submit. Progress shows runs checked, agreed “no” and agreement (at least), and what is still needed.
See: Verify.
W
Wilson lower bound
See agreement (at least).
Window
The span of time a number covers, such as the last 7 days. Every figure on a screen says its window, because the same agent can look different over 24 hours and over 30 days.
Workspace and environment
Your organization is the account and the only access boundary. A workspace groups work inside it. An environment, such as production or staging, is a lane inside a workspace and carries the API key; in the UI it appears as a filter, not a separate place.
See: Environments, Workspace settings.
Worth watching
The amber verdict: success has drifted from the agent’s baseline, a medium or low incident is open, a service target’s failure allowance is running low, or a recent change made a measure worse and nothing has measured it back. Nothing is failing outright yet.