Infrastructure spans
When your agent runs inside a Morse trace, the SDK records the database queries, outbound HTTP requests and queue publishes it makes as child spans of that trace. You change no code. The point is correlation: when an agent run is slow or fails, the call underneath it is on the same timeline.
This is trace-anchored only. Nothing is recorded unless a Morse trace is open, and Morse does not monitor your infrastructure on its own.
Requires SDK 0.7.0 (not yet released; 0.6.0 does not have this). The default-on behaviour,
the MORSE_CAPTURE_* switches and queue spans below ship in 0.7.0. On the released 0.6.0,
database capture needs MORSE_CAPTURE_DB=1.
From SDK 0.7.0 this is on by default. Earlier versions needed MORSE_CAPTURE_DB=1. To keep
the old behaviour, set MORSE_CAPTURE_INFRA=0 — see Turning it off.
What is captured
| Kind | Python | TypeScript | Span |
|---|---|---|---|
| DB | sqlite3, psycopg2, psycopg, Django | pg, mysql2, better-sqlite3 | db span named for the SQL verb |
| HTTP | httpx, requests | fetch (undici) and node:http / node:https | http span named METHOD host |
| Queue | Celery, SQS (boto3) | BullMQ, SQS (@aws-sdk/client-sqs) | queue span named publish <destination> |
An HTTP span also records the response status. A 5xx, 408 or 429 marks the span failed; an
ordinary 4xx does not, because that is your request being wrong rather than the dependency failing.
A failed db, http or queue span stays failed on its own row and still shows up in repeated
failures, but it never marks the whole trace failed. Your agent decides that: a caught database
error or a retried 503 does not turn a successful run into a failed one.
Queue spans carry the OpenTelemetry messaging attributes messaging.system,
messaging.operation.type and messaging.destination.name in the span metadata. On publish the
SDK adds a W3C traceparent to the message (Celery headers, SQS message attributes, BullMQ
telemetry.metadata) so a consumer can join the same trace — see Consumers.
LLM provider calls are not recorded as HTTP spans. The framework adapters already record them as
llm spans, so an http span as well would count them twice. The Morse ingest host is excluded
too.
What is never recorded
- Bound SQL parameters. Only the statement text is kept, never the values passed with it.
- Values written into the SQL string. If your code builds
WHERE email = 'a@b.com' AND id = 42by string formatting, the stored statement isWHERE email = ? AND id = ?. String, number, hex and dollar-quoted literals and SQL comments are removed before the span leaves your process. - URL paths and query strings. The HTTP span name is the method and the host.
GET api.example.com, notGET api.example.com/users/42?token=…. - Headers and bodies, in both directions.
- Message bodies, task arguments and job data.
- The text of an error. A failed infra span records the error class (and the driver’s error code for databases). Driver, URL and broker messages routinely contain row values, paths and credentials, so the message is dropped.
The SDK’s test suite seeds known canary values into all of the above and fails the build if any of them appears in an exported span.
Scrubbing is a safety net, not a licence to format values into SQL. Use bound parameters. Two
limits: double-quoted text is treated as an identifier, so a database that reads "abc" as a
string literal (MySQL without ANSI mode) is not scrubbed there; and a secret that is part of an
identifier, such as a table named after a customer, is not a literal and is kept.
How many spans one trace keeps
A trace keeps at most 200 db spans, 100 http spans and 100 queue publish spans. Calls past
the limit are not recorded individually. Instead the trace gets one summary span per kind, named
like +300 more db calls (capped at 200), with the count in infra.suppressed_count. An ORM
looping 500 queries inside a tool shows as 200 spans plus one line saying how many more there were.
Turning it off
| Variable | Effect |
|---|---|
MORSE_CAPTURE_INFRA=0 | Off for database, HTTP and queue |
MORSE_CAPTURE_DB=0 | Off for database only |
MORSE_CAPTURE_HTTP=0 | Off for HTTP only |
MORSE_CAPTURE_QUEUE=0 | Off for queue publishes only |
MORSE_CAPTURE_HTTP_EXCLUDE_HOSTS=a.com,b.com | Skip these hosts (and their subdomains) |
0, false, no and off all count as off. A specific variable wins over MORSE_CAPTURE_INFRA,
so MORSE_CAPTURE_INFRA=0 with MORSE_CAPTURE_DB=1 still captures the database. The variables are
read once, when the SDK initialises.
Expect more spans per trace after upgrading. If a dashboard of yours counts spans per trace, that count rises by the number of calls your agents make to a database, an HTTP service or a queue.
Consumers
Producer spans are on by default. Continuing the producer’s trace on the consumer side changes how
your worker traces are shaped, so it is opt-in: set MORSE_CAPTURE_QUEUE_CONSUMER=1.
- Celery (Python): automatic. The worker joins the producer’s trace and records a
process <queue>span. - SQS (Python): wrap the handler:
with morse_ai.queue_capture.continue_from_sqs_message(msg, queue_url=url): ... - BullMQ and SQS (TypeScript): neither library has a stable worker hook, so wrap the handler:
continueFromBullMqJob(job, "worker-name", () => handle(job)),continueFromSqsMessage(msg, "worker-name", () => ...).
If the message has no traceparent, or it is malformed, the worker starts a fresh trace. It never
raises.
Limits you should know
- TypeScript database and queue capture patch at
require()time. An ESM-only app needs OpenTelemetry’s import hook for thepg,mysql2,better-sqlite3, BullMQ and SQS patches to apply. HTTP capture usesdiagnostics_channeland works in ESM and CJS. - Python wraps
connect()forsqlite3,psycopg2andpsycopg, so a connection opened beforemorse_ai.init()runs is not covered. Django is covered either way. - Other clients are not covered:
aiohttp,urllib3used directly,asyncpg, Kafka and Redis Streams.