OTLP export (OpenTelemetry push)
Many container platforms run an in-cluster OpenTelemetry collector and expect workloads to push telemetry: they inject the standard OTEL_EXPORTER_OTLP_ENDPOINT / OTEL_EXPORTER_OTLP_HEADERS / OTEL_RESOURCE_ATTRIBUTES / OTEL_SERVICE_NAME variables into every container and scrape nothing. They do not scrape stdout or a /metrics route. On such platforms, one App option turns on a dependency-free OTLP/HTTP pipeline:
That tees the app logger's output to the collector as OTLP logs, records http.server.request.duration per the OTel HTTP semantic conventions, and (because of traces: true) traces every matched route. telemetry: true gives you logs and metrics; traces are opt-in. With no endpoint configured it is a silent no-op, so it is safe to keep enabled in development. There is no OTel SDK, loader hook, or runtime dependency.
- 01platform-injectedOTEL_* env varsendpoint, tenant headers, resource attributes
- 02logger + hookslog lines, request duration, one span per request
- 03batched exportersOTLP/HTTP JSON: /v1/logs, /v1/metrics, /v1/traces
- 04collectorroutes on tenant headers, converts, forwards
- 05standard dashboardsGrafanasemconv names; logs link to traces
What gets exported
- Logs: every line the app logger writes. JSON lines are decomposed into an OTLP record:
msg/message/eventbecomes the body,levelmaps to the OTLP severity, well-formedtrace_id/span_id/trace_flagsfields become the record's trace context, and remaining fields ship as attributes (structured metadata in Loki-style backends). Non-JSON lines ship verbatim. http.server.request.duration: a histogram with the spec bucket boundaries and attributeshttp.request.method(normalized to the well-known set, else_OTHER),http.route(the matched route template, e.g./books/:id, fromctx.routePath),http.response.status_code,url.scheme, anderror.typeon5xx. Only requests that matched a route are recorded. Unmatched 404 / 405 responses and synthetic OPTIONS preflights record nothing, so raw paths can never mint metric series.- Traces (
traces: true): one SERVER span per matched route fromotelTracing(), named after the route template (GET /books/:id), with the HTTP semantic-convention attributes. See Traces.
Configuration
Everything defaults from the standard environment variables. Override per signal or per exporter:
The conventional injected endpoint is the gRPC form (http://host:4317). A trailing :4317 is rewritten to :4318, the collector's OTLP/HTTP port. Header values often carry multi-tenant routing credentials. DaloyJS never logs them, and the endpoint in the telemetry.otlp boot log line has any userinfo and query string stripped. Spec values are percent-encoded (tenant_id=a%2Cb arrives as a,b).
Environment variables
| Variable | Effect |
|---|---|
OTEL_EXPORTER_OTLP_ENDPOINT | Base URL for every signal; /v1/logs, /v1/metrics, /v1/traces are appended. |
OTEL_EXPORTER_OTLP_{LOGS,METRICS,TRACES}_ENDPOINT | Full URL for one signal, used as-is. Wins over the base URL. |
OTEL_EXPORTER_OTLP_HEADERS, OTEL_EXPORTER_OTLP_{SIGNAL}_HEADERS | key=value,... request headers; the per-signal list overlays the generic one. |
OTEL_EXPORTER_OTLP_TIMEOUT, OTEL_EXPORTER_OTLP_{SIGNAL}_TIMEOUT | Export POST timeout in ms (default 5000). |
OTEL_RESOURCE_ATTRIBUTES, OTEL_SERVICE_NAME | Resource attributes on every export. |
OTEL_TRACES_SAMPLER, OTEL_TRACES_SAMPLER_ARG | always_on, always_off, traceidratio, parentbased_always_on, parentbased_always_off, parentbased_traceidratio. Default parentbased_always_on. |
OTEL_SDK_DISABLED=true | Kill switch: disables every exporter, even with an explicit endpoint. |
OTEL_{LOGS,METRICS,TRACES}_EXPORTER=none | Disables one signal. |
Explicit options in code win over the environment, except the two kill switches, which belong to the operator. DaloyJS speaks OTLP/HTTP with JSON bodies only, whatever OTEL_EXPORTER_OTLP_PROTOCOL says. Every collector accepts that on port 4318.
Traces
traces: true installs otelTracing() on every route backed by createOtlpTracer(), a dependency-free tracer that batches ended spans and pushes them to /v1/traces. Like the HTTP metrics it covers matched routes only, so a flood of unmatched paths produces no spans.
- Sampling is parent-based: a span with a parent follows the parent's sampled flag, so a trace is kept or dropped as a whole. New root traces are kept with probability
sampleRatio, decided from the trace id. Sampled-out spans are cheap no-ops that still carry a valid identity, sotraceparentkeeps flowing downstream. - Propagation continues an inbound
traceparentthrough the strictparseTraceparent(). On an internet-facing edge the client controls that header, so setpropagate: falsethere (or strip the header at the load balancer). Otherwise a client can pick your trace ids and opt its own requests out of sampling. - Logs link to traces. Handler lines written through
ctx.state.logcarry the span's ids and reach the collector as the log record's trace context. - Child spans take the request span as an explicit parent, because hooks cannot activate an OTel context:
Fail-safe by contract
- A dead or misconfigured collector never affects request serving: export errors are swallowed and counted in
droppedBatches. That counter mixes overflowed log lines or spans, failed POSTs, and series refused at the cardinality cap, so do not treat it as "POSTs that failed" alone. - Every export POST is aborted after 5 seconds (override with
exporter.flushTimeoutMsorOTEL_EXPORTER_OTLP_TIMEOUT) so a blackholed collector cannot stall flushing forever. Redirects are refused, so tenant headers cannot follow a 302 off-box. - The log queue (1,000 lines) and span queue (2,048 spans) are bounded, drop-oldest. Metric series are capped and attribute values length-truncated. Each span is capped at 128 attributes and 128 events, with long values truncated, so hostile input cannot grow memory.
- Metrics use cumulative temporality: totals survive a failed push and the next successful push carries them.
flush()never rejects. Called while an export is already in flight, it waits for that export and then sends what was queued meanwhile, so the shutdown flush and the serverless adapters'waitUntilnever finish early.
Standalone exporters
The pieces compose individually from @daloyjs/core/otlp for custom signals, for example domain-specific counters next to the built-in HTTP histogram:
Serverless and isolate runtimes
Long-lived Node, Bun, and Deno processes flush on an unref'd interval and again on graceful shutdown. Cloudflare Workers, Vercel isolates, and AWS Lambda freeze when the handler returns, so the interval never runs. Use the adapters:
toFetchHandlerfrom@daloyjs/core/cloudflarecallsctx.waitUntil(app.telemetry.flush())after each request.toFetchHandler/toWebHandlerfrom@daloyjs/core/vercelusesglobalThis.waitUntilwhen the runtime provides it.toLambdaHandlerawaits the flush before returning, so the export is billed as part of the invocation.
Workers without nodejs_compat cannot read process.env. Pass telemetry: { exporter: { endpoint, headers } } from the Worker bindings instead of relying on OTEL_*.
What this does not do
- A caller-supplied
Loggerinstance (pino, winston, a custom sink) is not intercepted. Tee it yourself withcreateOtlpLogExporterand your logger's write hook. The built-in logger, or{ level }, is teed automatically. - Export uses raw
fetch. It does not go throughfetchGuard. The collector URL is operator config (the same trust class as a database URL), and in-cluster collectors are typically private IPs that SSRF defaults would refuse. - No gRPC and no protobuf: OTLP/HTTP with JSON bodies only.
- No automatic context propagation. Outbound
fetchcalls are not instrumented, and there is no context manager. UseformatTraceparent(span.spanContext())to propagate, and pass parents explicitly for child spans. - Calling
app.fetchdirectly on an isolate, skipping the adapter, will queue telemetry and mostly never send it. UsetoFetchHandler/toLambdaHandler, or callawait app.telemetry.flush()yourself.
Pull vs push, and tracing
app.metrics()(Prometheus) is the pull pillar: a scrape endpoint with Prometheus naming. Use it when something scrapes you. Usetelemetrywhen a collector expects pushes. They coexist. ThehttpMetrics()route label also benefits fromctx.routePath.otelTracing()is the tracing hook either way.telemetry: { traces: true }pairs it with the built-in OTLP tracer. If you already run the OTel SDK, pass its tracer tootelTracing()yourself and leavetracesoff.
Querying in Grafana
After the collector's Prometheus conversion the histogram appears as http_server_request_duration_seconds_*:
Spans land in Tempo (or Jaeger) under the service.name resource attribute. With log correlation on, Loki log lines carry the trace id, so a Grafana "derived field" or the Tempo data source's "logs for this span" link jumps between the two.