Application observability
The governed OpenTelemetry contract for jackin❯ processes.
jackin❯ exports application observability directly to an OTLP/gRPC backend. The backend owns durable history. Product processes keep only bounded current-invocation state in memory for progress and operator UI; they do not write telemetry, diagnostics, or capsule log files.
This page is the living instrumentation contract. Schema changes start in the closed registry and update this reference in the same change.
Composition and delivery
When OTLP is configured and the network policy permits it, each host, daemon, and Capsule process owns its providers and exports traces, logs, and metrics directly. There is no embedded or configured Collector. Standard generic and per-signal endpoint, header, gzip compression, millisecond timeout, CA certificate, and client key/certificate variables are validated before provider construction; only the grpc protocol is supported. Per-signal values override generic values, client key and certificate must be paired, and each timeout is capped at five seconds. OTEL_TRACES_SAMPLER may be unset or parentbased_always_on. OTEL_SDK_DISABLED=true disables providers explicitly.
OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES are syntax-validated but cannot override jackin❯-owned service identity or inject arbitrary Resource fields. Header values, certificate and key paths, and resource input are sensitive configuration: they are never rendered or exported as telemetry attributes. Host exporter credentials and arbitrary Resource input do not cross into a Capsule; only routing and explicitly safe launch context are propagated when Capsule egress is allowed.
Provider construction is best-effort and product work stays fail-open. Export uses bounded batches, five-second per-export deadlines, bounded retry for retryable gRPC status codes, and no retry for authentication or partial-success responses. Saturation drops telemetry instead of blocking product threads. Graceful shutdown force-flushes traces, logs, and metrics concurrently within one shared budget, then shuts providers down in tracer, logger, meter order. Shutdown is idempotent.
jackin diagnostics validate emits one marker per signal and requests a force-flush. Success requires a new exporter-success delta for traces, logs, and metrics, no new per-signal failure or facade rejection, and a successful shared bounded flush. The host and Capsule daemon protocols expose typed sanitized health snapshots. Exporter acknowledgement confirms delivery to the configured OTLP endpoint, not downstream retention or indexing.
Schema authority
crates/jackin-telemetry/registry
Product instrumentation uses jackin-telemetry registered events, operation guards, metric handles, and ownership-aware spawn helpers. All legacy and generic telemetry macro call sites, definitions, and bridge machinery have been removed. A syntax-aware architecture gate rejects their reintroduction and constrains direct OpenTelemetry SDK use and arbitrary telemetry targets. Operator-facing prose remains a separate output port.
Identity and resources
Resources are stable for one process and identical across its three providers. They contain service identity and deployment-safe process attributes such as service.name, service.version, service.instance.id, and app.mode. Invocation, session, job, workspace, role, container, and other correlation values do not belong on the Resource.
The command process mints a cli.invocation.id. Interactive attachments mint a session.id, and detached work can use job.id. These identifiers are UUID correlation fields rather than backend-specific resource keys. Exhaustive dotted-command roots, reconnect linkage, detached-work identity, and correlation exclusion from Resource and metric dimensions are implemented and tested.
Internal launch handoff uses JACKIN_INVOCATION_ID and marks a network-safe Capsule endpoint with JACKIN_CAPSULE_OTLP_SAFE. They are process-boundary implementation variables, not operator configuration.
Trace model
The contract prohibits process-, console-, screen-, tab-, pane-, and session-lifetime spans. Bounded owners exist for launch stages, subprocess and provider boundaries, connections, cycles, RPCs, streams, and prewarm attempts. The active roadmap still requires integrated real-flow trace-shape, outcome, and privacy evidence across those owners.
RPC boundaries use CLIENT and SERVER span kinds, and detached scheduling uses producer/consumer links rather than false parentage. Joined, detached, cycle, stream, blocking, thread, and response-write helpers own bounded completion; the active audit still requires final integrated and source-policy evidence across the full executor matrix.
W3C context and versioned correlation fields are implemented across daemon, control, attach, jobs, and jackin-exec protocols. Focused tests cover sampling, malformed identifiers, response-write outcomes, peer closure, and payload privacy. The remaining acceptance work replaces helper-only evidence with representative serialized real-flow and concurrent-session conformance. Missing, malformed, unsupported, or disabled context creates a safe local root.
Events and metrics
The native event bridge emits OpenTelemetry log records with EventName, severity, and active trace/span correlation. Conformance proves one governed event produces one log record without a duplicate span event, with registry-wide canonical severity and field enforcement, exact Resource consistency, and matching trace/span correlation. INFO is for lifecycle and state transitions, WARN for handled degradation, ERROR for failed operations, DEBUG for diagnostic decisions, and TRACE for narrowly governed high-frequency detail. Default mode excludes DEBUG and TRACE.
The binding target is that every Err consumed or propagated at a semantic ownership boundary is recorded exactly once. Boundary-specific subprocess, HTTP, RPC, database, launch, and similar operation owners complete their span with a registered bounded error.type. ResultTelemetryExt::record_telemetry_error is available only when no more specific owner exists; it accepts any displayable Rust error type without formatting it during capture and returns a TelemetryError carrier that preserves the original source for operator reporting. Reapplying ownership to that carrier retains the first semantic owner and emits nothing again. The active audit is adding structural source-policy enforcement and a per-site census before claiming universal coverage. Automatic tracing-opentelemetry error inference stays disabled because it can duplicate ownership, poison a successful outer operation with a handled inner error, and export sensitive error text.
UI action causality, screen/focus lifecycle, agent and PTY/session state, attach controls, background cycles, and their low-cardinality metric families are implemented with focused exporter tests. Integrated host/launch/Capsule input-equivalence and lifecycle causality scenarios remain active acceptance work.
Registered metric handles, fixed views, exact 256-series cardinality caps, observable process/runtime health, exporter health, and second-line raw-SDK metric rejection are implemented and tested. Identifiers do not become metric dimensions unless the registry explicitly permits them. The active acceptance work is non-vacuous real-flow volume, soak, queue, memory, duration, and linkage evidence.
Privacy
Telemetry is allowlist-first. The registry permits semantic categories and bounded outcomes, not arbitrary product data. Do not emit:
- absolute paths, workspace or role names, container names, tab labels, or user-entered labels;
- raw argv, commands, environment values, URLs with queries, headers, credentials, tokens, or secrets;
- PTY bytes, rendered frames, keystrokes, clipboard content, prompts, model content, or mouse coordinates.
Generative-AI signals use standard semantic conventions only where content capture remains disabled. Secret scrubbing is defense in depth for operator output and explicit evidence flows; it does not make an unregistered telemetry field acceptable.
Runtime state and debugging
RunDiagnostics is a bounded in-memory progress/timing model consumed by current-run UI. It is not a sink, history store, or telemetry fallback.
--debug enables operator troubleshooting behavior. JACKIN_TELEMETRY_LEVEL and JACKIN_TELEMETRY_CATEGORIES govern registered telemetry detail independently. Neither mode creates a telemetry file. Use the backend and cli.invocation.id / session.id for history, and use Run Telemetry for operator setup.
Verification
The dev-only jackin-otlp-testbed
Run focused contract checks with:
cargo xtask telemetry-registry
cargo nextest run -p jackin-diagnostics -p jackin-otlp-testbed --all-features --locked -E 'test(/conformance/)'