# Traces in previews

> Get your app's traces into a preview's Traces tab: what Stagd sets, and the least it takes in Node.js, Python, Go and Java.

With `otel` in `stagd.yaml`, each preview gets its own OpenTelemetry endpoint, and the preview's **Traces** tab and `stagd preview traces` show what arrives. Stagd points your app at the endpoint, but it can't make your app send: **nothing arrives until your app has an OpenTelemetry SDK installed and started.** This page is the least it takes, per language.

```yaml
services:
  otel: {}
```

## What Stagd sets

Your app's own services (those with a `build:` in the compose file, or under [`build`](https://stagd.dev/docs/reference/stagd-yaml.md#build)) get the standard variables every OpenTelemetry SDK reads. There's no endpoint or service name to configure.

| Variable | Value |
|---|---|
| `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://stagd-otel:4318` |
| `OTEL_EXPORTER_OTLP_PROTOCOL` | `http/protobuf` |
| `OTEL_SERVICE_NAME` | The compose service's name |
| `OTEL_RESOURCE_ATTRIBUTES` | `deployment.environment.name=preview`, the preview's id, the repository and the commit |
| `OTEL_TRACES_SAMPLER` | `always_on`: a preview has little traffic, so every trace is kept |

A value your compose file sets wins over Stagd's.

The traces stay in the preview: it keeps the latest 20,000 in memory, and they go when the preview sleeps or goes away. Stagd never stores them. To keep a copy in your own backend (Dash0, Honeycomb, Grafana Cloud), add [`export`](https://stagd.dev/docs/reference/stagd-yaml.md#otel). OpenTelemetry logs and metrics are only sent on with `export`; without it they're taken and dropped, so SDKs that send them by default don't fail. To leave out health checks and other noise, list their paths under [`ignore`](https://stagd.dev/docs/reference/stagd-yaml.md#otel).

## Node.js and TypeScript

Install the SDK with every instrumentation it has:

```sh
npm install @opentelemetry/api @opentelemetry/auto-instrumentations-node
```

Start it before your app loads, with no code changes:

```sh
node --require @opentelemetry/auto-instrumentations-node/register server.js
```

Or, when you can't change the start command, in the compose file or the Dockerfile:

```yaml
environment:
  NODE_OPTIONS: --require @opentelemetry/auto-instrumentations-node/register
```

**ES modules** (`"type": "module"`, `.mjs`, or TypeScript compiled to ESM) need the loader hook as well, or only Node's own `http` is traced, without the route:

```yaml
environment:
  NODE_OPTIONS: --experimental-loader=@opentelemetry/instrumentation/hook.mjs --import @opentelemetry/auto-instrumentations-node/register
```

**What you'll see:** the incoming request as `GET /users/:id`, Express or Fastify middleware and handlers, outgoing `fetch` and `http` calls, and queries through `pg`, `mysql2`, `mongodb`, `ioredis` and most other drivers.

**Optional.** A span of your own, for work that isn't a request:

```js
import { trace } from "@opentelemetry/api";

const tracer = trace.getTracer("web");

await tracer.startActiveSpan("price the cart", async (span) => {
  span.setAttribute("cart.items", items.length);
  // …
  span.end();
});
```

`pino`, `winston` and `bunyan` lines get `trace_id` and `span_id` without more work, so on the Logs tab they link to their trace.

A warning, `MetadataLookupWarning`, comes from the Google Cloud detector looking for a metadata server. `OTEL_NODE_RESOURCE_DETECTORS=env,host,os` quiets it.

## Python

Install the distro and the exporter, then let `opentelemetry-bootstrap` add the instrumentations for the libraries you have. Run it after your own dependencies are installed:

```sh
pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install
```

Start your app through `opentelemetry-instrument`, with no code changes:

```dockerfile
CMD ["opentelemetry-instrument", "python", "app.py"]
```

It works in front of `gunicorn`, `uvicorn` and `flask run` as well: `opentelemetry-instrument gunicorn app:app`.

**What you'll see:** the incoming request as `GET /users/<id>` (Flask, Django, FastAPI and most others), outgoing calls through `requests`, `httpx` and `urllib3`, and queries through `psycopg`, `SQLAlchemy`, `redis` and most other drivers.

**Optional.** A span of your own:

```python
from opentelemetry import trace

tracer = trace.get_tracer("api")

with tracer.start_as_current_span("load the user") as span:
    span.set_attribute("user.id", user_id)
```

For log lines that link to their trace, set `OTEL_PYTHON_LOG_CORRELATION=true`: each log record then carries `otelTraceID`. The Logs tab reads JSON and logfmt lines, so write them as JSON:

```python
import json
import logging

class JsonLines(logging.Formatter):
    def format(self, record):
        return json.dumps({
            "level": record.levelname,
            "msg": record.getMessage(),
            "trace_id": getattr(record, "otelTraceID", ""),
        })

handler = logging.StreamHandler()
handler.setFormatter(JsonLines())
logging.basicConfig(level=logging.INFO, handlers=[handler], force=True)
```

## Go

Go has no auto-instrumentation: you start the SDK in `main` and wrap your handler and client. The current modules need Go 1.26 or later.

```sh
go get go.opentelemetry.io/otel go.opentelemetry.io/otel/sdk \
  go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp \
  go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
```

The exporter and the SDK read the endpoint, the service name and the attributes from the environment, so the setup is short:

```go
// otel.go
package main

import (
	"context"

	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
	"go.opentelemetry.io/otel/propagation"
	sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

// startTracing sends traces where OTEL_EXPORTER_OTLP_ENDPOINT says, named by
// OTEL_SERVICE_NAME. Call the function it returns before the program exits.
func startTracing(ctx context.Context) (func(context.Context) error, error) {
	exporter, err := otlptracehttp.New(ctx)
	if err != nil {
		return nil, err
	}

	provider := sdktrace.NewTracerProvider(sdktrace.WithBatcher(exporter))

	otel.SetTracerProvider(provider)
	otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagation.TraceContext{}, propagation.Baggage{}))

	return provider.Shutdown, nil
}
```

Then, in `main`:

```go
stop, err := startTracing(ctx)
if err != nil {
	log.Fatal(err)
}

defer stop(ctx)

// Incoming requests: one span each, named by the route.
http.ListenAndServe(":8080", otelhttp.NewHandler(mux, "server"))

// Outgoing calls: pass the request's context to them.
client := &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
```

Use `otlptracehttp`, not `otlptracegrpc`: Stagd's endpoint is the HTTP one.

**What you'll see:** the incoming request as `GET /users/{id}` and each outgoing call. For queries, wrap the driver with [`otelsql`](https://github.com/XSAM/otelsql) or, for pgx, [`otelpgx`](https://github.com/exaring/otelpgx).

**Optional.** A span of your own, and a log line that links to its trace:

```go
ctx, span := otel.Tracer("api").Start(r.Context(), "load the user")
defer span.End()

slog.InfoContext(ctx, "loaded user", "trace_id", trace.SpanContextFromContext(ctx).TraceID().String())
```

With `slog.NewJSONHandler` (or any logfmt or JSON logger) the Logs tab reads `trace_id` and links the line.

## Java and Kotlin

The OpenTelemetry Java agent instruments the app as it loads, with no code changes. Add it to the image:

```dockerfile
ADD https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar /otel/opentelemetry-javaagent.jar
ENV JAVA_TOOL_OPTIONS="-javaagent:/otel/opentelemetry-javaagent.jar"
```

`JAVA_TOOL_OPTIONS` reaches every JVM the container starts, so it works whatever runs your app: `java -jar`, Gradle, Spring Boot's launcher. Pin a version instead of `latest` once it works.

**What you'll see:** the incoming request with its route (Spring, Ktor, Micronaut, Quarkus, Servlet containers, the JDK's own `HttpServer`), outgoing calls through the JDK's `HttpClient`, OkHttp and Apache HttpClient, and JDBC, R2DBC, Redis and Kafka.

**Optional.** For a span of your own, add `io.opentelemetry.instrumentation:opentelemetry-instrumentation-annotations` and mark a method:

```java
@WithSpan("load the user")
User load(@SpanAttribute("user.id") String id) { … }
```

The agent puts `trace_id` and `span_id` in the logging MDC for Logback and Log4j. A JSON layout that writes the MDC, such as `logstash-logback-encoder`, makes each line link to its trace.

## Other languages

.NET, Ruby, PHP, Rust and the rest work the same way: install the SDK and the OTLP exporter over HTTP, and start it before the app. Each reads the variables above, so leave the endpoint and service name out of your code. [OpenTelemetry's docs](https://opentelemetry.io/docs/languages/) have the steps per language.

## If nothing shows up

Open the preview to make some requests first: traces only come from traffic. Then:

- **The SDK starts too late.** It has to load before your web framework and drivers, or it can't patch them. With Node.js, use `--require` or `NODE_OPTIONS`, not an `import` halfway down your entry file; with ES modules, add the loader hook.
- **The compose file sets its own endpoint.** A value there wins over Stagd's, so `OTEL_EXPORTER_OTLP_ENDPOINT: http://localhost:4318` from local development sends nowhere. Leave it out of the compose file, and set it where you run locally instead, in your shell or a `.env` file.
- **The exporter speaks gRPC.** Stagd's endpoint is OTLP over HTTP on 4318. An exporter that only speaks gRPC (Go's `otlptracegrpc`, or `OTEL_EXPORTER_OTLP_PROTOCOL=grpc` from the compose file) needs `http://stagd-otel:4317` instead.
- **The service isn't built from the repository.** A compose service with only `image:` is left alone and gets none of the variables. Give it the ones above in the compose file.
- **The traces are filtered out.** Check `ignore` in `stagd.yaml`, and the Traces tab's filters and time range.

`stagd preview logs <id>` shows what the SDK says as it starts; most print a line when they can't reach the endpoint.
