Hi, testing on v0.61.0, but the environments dropdown is not getting populated. I tried filling all possible attributes. Anyone know what I’m doing wrong?

Are you continuously sending data? For the first screenshot trace, what was the trace creation time?

Not yet, this was a test from a local dev environment. The timestamp is about 15minutes before the screenshot, with one restart of the entire SigNoz stack in between. Must the data be sent continously? If yes; for how long?

No, there should be data in the last two days at least so 15 minutes before is fine. I am not sure what might be the issue. Do you see any errors in query-service logs?

Not really, the only ERRORs in the log are unrelated as far as I can tell:

signoz-query-service  | {"level":"ERROR","timestamp":"2024-12-06T14:36:25.869Z","caller":"clickhouseReader/reader.go:4064","msg":"error while reading result","error":"context canceled","stacktrace":"<http://go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError|go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError>\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4064\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.readRowsForTimeSeriesResult\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:3918\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.(*ClickHouseReader).GetTimeSeriesResultV3\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4001\ngo.signoz.io/signoz/pkg/query-service/app/querier/v2.(*querier).execClickHouseQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/v2/querier.go:117\ngo.signoz.io/signoz/pkg/query-service/app/querier/v2.(*querier).runBuilderQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/v2/helper.go:249"}
signoz-query-service  | {"level":"ERROR","timestamp":"2024-12-06T14:36:25.870Z","caller":"clickhouseReader/reader.go:4064","msg":"error while reading result","error":"context canceled","stacktrace":"<http://go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError|go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError>\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4064\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.readRowsForTimeSeriesResult\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:3918\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.(*ClickHouseReader).GetTimeSeriesResultV3\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4001\ngo.signoz.io/signoz/pkg/query-service/app/querier/v2.(*querier).execClickHouseQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/v2/querier.go:117\ngo.signoz.io/signoz/pkg/query-service/app/querier/v2.(*querier).runBuilderQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/v2/helper.go:249"}
signoz-query-service  | {"level":"ERROR","timestamp":"2024-12-06T14:36:25.871Z","caller":"clickhouseReader/reader.go:4064","msg":"error while reading result","error":"context canceled","stacktrace":"<http://go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError|go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError>\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4064\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.readRowsForTimeSeriesResult\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:3918\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.(*ClickHouseReader).GetTimeSeriesResultV3\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4001\ngo.signoz.io/signoz/pkg/query-service/app/querier/v2.(*querier).execClickHouseQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/v2/querier.go:117\ngo.signoz.io/signoz/pkg/query-service/app/querier/v2.(*querier).runBuilderQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/v2/helper.go:249"}

No, not needed, when you land on the services page and look at the n/w requests and query-service logs and see if there is any error

No errors:

signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.271Z","caller":"app/server.go:400","msg":"/api/v1/featureFlags","timeTaken":0,"path":"/api/v1/featureFlags"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.278Z","caller":"app/server.go:400","msg":"/api/v1/login","timeTaken":0,"path":"/api/v1/login"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.353Z","caller":"app/server.go:400","msg":"/api/v1/user/{id}","timeTaken":0,"path":"/api/v1/user/{id}"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.453Z","caller":"app/server.go:400","msg":"/api/v1/org/preferences","timeTaken":0,"path":"/api/v1/org/preferences"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.453Z","caller":"app/server.go:400","msg":"/api/v2/licenses","timeTaken":0,"path":"/api/v2/licenses"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.453Z","caller":"app/server.go:400","msg":"/api/v3/licenses/active","timeTaken":0,"path":"/api/v3/licenses/active"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.453Z","caller":"app/server.go:400","msg":"/api/v1/featureFlags","timeTaken":0,"path":"/api/v1/featureFlags"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.454Z","caller":"app/server.go:400","msg":"/api/v1/orgUsers/{id}","timeTaken":0,"path":"/api/v1/orgUsers/{id}"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.712Z","caller":"app/server.go:400","msg":"/api/v2/licenses","timeTaken":0,"path":"/api/v2/licenses"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.728Z","caller":"app/server.go:400","msg":"/api/v1/version","timeTaken":0,"path":"/api/v1/version"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.904Z","caller":"app/server.go:400","msg":"/api/v2/licenses","timeTaken":0,"path":"/api/v2/licenses"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.910Z","caller":"app/server.go:400","msg":"/api/v3/autocomplete/attribute_keys","timeTaken":7,"path":"/api/v3/autocomplete/attribute_keys"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:39.922Z","caller":"app/server.go:400","msg":"/api/v1/services","timeTaken":17,"path":"/api/v1/services"}
signoz-query-service  | {"level":"INFO","timestamp":"2024-12-08T08:25:40.004Z","caller":"app/server.go:400","msg":"/api/v1/event","timeTaken":0,"path":"/api/v1/event"}

There should be one request with

api/v3/autocomplete/attribute_values

The request to

api/v3/autocomplete/attribute_keys?aggregateOperator=rate&searchText=resource_deployment_environment&dataSource=metrics&aggregateAttribute=signoz_calls_total

returns:

{
  "status": "success",
  "data": {
    "attributeKeys": null
  }
}

It should be part of resource attributes not the span attributes. How are you setting

deployment.environment

?

Inside the OTEL collector of the deployment:

processors:
  batch:
  attributes:
    actions:
      - key: "deployment.environment.name"
        value: development
        action: upsert
      - key: "deployment.environment"
        value: development
        action: upsert
      - key: "service.namespace"
        value: development
        action: upsert
      - key: "environment"
        value: development
        action: upsert

service:
  extensions: [health_check, pprof, zpages]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, attributes]
      exporters: [otlphttp]
    metrics:
      receivers: [otlp]
      processors: [batch, attributes]
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: [batch, attributes]
      exporters: [otlphttp]

You should use resource processor and

deployment.environment

should go in resource attributes.

💡 Ahaaa