https://signoz.io
Join Slack
This message was deleted.
# support
s
This message was deleted.
s
That should not be the case. Just to double check did you try increasing the time range of services tab and still didn't see it in the table?
s
No. The spans keep coming just fine. But I dont see the service name in that tab. Even if I increase the time range a lot
s
@Srikanth Chekuri In response of the api - 
/api/v1/getSpanFilters
serviceName
key has a service with a count as the value Where as in the response of the api - 
/api/v1/services
a particular service is not there in the list. What could be the reason?
s
There might be some issues with the specific service operation/span names. Can you a look at the query-service logs when you load the Services tab and see if there is any query error?
s
Couldn’t find any error log. There are only info logs.
I ran this query - 
select distinct serviceName from distributed_signoz_index_v2 where toDate(timestamp) > now() - interval 1 DAY format CSV;
This query’s output has that service name.
on further investigation - 
select count( distinct name) from distributed_top_level_operations;
has roughly 7.87 lacs entries.
Is that the reason that some services are getting clipped while running the query - 
SELECT DISTINCT name, serviceName FROM distributed_top_level_operations
Also I realized that there is ServiceSkipConfig. Where can I find that?
s
Is that the reason that some services are getting clipped while running the query -
Yes, the large number of entries indicates something wrong with your instrumentation. The entry is based on the span name. Make sure your span names are not very high cardinal https://github.com/open-telemetry/opentelemetry-specification/blob/v1.26.0/specification/trace/api.md#span. The 
ServiceSkipConfig
won't be helpful if you have so many entries to ignore.
s
@Srikanth Chekuri - Figured out the problem with high cardinality. Is there a way to delete already ingested operations in clickhouse?
s
Truncate the table 
distributed_top_level_operations
s
Thanks @Srikanth Chekuri
@Srikanth Chekuri - One last question, after truncating the table - how soon this table will get populated again?
s
Should start fill up very quickly.
s
ok thanks @Srikanth Chekuri
38 Views
Open in Slack
PreviousNext
Powered by

SigNoz Community

SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.

Powered by