Greetings, I’d like to ask about the `Database Cal...
# support
s
Greetings, I’d like to ask about the
Database Calls
tab having no data despite there are traces that are related to PostgreSQL queries (see 2nd screenshot) Is there any extra steps or documentations that I missed that I needed to take to gain visibility on these database metrics? I am currently using a java auto instrumentation from OpenTelemetry 1.13.1 w/ the startup settings as follows
Copy code
-Dotel.metrics.exporter=none -Dotel.exporter.otlp.endpoint=<collector endpoint of signoz>:4317 -Dotel.resource.attributes=service.name=servicenamesomething -javaagent:/path/to/otel_agent.jar
Currently using Signoz installed from a helm chart with the collector version
0.43.0-0.1
Thanks in advance!
s
Do you see
db.system
in the trace attributes for the db calls?
s
Yes, there is
db.system
with the value
postgresql
s
Does you time range for db calls cover the db trace timestamps range?
s
Yes, I have also tried to set it until 1 week ago, still no results unfortunately
s
Interesting
I am trying to think what could be the reason
since you can clearly see the data in trace.
s
Yes, tracing works normal to me Okay then, thanks, let me know anytime if you need more information!
s
@Steven Sim Can you help with some troubleshooting here. Can you access the clickhose and share the result of this query /
Copy code
SELECT * from signoz_metrics.time_series WHERE JSONExtractString(labels, '__name__') = 'signoz_db_latency_count' LIMIT 10;
s
Yes, I’d be glad if I can help This are the results, I found no result w/ the
WHERE
statement
might not be related but I’d like to also remind that I’m using
-Dotel.metrics.exporter=none
on my java command as written in the installation guide
s
Yeah, it's unrelated
Can you share what does this give?
Copy code
SELECT DISTINCT(JSONExtractString(labels, '__name__')) from signoz_metrics.time_series WHERE like(JSONExtractString(labels, '__name__'), 'signoz_%');
s
also empty unfortunately
s
Ok there is no data at all? This is getting mysterious.
s
yes, no data at all, there’s data if I remove the
WHERE
statement though
s
What data do you see? Would you be able to share the some metric names?
s
Here’s if I were to remove the
WHERE
statement second if I were to use
*
, all of the
__name____
starts with
system
s
Thanks, there are no metrics generated from the trace data which are essential for the
Database Calls
tab. This is probably a bug
I am going to try reproduce and see what's happening . Thank you.
s
Ok thanks!, let me know if you have any updates on this topic
s
@Steven Sim Can you share the environment details to reproduce this? That would be helpful.
s
I’m using Signoz installed on a kubernetes cluster w/ the current latest helm chart Java version 8 on AWS EC2, postgres 13.4 on AWS RDS, Opentelemetry auto instrumentation jar v1.13.1
s
Alright, that helps. Will look into this.
1
s
Hi @Srikanth Chekuri , sorry to bother you again but Is there any updates on this topic?, and if not yet is there a Github issue already that I can track for the upcoming time?
s
@Steven Sim Spent some time debugging, this error shows up inconsistently. Do you not see data for just after one request or no data at all? . When I was debugging It was observed this is an issue if I just make one to two requests but If I have an app running with near real work load I was able to see them although some of them were getting dropped as mentioned above.
s
In my case there was no data at all, I installed it on 3 servers running the same application and the workload was quite medium in my opinion Maybe is there a built-in sampling for these DB traces?
s
No there is no sampling as of now.
How often does your workload involve sql queries?
s
I’m quite sure at least 95% of the workload has something to do with postgres
s
Will you be able to share the logs for your collector instance?
s
Sure, I’ll get back to you soon