Title
m
Muhammed Hussein Karimi
02/28/2023, 9:17 AMHi,
After adding json parser to my logs in
k8s-infralogsa
Ankit Nayan
02/28/2023, 9:25 AMcc: @nitya-signoz
n
nitya-signoz
02/28/2023, 9:39 AMCan you share the configuration?
Also if your fields are loaded but not visible in the UI there might be issues with timestamp.
Can you run the commands and share output
kubectl exec -n platform -it chi-my-release-clickhouse-cluster-0-0-0 -- sh
clickhouse client
use signoz_logs;
select timestamp,observed_timestamp from distributed_logs order by timestamp desc limit 10;m
Muhammed Hussein Karimi
02/28/2023, 9:41 AM┌───────────timestamp─┬──observed_timestamp─┐
│ 1677577033101391005 │ 1677577033114197754 │
│ 1677577033087210293 │ 1677577033114149167 │
│ 1677577033081940586 │ 1677577033113942963 │
│ 1677577033079232086 │ 1677577033114117714 │
│ 1677577033074985781 │ 1677577033114088974 │
│ 1677577033071399878 │ 1677577033113746096 │
│ 1677577033068867909 │ 1677577033114495821 │
│ 1677577033058288277 │ 1677577033114040042 │
│ 1677577033054256687 │ 1677577033114446222 │
│ 1677577033036741379 │ 1677577033114394174 │
└─────────────────────┴─────────────────────┘image.png
here you go
n
nitya-signoz
02/28/2023, 9:43 AMinteresting, can you share the API response of
/logsAlong with the request params ^
m
Muhammed Hussein Karimi
02/28/2023, 9:49 AMThere is no request with
/logsaggregaten
nitya-signoz
02/28/2023, 9:50 AMWhat version of SigNoz are you using?
@Palash Gupta can there be a case where /logs is not triggered on the UI ?
m
Muhammed Hussein Karimi
02/28/2023, 9:51 AMv0.16.0
Oh found some after a page refresh
{
"status": "error",
"data": "Failed to fetch logs from the DB",
"errorType": "internal",
"error": "code: 241, message: Memory limit (total) exceeded: would use 3.62 GiB (attempt to allocate chunk of 5064994 bytes), maximum: 3.60 GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker.: (while reading column attributes_string_value): While executing MergeTreeReverse"
}here is the clickhouse resources
resources:
requests:
cpu: 100m
memory: 200Mi
limits:
cpu: 2000m
memory: 4GiI should increase resources?
or there are other settings?
n
nitya-signoz
02/28/2023, 9:56 AMYeah please increase the resources and try it out first. There are issues about this on clickhouse https://github.com/ClickHouse/ClickHouse/issues/17631 . But you can start by doubling up the resources and trying it out.
p
Palash Gupta
02/28/2023, 10:34 AMcan there be a case where /logs is not triggered on the UI ?I am not aware of such case... @Muhammed Hussein Karimi can you please share the loom video for the same
m
Muhammed Hussein Karimi
02/28/2023, 10:45 AMFixed by increasing the memory limits
p
Palash Gupta
02/28/2023, 10:48 AMoh great ... @nitya-signoz can we update same in the docs