Anil Kumar Bandrapalli
07/14/2022, 10:40 AMAnkit Nayan
2m
or 5m
histogram_quantile(0.99, sum(rate(signoz_latency_bucket{service_name="workflow-service", operation="/api/task/complete"}[5m])) by (le)) > 100
Anil Kumar Bandrapalli
07/14/2022, 11:22 AMAnkit Nayan
Anil Kumar Bandrapalli
07/14/2022, 1:15 PMAnkit Nayan
v0.9.2
and follow migration docs to do thatPriyansh
07/14/2022, 2:32 PMRahul Tiwari
07/15/2022, 5:59 AM-- -host=my-release-clickhouse -port=9000 -userName=admin -password=27ff0399-0d3a-4bd8-919d-17c2181e6fb9kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead. If you don't see a command prompt, try pressing enter. Writing samples to DB 2022/07/15 054409 Error while writing samples to DB code: 60, message: Table signoz_metrics.samples_v2 doesn't exist Session ended, resume using 'kubectl attach signoz-migrate-clickhouse-56767c457-sqpl2 -c signoz-migrate-clickhouse -i -t' command when the pod is running [ec2-user@ip-10-0-4-191 ~]$ [ec2-user@ip-10-0-4-191 ~]$ kubectl logs signoz-migrate-clickhouse-56767c457-kfmdj -n platform -f my-release-clickhouse 9000 admin 27ff0399-0d3a-4bd8-919d-17c2181e6fb9 signoz_metrics Total Rows: 63262424 There are total 63262424 samples rows, starting migration... Total Rows: 2555 There are total 2555 time series rows, starting migration... Writing samples to DB 2022/07/15 055808 Error while writing samples to DB code: 60, message: Table signoz_metrics.samples_v2 doesn't exist [ec2-user@ip-10-0-4-191 ~]$
Anil Kumar Bandrapalli
07/15/2022, 10:43 AMAnkit Nayan
Amol Umbark
07/15/2022, 11:44 AMAnil Kumar Bandrapalli
07/15/2022, 11:48 AMAmol Umbark
07/15/2022, 11:52 AMAnil Kumar Bandrapalli
07/15/2022, 12:27 PMAnkit Nayan
histogram_quantile(0.99, sum(rate(signoz_latency_bucket{service_name="workflow-service", operation="/api/task/complete"}[1m])) by (le))
Anil Kumar Bandrapalli
07/15/2022, 1:00 PMAnkit Nayan
[2m]
Anil Kumar Bandrapalli
07/15/2022, 1:06 PMAnkit Nayan
Anil Kumar Bandrapalli
07/15/2022, 1:10 PMAnkit Nayan
exec -it
into your clickhouse container and connect to db by running clickhouse client
inside the containerAnil Kumar Bandrapalli
07/15/2022, 1:12 PMAnkit Nayan
use signoz_metrics;
and
select * from time_series_v2 where metric_name='signoz_latency_bucket';
and try to search for rows which has workflow-service
and /api/task/complete
2m
time range..your alert won't workAnil Kumar Bandrapalli
07/15/2022, 1:40 PMAmol Umbark
07/15/2022, 1:42 PMclickhouse client --host localhost --port 9000
Ankit Nayan
Prashant Shahi
07/15/2022, 3:16 PMkubectl -n platform exec -i --tty pod/chi-signoz-cluster-0-0-0 -- bash
Followed by:
clickhouse-client
Anil Kumar Bandrapalli
07/15/2022, 4:09 PMAnkit Nayan
Anil Kumar Bandrapalli
07/15/2022, 4:38 PMAnkit Nayan
Anil Kumar Bandrapalli
07/15/2022, 5:08 PMAnkit Nayan
Rahul Tiwari
07/18/2022, 6:10 AMPrashant Shahi
07/18/2022, 6:27 AMVishal Sharma
07/18/2022, 6:33 AMRahul Tiwari
07/18/2022, 6:53 AMVishal Sharma
07/18/2022, 6:54 AMRahul Tiwari
07/18/2022, 6:57 AMVishal Sharma
07/18/2022, 6:58 AMkubectl -n platform delete pod signoz-migrate
Rahul Tiwari
07/18/2022, 7:01 AMAnil Kumar Bandrapalli
07/18/2022, 1:08 PMAmol Umbark
07/18/2022, 1:11 PMAnil Kumar Bandrapalli
07/18/2022, 1:13 PMAmol Umbark
07/18/2022, 1:14 PMAnil Kumar Bandrapalli
07/18/2022, 1:14 PMAmol Umbark
07/18/2022, 1:15 PMAnil Kumar Bandrapalli
07/18/2022, 1:15 PMAmol Umbark
07/18/2022, 1:17 PMAnil Kumar Bandrapalli
07/18/2022, 1:23 PMAmol Umbark
07/18/2022, 1:25 PMAnil Kumar Bandrapalli
07/18/2022, 1:27 PMAmol Umbark
07/18/2022, 1:27 PMAnil Kumar Bandrapalli
07/18/2022, 1:27 PMAmol Umbark
07/18/2022, 1:28 PMAnil Kumar Bandrapalli
07/18/2022, 1:32 PMAmol Umbark
07/18/2022, 1:37 PMAnil Kumar Bandrapalli
07/18/2022, 1:37 PMAnkit Nayan
histogram_quantile(0.99, sum(rate(signoz_latency_bucket{service_name="workflow-service", operation="POST /api/task/complete"}[1m])) by (le))
try changing the [1m]
to [5m]
. Does the chart plot now?Anil Kumar Bandrapalli
07/18/2022, 5:48 PMAnkit Nayan
Anil Kumar Bandrapalli
07/18/2022, 6:16 PMAnkit Nayan
Anil Kumar Bandrapalli
07/18/2022, 6:20 PMAnkit Nayan
Srikanth Chekuri
07/25/2022, 10:16 AM