Slackbot
07/14/2022, 10:40 AMAnkit Nayan
2m
or 5m
Ankit Nayan
histogram_quantile(0.99, sum(rate(signoz_latency_bucket{service_name="workflow-service", operation="/api/task/complete"}[5m])) by (le)) > 100
Anil Kumar Bandrapalli
07/14/2022, 11:22 AMAnil Kumar Bandrapalli
07/14/2022, 11:23 AMAnil Kumar Bandrapalli
07/14/2022, 12:03 PMAnil Kumar Bandrapalli
07/14/2022, 12:37 PMAnkit Nayan
Ankit Nayan
Anil Kumar Bandrapalli
07/14/2022, 1:15 PMAnil Kumar Bandrapalli
07/14/2022, 1:16 PMAnkit Nayan
v0.9.2
and follow migration docs to do thatAnkit Nayan
Ankit Nayan
Ankit Nayan
Priyansh
07/14/2022, 2:32 PMRahul Tiwari
07/15/2022, 5:59 AMRahul Tiwari
07/15/2022, 5:59 AM-- -host=my-release-clickhouse -port=9000 -userName=admin -password=27ff0399-0d3a-4bd8-919d-17c2181e6fb9kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead. If you don't see a command prompt, try pressing enter. Writing samples to DB 2022/07/15 054409 Error while writing samples to DB code: 60, message: Table signoz_metrics.samples_v2 doesn't exist Session ended, resume using 'kubectl attach signoz-migrate-clickhouse-56767c457-sqpl2 -c signoz-migrate-clickhouse -i -t' command when the pod is running [ec2-user@ip-10-0-4-191 ~]$ [ec2-user@ip-10-0-4-191 ~]$ kubectl logs signoz-migrate-clickhouse-56767c457-kfmdj -n platform -f my-release-clickhouse 9000 admin 27ff0399-0d3a-4bd8-919d-17c2181e6fb9 signoz_metrics Total Rows: 63262424 There are total 63262424 samples rows, starting migration... Total Rows: 2555 There are total 2555 time series rows, starting migration... Writing samples to DB 2022/07/15 055808 Error while writing samples to DB code: 60, message: Table signoz_metrics.samples_v2 doesn't exist [ec2-user@ip-10-0-4-191 ~]$
Rahul Tiwari
07/15/2022, 7:23 AMAnil Kumar Bandrapalli
07/15/2022, 10:43 AMAnil Kumar Bandrapalli
07/15/2022, 10:43 AMAnkit Nayan
Amol Umbark
07/15/2022, 11:44 AMAnil Kumar Bandrapalli
07/15/2022, 11:48 AMAnil Kumar Bandrapalli
07/15/2022, 11:50 AMAmol Umbark
07/15/2022, 11:52 AMAnil Kumar Bandrapalli
07/15/2022, 12:27 PMAnil Kumar Bandrapalli
07/15/2022, 12:31 PMAnil Kumar Bandrapalli
07/15/2022, 12:32 PMAnil Kumar Bandrapalli
07/15/2022, 12:37 PMAnkit Nayan
histogram_quantile(0.99, sum(rate(signoz_latency_bucket{service_name="workflow-service", operation="/api/task/complete"}[1m])) by (le))
Anil Kumar Bandrapalli
07/15/2022, 1:00 PMAnkit Nayan
[2m]
Ankit Nayan
Anil Kumar Bandrapalli
07/15/2022, 1:06 PMAnkit Nayan
Anil Kumar Bandrapalli
07/15/2022, 1:10 PMAnkit Nayan
exec -it
into your clickhouse container and connect to db by running clickhouse client
inside the containerAnil Kumar Bandrapalli
07/15/2022, 1:12 PMAnkit Nayan
use signoz_metrics;
and
select * from time_series_v2 where metric_name='signoz_latency_bucket';
and try to search for rows which has workflow-service
and /api/task/complete
Ankit Nayan
2m
time range..your alert won't workAnil Kumar Bandrapalli
07/15/2022, 1:40 PMAnil Kumar Bandrapalli
07/15/2022, 1:40 PMAnil Kumar Bandrapalli
07/15/2022, 1:41 PMAmol Umbark
07/15/2022, 1:42 PMclickhouse client --host localhost --port 9000
Ankit Nayan
Prashant Shahi
07/15/2022, 3:16 PMkubectl -n platform exec -i --tty pod/chi-signoz-cluster-0-0-0 -- bash
Followed by:
clickhouse-client
Anil Kumar Bandrapalli
07/15/2022, 4:09 PMAnil Kumar Bandrapalli
07/15/2022, 4:10 PMAnkit Nayan
Anil Kumar Bandrapalli
07/15/2022, 4:38 PMAnkit Nayan
Anil Kumar Bandrapalli
07/15/2022, 5:08 PMAnkit Nayan
Rahul Tiwari
07/18/2022, 6:10 AMRahul Tiwari
07/18/2022, 6:11 AMPrashant Shahi
07/18/2022, 6:27 AMPrashant Shahi
07/18/2022, 6:29 AMVishal Sharma
07/18/2022, 6:33 AMRahul Tiwari
07/18/2022, 6:53 AMVishal Sharma
07/18/2022, 6:54 AMRahul Tiwari
07/18/2022, 6:57 AMRahul Tiwari
07/18/2022, 6:58 AMVishal Sharma
07/18/2022, 6:58 AMkubectl -n platform delete pod signoz-migrate
Rahul Tiwari
07/18/2022, 7:01 AMRahul Tiwari
07/18/2022, 7:02 AMRahul Tiwari
07/18/2022, 12:00 PMAnil Kumar Bandrapalli
07/18/2022, 1:08 PMAmol Umbark
07/18/2022, 1:11 PMAnil Kumar Bandrapalli
07/18/2022, 1:13 PMAmol Umbark
07/18/2022, 1:14 PMAmol Umbark
07/18/2022, 1:14 PMAnil Kumar Bandrapalli
07/18/2022, 1:14 PMAmol Umbark
07/18/2022, 1:15 PMAnil Kumar Bandrapalli
07/18/2022, 1:15 PMAnil Kumar Bandrapalli
07/18/2022, 1:16 PMAmol Umbark
07/18/2022, 1:17 PMAmol Umbark
07/18/2022, 1:19 PMAnil Kumar Bandrapalli
07/18/2022, 1:23 PMAnil Kumar Bandrapalli
07/18/2022, 1:23 PMAnil Kumar Bandrapalli
07/18/2022, 1:24 PMAmol Umbark
07/18/2022, 1:25 PMAmol Umbark
07/18/2022, 1:25 PMAnil Kumar Bandrapalli
07/18/2022, 1:27 PMAmol Umbark
07/18/2022, 1:27 PMAnil Kumar Bandrapalli
07/18/2022, 1:27 PMAmol Umbark
07/18/2022, 1:28 PMAmol Umbark
07/18/2022, 1:28 PMAmol Umbark
07/18/2022, 1:29 PMAnil Kumar Bandrapalli
07/18/2022, 1:32 PMAnil Kumar Bandrapalli
07/18/2022, 1:33 PMAmol Umbark
07/18/2022, 1:37 PMAnil Kumar Bandrapalli
07/18/2022, 1:37 PMAnkit Nayan
histogram_quantile(0.99, sum(rate(signoz_latency_bucket{service_name="workflow-service", operation="POST /api/task/complete"}[1m])) by (le))
try changing the [1m]
to [5m]
. Does the chart plot now?Anil Kumar Bandrapalli
07/18/2022, 5:48 PMAnkit Nayan
Anil Kumar Bandrapalli
07/18/2022, 6:16 PMAnkit Nayan
Anil Kumar Bandrapalli
07/18/2022, 6:20 PMAnil Kumar Bandrapalli
07/18/2022, 6:21 PMAnil Kumar Bandrapalli
07/25/2022, 9:11 AMAnkit Nayan
Srikanth Chekuri
07/25/2022, 10:16 AM