https://signoz.io logo
#support
Title
# support
g

Garegin Tamrazyan

01/23/2024, 2:42 PM
has anybody have experience with Signoz ingesting 500 gb per day ? what hardware you will need for that ?
a

Alexei Zenin

01/23/2024, 4:24 PM
you'll definitely want to use a clustered clickhouse with a multi-disk setup where you keep N days on SSD and the rest in S3 for the tail data. Most queries I find are the last 5 minutes and rarely do people look at the last week or month
g

Garegin Tamrazyan

01/23/2024, 4:25 PM
sure, but is there a documentation for that ?
what kind of VMs can handle that ?
a

Alexei Zenin

01/23/2024, 4:29 PM
yeah there aren't extensive docs for this, the closest is this: https://signoz.io/docs/operate/clickhouse/distributed-clickhouse/
s

Srikanth Chekuri

01/24/2024, 3:28 AM
@Garegin Tamrazyan How long do you want to retain the data? Are you just interested in APM metrics or do you also want to retain the traces longer?
g

Garegin Tamrazyan

01/24/2024, 4:49 AM
just APM, , need to retain for shorter period
1 week maybe
s

Srikanth Chekuri

01/24/2024, 5:52 PM
@Garegin Tamrazyan so you want to just retain the APM metrics. So when you say 500 GB per day, will it just be APM or other data?
g

Garegin Tamrazyan

01/25/2024, 6:12 AM
APM
s

Srikanth Chekuri

01/25/2024, 10:09 AM
Can you share more details on how your APM data will be 500 GB? Like how many approximate timeseries and samples and How did you get the estimate of 500 GB?
g

Garegin Tamrazyan

01/25/2024, 10:10 AM
i see the data in our NewRelic
i cannot share anything more here,, applications are written in Ruby
s

Srikanth Chekuri

01/25/2024, 10:14 AM
Do you also use tracing NewRelic provides or is it just APM metrics?
g

Garegin Tamrazyan

01/25/2024, 10:14 AM
no tracing
just APM
s

Srikanth Chekuri

01/25/2024, 10:18 AM
Ok, I don't think you need a big resource if you just want to use APM metrics. Set up the SigNoz or decent machine with a disk capacity of 1TB (The APM metrics are stored in optimised storage format and storage should be very less) and 8 CPU machine with 16 GB RAM is a good start and then depending on the usage you can try to optimize. When you start sending the data make sure you get rid of the tracing pipeline https://github.com/SigNoz/signoz/blob/085cf34a49434e90f8ec50428a52017e14a985a3/deploy/docker/clickhouse-setup/otel-collector-config.yaml#L156-L159 so that you don't end up writing tracing data (You can always enable it back when needed).
g

Garegin Tamrazyan

01/25/2024, 10:19 AM
i don’t need distributed setup for Clickhouse
?
s

Srikanth Chekuri

01/25/2024, 10:21 AM
In general, you should scale vertically as much as possible so the above suggestion is good enough. You mentioned you only want APM metrics so I don't think you need distributed setup yet. I'd start with the above setup and iterate on it if needed.
g

Garegin Tamrazyan

01/25/2024, 10:22 AM
sure , thanks
@Srikanth Chekuri last question.. i got pushback from devs that with Signoz we cannot instrument Rails 4 ,, is that correct ?
s

Srikanth Chekuri

01/25/2024, 10:32 AM
Yes, the old EOL versions are not supported and 4 is really old.
g

Garegin Tamrazyan

01/25/2024, 10:32 AM
so we have no option as a workaround, right ?
s

Srikanth Chekuri

01/25/2024, 10:39 AM
Yeah, I am not familiar with any workaround for that.
5 Views