I m having a few problems with the otel collector not sure i

Question

I m having a few problems with the otel collector not sure if I should ask here or somewhere else since its not exactly what you guys do however I keep having this problem where there the otel collector will balloon in ram usage and will eat most of the ram and almost all of my swap I just check the my server and it using 6 5 gb of swap alone I killed it with htop and swap usage dropped back down to 190 mb However this is at best temporary has anyone else ran into this problem

Onur Yikilmazoglu · Answer

all have to show is this for now I ll post a more relevant screenshot when it gets worse again

Pranay · Answer

< U030EBNGG5Q> We have seen some users reporting this issue How long have you been running this otel collector for

Pranay · Answer

< U02SS3ZAMKQ> may also have more insights on this

Ankit Nayan · Answer

< U030EBNGG5Q> please change the memory limit of otel collector from

Ankit Nayan · Answer

and you can add similar limit to this instance too <https github com SigNoz signoz blob main deploy docker clickhouse setup docker compose yaml L92>

Onur Yikilmazoglu · Answer

< U01HPVB0D2S> I run it constantly about 15 20 at least basically

Onur Yikilmazoglu · Answer

< U01HWQ1RTC2> will try that

Onur Yikilmazoglu · Answer

I ve been more meticulously logging my ram usage and this is still happening the otel collector and the metrics collector are ballooning in size over the span of hours you guys got any idea why

Pranay · Answer

thanks for the detailed report < U030EBNGG5Q> We will need to investigate this further Can you also create an issue for this so that this doesn t get lost in slack threads Would be very helpful

Ankit Nayan · Answer

< U030EBNGG5Q> how many spans hr or spans day are you ingesting You can see this at `Usage Explorer` menu item

Ankit Nayan · Answer

Also are you using metrics too

Onur Yikilmazoglu · Answer

< U01HWQ1RTC2> we peak at about 170k h on an average day and yes we also use metrics

Onur Yikilmazoglu · Answer

< U01HPVB0D2S> will do

Onur Yikilmazoglu · Answer

I m thinking of putting a limit on swap so it doesn t go out of control

Onur Yikilmazoglu · Answer

oh man you guys are going to make fun of me for this one turns out the machine Im using didn t have swap memory limits enabled for cgroups

Onur Yikilmazoglu · Answer

I m crossing my fingers this sorts its out if not Im going to have to open an issue on github

Pranay · Answer

stuck out tongue

Pranay · Answer

Irrespective please open an issue wink Will be good reference for users in future

Onur Yikilmazoglu · Answer

sure will do