Hi Team, I'm having issue with sending logs from l...
# supportc
Chủ Huỳnh Minh
10/10/2022, 5:02 AMHi Team, I'm having issue with sending logs from logstash to signoz.
I followed this guide: https://signoz.io/docs/userguide/logstash_to_signoz/ but in signoz dashboard there's still no logs at all.
Chủ Huỳnh Minh
10/10/2022, 5:07 AMHere's all my configurations:
• otel-collector-config.yaml
receivers:tcplog/logstash:max_log_size: 1MiBlisten_address: "0.0.0.0:2255"attributes: {}resource: {}add_attributes: falseoperators: []service:...logs:receivers: [otlp, filelog/dockercontainers, tcplog/logstash]processors: [ batch ]exporters: [ clickhouselogsexporter ]output {tcp {codec => json_lines # this is required otherwise it will send eveything in a single linehost => "my-otelcollector-host-private-IP"port => 2255}}p
Pranay
10/10/2022, 6:13 AM
@nitya-signoz do you have insights on this
n
nitya-signoz
10/10/2022, 6:15 AMHi, @Chủ Huỳnh Minh can you share if you are able to see any error logs in otel collector or logstash?
c
Chủ Huỳnh Minh
10/10/2022, 10:19 AMHi @nitya-signoz. I only see error logs in logstash. Its something like this:
`tcp output exception {:host=>"10.100.6.91", :port=>54525, :exception=>#<EOFError: End of file reached>, :backtrace=>["org/jruby/RubyIO.java3021in `sysread'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-tcp-6.0.2/lib/logstash/outputs/tcp.rb161in `block in register'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-json_lines-3.1.0/lib/logstash/codecs/json_lines.rb67in `encode'", "/usr/share/logstash/logstash-core/lib/logstash/codecs/delegator.rb48in `block in encode'", "org/logstash/instrument/metrics/AbstractSimpleMetricExt.java65in `time'", "org/logstash/instrument/metrics/AbstractNamespacedMetricExt.java64in `time'", "/usr/share/logstash/logstash-core/lib/logstash/codecs/delegator.rb47in `encode'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-tcp-6.0.2/lib/logstash/outputs/tcp.rb209in `receive'", "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb105in `block in multi_receive'", "org/jruby/RubyArray.java1821in `each'", "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb105in `multi_receive'", "org/logstash/config/ir/compiler/OutputStrategyExt.java143in `multi_receive'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java121in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb300in
block in start_workers'"]}Chủ Huỳnh Minh
10/10/2022, 10:20 AMMy input format in case you want to know:
{"@timestamp":"$time_iso8601","hostname":"$hostname","source":"$server_addr","client":"$remote_addr",..."request_length":$request_length,"protocol":"$server_protocol","upstreamhost":"$upstream_addr",}Chủ Huỳnh Minh
10/10/2022, 10:24 AMI also used netstat and telnet to verify that the logstash receiver is listening on port 2255 as well so i don't think its firewall related
n
nitya-signoz
10/10/2022, 10:27 AMCan you share your logstash version so that I can try to replicate this issue, once I get some bandwidth.
c
Chủ Huỳnh Minh
10/11/2022, 2:30 AMi'm using logstash 7.8.1 and newest signoz
Chủ Huỳnh Minh
10/11/2022, 3:14 AMHi, I've just tested again and see this error in otel collector:
2022-10-11T03:13:02.876Z error tcp/tcp.go:282 Scanner error {"kind": "receiver", "name": "tcplog/logstash", "pipeline": "logs", "operator_id": "tcp_input", "operator_type": "tcp_input", "error": "bufio.Scanner: token too long"}Chủ Huỳnh Minh
10/11/2022, 3:55 AMI tried to increase max_log_size to 10MiB in otel collector config but it doesnt seem to fix it (I dont see any error logs at all)
6 Views