Juha Patrikainen
03/20/2023, 1:48 PM2023-03-20T11:57:39.619Z info fileconsumer/file.go:171 Started watching file {"kind": "receiver", "name": "filelog/k8s", "pipeline": "logs", "component": "fileconsumer", "path": "/var/log/pods/*********/configuration/0.log"}
I have verified that file contains all the logs for the container.
There are no errors visible in agent logs, signoz-otel-collector logs or clickhouse logs, but the logs are not visible in SigNoz.Kurt Crockett
03/20/2023, 2:20 PMNilanjan Roy
03/20/2023, 5:20 PMNilanjan Roy
03/21/2023, 5:44 AMNilanjan Roy
03/21/2023, 10:22 AMNilanjan Roy
03/21/2023, 11:32 AMDavid Glick
03/21/2023, 6:27 PMsudhanshu dev
03/22/2023, 10:16 AM"10.21.18.240 - - [22/Mar/2023:08:19:04 +0000] \"GET /status HTTP/1.1\" 200 1249 \"-\" \"kube-probe/1.22+\"\n"
Below is the configuration of custom processor.
logstransform/keepattrs:
operators:
- type: remove
id: remove-1
field: attributes
logstransform/parselog:
operators:
- type: router
id: get-access
routes:
- output: parser-access
expr: 'body matches "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)){3}"'
- type: grok_parser
id: parser-access
parse_from: body
pattern: '%{IP:ip}%{SPACE}-%{SPACE}-%{SPACE}\[%{HTTPDATE:timestamp}\]%{SPACE}\\"%{WORD:httpmethod}%{SPACE}%{DATA:request}\"%{SPACE}%{NUMBER:status}'
parse_to: attributes
Grok debugger output.
I am getting below error.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x24701e1]
goroutine 1 [running]:
<http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/adapter.(*receiver).Shutdown(0xc00107ccf0|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/adapter.(*receiver).Shutdown(0xc00107ccf0>, {0x52eba28, 0xc000076028})
/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.66.0/adapter/receiver.go:148 +0x81
<http://go.opentelemetry.io/collector/service/internal/pipelines.(*Pipelines).ShutdownAll(0xc000f781e0|go.opentelemetry.io/collector/service/internal/pipelines.(*Pipelines).ShutdownAll(0xc000f781e0>, {0x52eba28, 0xc000076028})
/go/pkg/mod/go.opentelemetry.io/collector@v0.66.0/service/internal/pipelines/pipelines.go:121 +0x499
<http://go.opentelemetry.io/collector/service.(*service).Shutdown(0xc000f5a800|go.opentelemetry.io/collector/service.(*service).Shutdown(0xc000f5a800>, {0x52eba28, 0xc000076028})
/go/pkg/mod/go.opentelemetry.io/collector@v0.66.0/service/service.go:121 +0xd4
<http://go.opentelemetry.io/collector/service.(*Collector).shutdownServiceAndTelemetry(0xc001717a88|go.opentelemetry.io/collector/service.(*Collector).shutdownServiceAndTelemetry(0xc001717a88>, {0x52eba28?, 0xc000076028?})
/go/pkg/mod/go.opentelemetry.io/collector@v0.66.0/service/collector.go:264 +0x36
<http://go.opentelemetry.io/collector/service.(*Collector).setupConfigurationComponents(0xc001717a88|go.opentelemetry.io/collector/service.(*Collector).setupConfigurationComponents(0xc001717a88>, {0x52eba28, 0xc000076028})
/go/pkg/mod/go.opentelemetry.io/collector@v0.66.0/service/collector.go:166 +0x27d
<http://go.opentelemetry.io/collector/service.(*Collector).Run(0xc001717a88|go.opentelemetry.io/collector/service.(*Collector).Run(0xc001717a88>, {0x52eba28, 0xc000076028})
/go/pkg/mod/go.opentelemetry.io/collector@v0.66.0/service/collector.go:190 +0x46
<http://go.opentelemetry.io/collector/service.NewCommand.func1(0xc0005cac00|go.opentelemetry.io/collector/service.NewCommand.func1(0xc0005cac00>, {0x490a0cd?, 0x1?, 0x1?})
/go/pkg/mod/go.opentelemetry.io/collector@v0.66.0/service/command.go:53 +0x479
<http://github.com/spf13/cobra.(*Command).execute(0xc0005cac00|github.com/spf13/cobra.(*Command).execute(0xc0005cac00>, {0xc00006e070, 0x1, 0x1})
/go/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:916 +0x862
<http://github.com/spf13/cobra.(*Command).ExecuteC(0xc0005cac00)|github.com/spf13/cobra.(*Command).ExecuteC(0xc0005cac00)>
/go/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:1044 +0x3bc
<http://github.com/spf13/cobra.(*Command).Execute(...)|github.com/spf13/cobra.(*Command).Execute(...)>
/go/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:968
main.runInteractive({{0xc0006bdb90, 0xc00082aea0, 0xc00082a120, 0xc0006bd7d0}, {{0x4943e48, 0x15}, {0x49417a5, 0x15}, {0x49065d8, 0x6}}, ...})
/src/cmd/signozcollector/main.go:37 +0x5e
main.run(...)
/src/cmd/signozcollector/main_others.go:8
main.main()
/src/cmd/signozcollector/main.go:30 +0x1d8
Alok Singh
03/23/2023, 8:06 AMSachin Kumar
03/23/2023, 8:32 AMhttps://youtu.be/IsOQxc3wqyc▾
receivers:
filelog/dockercontainers:
include: [ "/var/lib/docker/containers/*/*.log" ]
start_at: end
include_file_path: true
include_file_name: false
operators:
- type: json_parser
id: parser-docker
output: extract_metadata_from_filepath
timestamp:
parse_from: attributes.time
layout: '%Y-%m-%dT%H:%M:%S.%LZ'
- type: regex_parser
id: extract_metadata_from_filepath
regex: '^.*containers/(?P<container_id>[^_]+)/.*log$'
parse_from: attributes["log.file.path"]
output: parse_body
- type: move
id: parse_body
from: attributes.log
to: body
output: time
- type: remove
id: time
field: attributes.time
opencensus:
endpoint: 0.0.0.0:55678
otlp/spanmetrics:
protocols:
grpc:
endpoint: localhost:12345
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
cors:
allowed_origins:
- <http://65.0.130.56:3000>
jaeger:
protocols:
grpc:
endpoint: 0.0.0.0:14250
thrift_http:
endpoint: 0.0.0.0:14268
# thrift_compact:
# endpoint: 0.0.0.0:6831
# thrift_binary:
# endpoint: 0.0.0.0:6832
Please help me figure out the errorsudhanshu dev
03/23/2023, 9:55 AMAshna
03/23/2023, 12:52 PMAshna
03/23/2023, 12:53 PMSindhu S
03/23/2023, 4:20 PMlocalhost:3334/metrics
. I do not want to run a prometheus instance to consume these. What configuration should I add to signoz to consume the metrics from my app's endpoint?
Also I do not see a "Metrics" menu item in the Signoz dashboard. Does this need some sort of config to enable?Ashna
03/24/2023, 5:02 AMAshna
03/24/2023, 5:02 AMAshna
03/24/2023, 5:05 AMsudhanshu dev
03/24/2023, 6:47 AMAlex Grönholm
03/24/2023, 9:20 AMAlex Grönholm
03/24/2023, 9:47 AMStewart Thomson
03/24/2023, 8:40 PMBhuwan Kaushik
03/24/2023, 8:45 PMBhuwan Kaushik
03/24/2023, 11:23 PMLeong
03/27/2023, 7:13 AMShreyash
03/27/2023, 9:45 AM2023.03.27 09:41:32.756403 [ 7 ] {} <Information> Application: Shutting down storages.
2023.03.27 09:41:32.756425 [ 7 ] {} <Information> Context: Shutdown disk default
2023.03.27 09:41:32.756432 [ 7 ] {} <Information> Context: Shutdown disk s3
2023.03.27 09:41:32.756448 [ 7 ] {} <Information> DiskObjectStorage(DiskS3): Shutting down disk s3
2023.03.27 09:41:32.756455 [ 7 ] {} <Information> DiskObjectStorage(DiskS3): Disk s3 shut down
2023.03.27 09:41:33.197310 [ 76 ] {} <Information> system.metric_log (ee60f679-08f7-43a6-b64a-b1c5b6033ba0): Delaying inserting block by 131.8256738556407 ms. because there are 255 parts
2023.03.27 09:41:33.816440 [ 7 ] {} <Error> Application: DB::Exception: Suspiciously many (20 parts, 259.02 KiB in total) broken parts to remove while maximum allowed broken parts count is 10. You can change the maximum value with merge tree setting 'max_suspicious_broken_parts' in <merge_tree> configuration section or in table settings in .sql file (don't forget to return setting back to default value): Cannot attach table `signoz_logs`.`logs` from metadata file /var/lib/clickhouse/store/5f9/5f9d716d-9913-47d3-90b0-c95f1c62bfa5/logs.sql from query ATTACH TABLE signoz_logs.logs UUID 'dbc363d8-5e0b-40c5-9df5-9e6260cd082f' (`timestamp` UInt64 CODEC(DoubleDelta, LZ4), `observed_timestamp` UInt64 CODEC(DoubleDelta, LZ4), `id` String CODEC(ZSTD(1)), `trace_id` String CODEC(ZSTD(1)), `span_id` String CODEC(ZSTD(1)), `trace_flags` UInt32, `severity_text` LowCardinality(String) CODEC(ZSTD(1)), `severity_number` UInt8, `body` String CODEC(ZSTD(2)), `resources_string_key` Array(String) CODEC(ZSTD(1)), `resources_string_value` Array(String) CODEC(ZSTD(1)), `attributes_string_key` Array(String) CODEC(ZSTD(1)), `attributes_string_value` Array(String) CODEC(ZSTD(1)), `attributes_int64_key` Array(String) CODEC(ZSTD(1)), `attributes_int64_value` Array(Int64) CODEC(ZSTD(1)), `attributes_float64_key` Array(String) CODEC(ZSTD(1)), `attributes_float64_value` Array(Float64) CODEC(ZSTD(1)), `k8s_cluster_name` String MATERIALIZED resources_string_value[indexOf(resources_string_key, 'k8s_cluster_name')] CODEC(LZ4), INDEX body_idx body TYPE tokenbf_v1(10240, 3, 0) GRANULARITY 4, INDEX id_minmax id TYPE minmax GRANULARITY 1, INDEX k8s_cluster_name_idx k8s_cluster_name TYPE bloom_filter(0.01) GRANULARITY 64) ENGINE = MergeTree PARTITION BY toDate(timestamp / 1000000000) ORDER BY (timestamp, id) TTL toDateTime(timestamp / 1000000000) + toIntervalSecond(345600), toDateTime(timestamp / 1000000000) + toIntervalSecond(172800) TO VOLUME 's3' SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1, storage_policy = 'tiered'
2023.03.27 09:41:33.816490 [ 7 ] {} <Information> Application: shutting down
2023.03.27 09:41:33.816548 [ 8 ] {} <Information> BaseDaemon: Stop SignalListener thread
2023.03.27 09:41:33.843565 [ 1 ] {} <Information> Application: Child process exited normally with code 70.
Travis Chambers
03/27/2023, 5:55 PMAl
03/27/2023, 6:44 PMBhuwan Kaushik
03/27/2023, 9:18 PMDivyanshu Negi
03/28/2023, 4:02 AMabhinav sharma
03/28/2023, 5:41 AMabhinav sharma
03/28/2023, 5:41 AMSrikanth Chekuri
03/28/2023, 9:36 AMbecause I am unable to use it in one of my application (NodeJS) which is hosted on GravitonThis is not related to SigNoz. If your application doesn’t run on Graviton then it’s an application issue. Where is SigNoz running?