Title
a
Androw Said
05/15/2023, 12:49 PMHey team, trying to json parse our logs but cant seem to get it working
need to parse the body
{
"timestamp": 1684154685443880400,
"id": "2PpWeXe4qjcHXfMZxu0FL86HjVg",
"trace_id": "",
"span_id": "",
"trace_flags": 0,
"severity_text": "",
"severity_number": 0,
"body": "{\"level\":\"info\",\"ts\":1684154685.4437983,\"logger\":\"controllers.webhook-certs-updater\",\"msg\":\"ignoring webhook due to missing labels\",\"Webhookconfig\":\"/istiod-default-validator\",\"<http://external-secrets.io/component\|external-secrets.io/component\>":\"webhook\"}",
"resources_string": {
"k8s_cluster_name": "",
"k8s_container_name": "",
"k8s_container_restart_count": "",
"k8s_deployment_name": "",
"k8s_namespace_name": "",
"k8s_node_name": "",
"k8s_pod_name": "",
"k8s_pod_start_time": "",
"k8s_pod_uid": ""
},
"attributes_string": {
"log_file_path": "",
"log_iostream": "stderr",
"logtag": "F",
"time": ""
},
"attributes_int": {},
"attributes_float": {}
}- type: json_parser
parse_from: attributes.body
if: '$matches "^{.*}$"'cc: @Tal Cohen
n
nitya-signoz
05/15/2023, 3:22 PMplease share your entire pipeline, it will be easier to suggest a change.
a
Androw Said
05/15/2023, 3:26 PMI push this file to a git repo that is connected with an argocd and argocd deploys it
Is this enough info for you Nitya? if not please tell me what else i can provide to make it easier
n
nitya-signoz
05/15/2023, 3:28 PMcool, jush change the value of
attributes.body to body- type: json_parser
parse_from: body
if: '$matches "^{.*}$"'a
Androw Said
05/15/2023, 3:28 PMok on it
still the same
@nitya-signoz how can i verify that the operator that i added to the helm values is in fact added?
where should i look at?
n
nitya-signoz
05/15/2023, 3:56 PMthe parsed values will be present in the attributes map, can you share the entire screenshot as I cannot see the attributes key in the above screenshot.
a
Androw Said
05/15/2023, 3:57 PM{
"timestamp": 1684165920042734300,
"id": "2PpsKtQJSzrNcUu9JxN7j3cgICK",
"trace_id": "",
"span_id": "",
"trace_flags": 0,
"severity_text": "",
"severity_number": 0,
"body": "{\"level\":\"info\",\"msg\":\"Done LoadModelsTask \",\"time\":\"2023-05-15T15:52:00Z\"}",
"resources_string": {
"k8s_cluster_name": "",
"k8s_container_name": "model-analysis-service",
"k8s_container_restart_count": "3",
"k8s_deployment_name": "mod1-model-analysis-service",
"k8s_namespace_name": "imubit-apps",
"k8s_node_name": "",
"k8s_pod_name": "",
"k8s_pod_start_time": "",
"k8s_pod_uid": ""
},
"attributes_string": {
"log_file_path": "/var/log/pods/imubit-apps_mod1-model-analysis-service-7b8895f9f6-lsdjs_2ec5f13f-85d1-456f-8ee0-2fd4e7eae052/model-analysis-service/3.log",
"log_iostream": "stdout",
"logtag": "F",
"time": "2023-05-15T15:52:00.042734442Z"
},
"attributes_int": {},
"attributes_float": {}
}n
nitya-signoz
05/15/2023, 3:59 PMThe if condition is wrong
'body matches "^{.*}$"'This is the correct one ^
a
Androw Said
05/15/2023, 3:59 PMok
- type: json_parser
parse_from: body
if: 'body matches "^{.*}$"'n
nitya-signoz
05/15/2023, 3:59 PMright
a
Androw Said
05/15/2023, 3:59 PMthank u
n
nitya-signoz
05/15/2023, 4:00 PMlet me know how it goes.
a
Androw Said
05/15/2023, 4:21 PMsame thing 😞
"attributes_string": {
"log_file_path": "/var/log/pods/imubit-apps_mod1-model-analysis-service-7b8895f9f6-lsdjs_2ec5f13f-85d1-456f-8ee0-2fd4e7eae052/model-analysis-service/3.log",
"log_iostream": "stdout",
"logtag": "F",
"time": "2023-05-15T16:21:00.109518425Z"
},Screenshot 2023-05-15 at 19.21.56.png
n
nitya-signoz
05/15/2023, 4:22 PMany logs on the otel collector ?
a
Androw Said
05/15/2023, 4:26 PMu want me to check the pod logs for something specific? or in signoz ui and see if its being parsed?
otel collector logs
n
nitya-signoz
05/16/2023, 7:50 AMI have tested this locally and it seems to be working, can you check if the actual configmap of otel-collector is updated with the pipeline changes ?
a
Androw Said
05/16/2023, 8:52 AMhow would it look if its correctly added/updated?
can u share your configmap of the otel-collector?
n
nitya-signoz
05/16/2023, 8:55 AMI have used docker to replicate it and test. But I would want you to check the configmap in k8s
a
Androw Said
05/16/2023, 8:58 AMUntitled
n
nitya-signoz
05/16/2023, 8:59 AMThere should be one configmap for your k8s infra agent.
a
Androw Said
05/16/2023, 8:59 AMfrom
signoz-otel-collectorok checking
ok i see all the other operators but i dont see the one i added there
Untitled
maybe im setting it wrong in the helm values?
n
nitya-signoz
05/16/2023, 9:00 AMYeah thats the issue, your argocd is not able to push the changes
I haven’t worked with argocd much, but in the end this file should be updated with the operators you want to add
a
Androw Said
05/16/2023, 9:07 AMwell we can put argo to the side this is probably a helm "issue"
k8s-infra -> presets -> operators
is this not the correct path to add the json_parser?
n
nitya-signoz
05/16/2023, 9:08 AMhttps://github.com/SigNoz/charts/blob/main/charts/k8s-infra/values.yaml the k8s infra values.yaml
You can also create an override file https://signoz.io/docs/userguide/collect_kubernetes_pod_logs/#steps-to-filterexclude-logs-collection
a
Androw Said
05/16/2023, 9:38 AMhttps://github.com/SigNoz/charts/blob/main/charts/signoz/values.yaml#L1998
it says here it is ignored
prob why its not updating
n
nitya-signoz
05/16/2023, 9:39 AM@Prashant Shahi might be able to help with this.
a
Androw Said
05/16/2023, 10:25 AM🤦
i was missing a field
logsCollection