https://signoz.io logo
#support
Title
# support
a

Androw Said

05/15/2023, 12:49 PM
Hey team, trying to json parse our logs but cant seem to get it working need to parse the body
Copy code
{
  "timestamp": 1684154685443880400,
  "id": "2PpWeXe4qjcHXfMZxu0FL86HjVg",
  "trace_id": "",
  "span_id": "",
  "trace_flags": 0,
  "severity_text": "",
  "severity_number": 0,
  "body": "{\"level\":\"info\",\"ts\":1684154685.4437983,\"logger\":\"controllers.webhook-certs-updater\",\"msg\":\"ignoring webhook due to missing labels\",\"Webhookconfig\":\"/istiod-default-validator\",\"<http://external-secrets.io/component\|external-secrets.io/component\>":\"webhook\"}",
  "resources_string": {
    "k8s_cluster_name": "",
    "k8s_container_name": "",
    "k8s_container_restart_count": "",
    "k8s_deployment_name": "",
    "k8s_namespace_name": "",
    "k8s_node_name": "",
    "k8s_pod_name": "",
    "k8s_pod_start_time": "",
    "k8s_pod_uid": ""
  },
  "attributes_string": {
    "log_file_path": "",
    "log_iostream": "stderr",
    "logtag": "F",
    "time": ""
  },
  "attributes_int": {},
  "attributes_float": {}
}
tried adding
Copy code
- type: json_parser
                  parse_from: attributes.body
                  if: '$matches "^{.*}$"'
to the k8s-infra -> presets -> operators using helm+argocd to deploy signoz
cc: @Tal Cohen
n

nitya-signoz

05/15/2023, 3:22 PM
please share your entire pipeline, it will be easier to suggest a change.
a

Androw Said

05/15/2023, 3:26 PM
I push this file to a git repo that is connected with an argocd and argocd deploys it
Is this enough info for you Nitya? if not please tell me what else i can provide to make it easier
n

nitya-signoz

05/15/2023, 3:28 PM
cool, jush change the value of
attributes.body to body
so it will be
Copy code
- type: json_parser
                  parse_from: body
                  if: '$matches "^{.*}$"'
a

Androw Said

05/15/2023, 3:28 PM
ok on it
still the same @nitya-signoz how can i verify that the operator that i added to the helm values is in fact added? where should i look at?
n

nitya-signoz

05/15/2023, 3:56 PM
the parsed values will be present in the attributes map, can you share the entire screenshot as I cannot see the attributes key in the above screenshot.
a

Androw Said

05/15/2023, 3:57 PM
Copy code
{
  "timestamp": 1684165920042734300,
  "id": "2PpsKtQJSzrNcUu9JxN7j3cgICK",
  "trace_id": "",
  "span_id": "",
  "trace_flags": 0,
  "severity_text": "",
  "severity_number": 0,
  "body": "{\"level\":\"info\",\"msg\":\"Done LoadModelsTask \",\"time\":\"2023-05-15T15:52:00Z\"}",
  "resources_string": {
    "k8s_cluster_name": "",
    "k8s_container_name": "model-analysis-service",
    "k8s_container_restart_count": "3",
    "k8s_deployment_name": "mod1-model-analysis-service",
    "k8s_namespace_name": "imubit-apps",
    "k8s_node_name": "",
    "k8s_pod_name": "",
    "k8s_pod_start_time": "",
    "k8s_pod_uid": ""
  },
  "attributes_string": {
    "log_file_path": "/var/log/pods/imubit-apps_mod1-model-analysis-service-7b8895f9f6-lsdjs_2ec5f13f-85d1-456f-8ee0-2fd4e7eae052/model-analysis-service/3.log",
    "log_iostream": "stdout",
    "logtag": "F",
    "time": "2023-05-15T15:52:00.042734442Z"
  },
  "attributes_int": {},
  "attributes_float": {}
}
n

nitya-signoz

05/15/2023, 3:59 PM
The if condition is wrong
'body matches "^{.*}$"'
This is the correct one ^
a

Androw Said

05/15/2023, 3:59 PM
ok
Copy code
- type: json_parser
                  parse_from: body
                  if: 'body matches "^{.*}$"'
so this?
n

nitya-signoz

05/15/2023, 3:59 PM
right
a

Androw Said

05/15/2023, 3:59 PM
thank u
n

nitya-signoz

05/15/2023, 4:00 PM
let me know how it goes.
a

Androw Said

05/15/2023, 4:21 PM
same thing 😞
Copy code
"attributes_string": {
    "log_file_path": "/var/log/pods/imubit-apps_mod1-model-analysis-service-7b8895f9f6-lsdjs_2ec5f13f-85d1-456f-8ee0-2fd4e7eae052/model-analysis-service/3.log",
    "log_iostream": "stdout",
    "logtag": "F",
    "time": "2023-05-15T16:21:00.109518425Z"
  },
Screenshot 2023-05-15 at 19.21.56.png
n

nitya-signoz

05/15/2023, 4:22 PM
any logs on the otel collector ?
a

Androw Said

05/15/2023, 4:26 PM
u want me to check the pod logs for something specific? or in signoz ui and see if its being parsed?
otel collector logs
n

nitya-signoz

05/16/2023, 7:50 AM
I have tested this locally and it seems to be working, can you check if the actual configmap of otel-collector is updated with the pipeline changes ?
a

Androw Said

05/16/2023, 8:52 AM
how would it look if its correctly added/updated?
can u share your configmap of the otel-collector?
n

nitya-signoz

05/16/2023, 8:55 AM
I have used docker to replicate it and test. But I would want you to check the configmap in k8s
a

Androw Said

05/16/2023, 8:58 AM
Untitled
n

nitya-signoz

05/16/2023, 8:59 AM
There should be one configmap for your k8s infra agent.
a

Androw Said

05/16/2023, 8:59 AM
from
signoz-otel-collector
ok checking
ok i see all the other operators but i dont see the one i added there
Untitled
maybe im setting it wrong in the helm values?
n

nitya-signoz

05/16/2023, 9:00 AM
Yeah thats the issue, your argocd is not able to push the changes
I haven’t worked with argocd much, but in the end this file should be updated with the operators you want to add
a

Androw Said

05/16/2023, 9:07 AM
well we can put argo to the side this is probably a helm "issue" k8s-infra -> presets -> operators is this not the correct path to add the json_parser?
n

nitya-signoz

05/16/2023, 9:08 AM
a

Androw Said

05/16/2023, 9:38 AM
prob why its not updating
n

nitya-signoz

05/16/2023, 9:39 AM
@Prashant Shahi might be able to help with this.
a

Androw Said

05/16/2023, 10:25 AM
🤦 i was missing a field
logsCollection
k8s-infra -> presets -> logsCollection -> operators sorry and thanks for the help
2 Views