https://signoz.io logo
Join Slack
Powered by
Hey team, trying to json parse our logs but cant s...
# support
a
Hey team, trying to json parse our logs but cant seem to get it working need to parse the body
Copy code
{
  "timestamp": 1684154685443880400,
  "id": "2PpWeXe4qjcHXfMZxu0FL86HjVg",
  "trace_id": "",
  "span_id": "",
  "trace_flags": 0,
  "severity_text": "",
  "severity_number": 0,
  "body": "{\"level\":\"info\",\"ts\":1684154685.4437983,\"logger\":\"controllers.webhook-certs-updater\",\"msg\":\"ignoring webhook due to missing labels\",\"Webhookconfig\":\"/istiod-default-validator\",\"<http://external-secrets.io/component\|external-secrets.io/component\>":\"webhook\"}",
  "resources_string": {
    "k8s_cluster_name": "",
    "k8s_container_name": "",
    "k8s_container_restart_count": "",
    "k8s_deployment_name": "",
    "k8s_namespace_name": "",
    "k8s_node_name": "",
    "k8s_pod_name": "",
    "k8s_pod_start_time": "",
    "k8s_pod_uid": ""
  },
  "attributes_string": {
    "log_file_path": "",
    "log_iostream": "stderr",
    "logtag": "F",
    "time": ""
  },
  "attributes_int": {},
  "attributes_float": {}
}
tried adding
Copy code
- type: json_parser
                  parse_from: attributes.body
                  if: '$matches "^{.*}$"'
to the k8s-infra -> presets -> operators using helm+argocd to deploy signoz
cc: @Tal Cohen
n
please share your entire pipeline, it will be easier to suggest a change.
a
I push this file to a git repo that is connected with an argocd and argocd deploys it
Untitled
Is this enough info for you Nitya? if not please tell me what else i can provide to make it easier
n
cool, jush change the value of 
attributes.body to body
so it will be
Copy code
- type: json_parser
                  parse_from: body
                  if: '$matches "^{.*}$"'
a
ok on it
still the same @nitya-signoz how can i verify that the operator that i added to the helm values is in fact added? where should i look at?
n
the parsed values will be present in the attributes map, can you share the entire screenshot as I cannot see the attributes key in the above screenshot.
a
Copy code
{
  "timestamp": 1684165920042734300,
  "id": "2PpsKtQJSzrNcUu9JxN7j3cgICK",
  "trace_id": "",
  "span_id": "",
  "trace_flags": 0,
  "severity_text": "",
  "severity_number": 0,
  "body": "{\"level\":\"info\",\"msg\":\"Done LoadModelsTask \",\"time\":\"2023-05-15T15:52:00Z\"}",
  "resources_string": {
    "k8s_cluster_name": "",
    "k8s_container_name": "model-analysis-service",
    "k8s_container_restart_count": "3",
    "k8s_deployment_name": "mod1-model-analysis-service",
    "k8s_namespace_name": "imubit-apps",
    "k8s_node_name": "",
    "k8s_pod_name": "",
    "k8s_pod_start_time": "",
    "k8s_pod_uid": ""
  },
  "attributes_string": {
    "log_file_path": "/var/log/pods/imubit-apps_mod1-model-analysis-service-7b8895f9f6-lsdjs_2ec5f13f-85d1-456f-8ee0-2fd4e7eae052/model-analysis-service/3.log",
    "log_iostream": "stdout",
    "logtag": "F",
    "time": "2023-05-15T15:52:00.042734442Z"
  },
  "attributes_int": {},
  "attributes_float": {}
}
n
The if condition is wrong 
'body matches "^{.*}$"'
This is the correct one ^
a
ok
Copy code
- type: json_parser
                  parse_from: body
                  if: 'body matches "^{.*}$"'
so this?
n
right
a
thank u
n
let me know how it goes.
a
same thing 😞
Copy code
"attributes_string": {
    "log_file_path": "/var/log/pods/imubit-apps_mod1-model-analysis-service-7b8895f9f6-lsdjs_2ec5f13f-85d1-456f-8ee0-2fd4e7eae052/model-analysis-service/3.log",
    "log_iostream": "stdout",
    "logtag": "F",
    "time": "2023-05-15T16:21:00.109518425Z"
  },
Screenshot 2023-05-15 at 19.21.56.png
n
any logs on the otel collector ?
a
u want me to check the pod logs for something specific? or in signoz ui and see if its being parsed?
otel collector logs
Untitled
n
I have tested this locally and it seems to be working, can you check if the actual configmap of otel-collector is updated with the pipeline changes ?
a
how would it look if its correctly added/updated?
can u share your configmap of the otel-collector?
n
I have used docker to replicate it and test. But I would want you to check the configmap in k8s
a
Untitled
Untitled
n
There should be one configmap for your k8s infra agent.
a
from 
signoz-otel-collector
ok checking
ok i see all the other operators but i dont see the one i added there
Untitled
Untitled
maybe im setting it wrong in the helm values?
n
Yeah thats the issue, your argocd is not able to push the changes
I haven’t worked with argocd much, but in the end this file should be updated with the operators you want to add
a
well we can put argo to the side this is probably a helm "issue" k8s-infra -> presets -> operators is this not the correct path to add the json_parser?
n
a
prob why its not updating
n
@Prashant Shahi might be able to help with this.
a
🤦 i was missing a field 
logsCollection
k8s-infra -> presets -> logsCollection -> operators sorry and thanks for the help
6 Views
Open in Slack
PreviousNext
Powered by