Slackbot
02/08/2024, 8:24 AMRaj Kamal
02/08/2024, 8:34 AM^.*?(?P<body>{.*})
- note the ? after .* - which makes it ungreedy
You can see the diff between not having the ? and having it
If you still have issues, please send your otel collector logs and some samples of the processed logs from SigNoz UINikhil Jain
02/08/2024, 10:48 AMNikhil Jain
02/08/2024, 10:59 AMNikhil Jain
02/08/2024, 11:01 AMNikhil Jain
02/08/2024, 11:01 AMNikhil Jain
02/08/2024, 11:02 AMRaj Kamal
02/08/2024, 11:55 AMerror helper/transformer.go:98 Failed to process entry {"kind": "receiver", "name": "filelog", "data_type": "logs", "operator_id": "parsedata", "operator_type": "json_parser", "error": "Unmarshal: there are bytes left after unmarshal, error found in #10 byte of ...|_logger\"}},\"amzn_tra|...,
Would come if you take your first sample log below
{
"channel": "crm_process_booking_log",
"log_type": "DEBUG",
"index_name": "crm_process_booking_log",
"index_type": "crm_process_booking_log",
"error_type": 0,
"message": {
"data": {
"STEP": "Adding Post Data",
"post_data": {
"customer": [
{
"relation": "others",
"age": 28,
"dob": null,
"gender": "F",
"email": null,
"customer_id": "66aff7d8-cd9b-4796-bf17-e66088c3565a",
"customer_name": "Prena",
"contact_number": null
}
],
"slot": {
"slot_id": "684178718"
},
"gender": "M",
"mobile": "6700670875",
"email": "siddhant.khanna_testi_blr+rahul@acko.tech",
"state": "26",
"cityId": "23",
"latitude": 28.5101825,
"longitude": 77.0682661,
"address": "sector 21, gurgaon, Gurgaon, Haryana, 122016",
"zipcode": "122016",
"landmark": "home",
"altmobile": false,
"altemail": false,
"package": [
{
"deal_id": [
"package_216"
]
}
],
"customer_calling_number": "6700670875",
"billing_cust_name": "Rahu",
"sub_locality": "sector 21, gurgaon, Gurgaon, Haryana, 122016",
"hard_copy": 0,
"vendor_booking_id": false,
"vendor_billing_user_id": "4rWzYn5NOhEgNPdiVGWiRA",
"payment_option": "invoice",
"discounted_price": 0,
"sample_collected": "n",
"role_id": 0,
"search_by": "customer_mobile",
"channel_partner": "3",
"channel_partner_user": "5490",
"booked_source": "api",
"zip_search": 2,
"calling_from": "external_api",
"b2bNewExistingUser": false,
"user_id_ajax": "",
"whatsappconcent": 1,
"billing_zone_id": "53",
"zone_id": "53",
"amzn_trace_id": "Root=1-65c33f1c-29fe535658001b076140677c",
"billing_cust_city": "gurgaon",
"billing_cust_city_id": "23",
"locality": "541",
"channel_type": "3",
"channel_user": "5490"
},
"amzn_trace_id": "Root=1-65c33f1c-29fe535658001b076140677c"
},
"calling_trace": {
"file": "/var/www/newcrm-t25/application/controllers/crmuser/crmorder.php",
"line": 1227,
"function": "addProcessBookingLog",
"class": "Enterprise_logger"
}
},
"amzn_trace_id": "Root=1-65c33f1c-29fe535658001b076140677c",
"nginx_request_id": "c93176430784c9dd4e32e35541e6e31a",
"timestamp": "2024-02-07 13:58:12"
}
and use the regex ^.*(?P<message>{.*})
(without ? after .*)
The regex would match just the last part, leading to following value in attributes.message
(as you can see at https://regex101.com/r/cjxTv9/1)
{"file":"\/var\/www\/newcrm-t25\/application\/controllers\/crmuser\/crmorder.php","line":1227,"function":"addProcessBookingLog","class":"Enterprise_logger"}},"amzn_trace_id":"Root=1-65c33f1c-29fe535658001b076140677c","nginx_request_id":"c93176430784c9dd4e32e35541e6e31a","timestamp":"2024-02-07 13:58:12"}
And if you try JSON parsing that message, you would end up with the error in your otel collector
Atleast your 1st sample log will not error out if you use the regex ^.*?(?P<message>{.*})
Nikhil Jain
02/09/2024, 6:29 AMRaj Kamal
02/09/2024, 6:33 AMNikhil Jain
02/09/2024, 7:09 AMNikhil Jain
02/09/2024, 7:10 AMRaj Kamal
02/09/2024, 9:19 AM^.*(?P<body>{.*})
in the regex parser in your filelog receiver in otel collector config. Please change that to the suggested regex ^.*?(?P<body>{.*})
and try againRaj Kamal
02/09/2024, 9:20 AMSigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.
Powered by