Is this common error in signoz when testing local host log f

Question

Is this common error in signoz when testing local host log file 2023 10 06T15 50 05 809Z error helper transformer go 99 Failed to process entry kind receiver name filelog data type logs operator id remove operator type remove error remove field does not exist action send entry observed timestamp 2023 10 06T15 50 05 809132405Z timestamp 0001 01 01T00 00 00Z body at <http com ibm ws|com ibm ws> management tools AdminTool executeUtility AdminTool java 282 attributes log file name startServer log severity 0 scope name

Srikanth Chekuri · Answer

You can put a condition to avoid this error `if key in attributes` or something like that

Noor Ali · Answer

Srikanth one more question this is my operator filelog include tmp startServer log start at beginning operators type regex parser regex D+ S+ 0 9 0 9 + S+ D + W+ S 8 D+ D+ A Z 0 9 w+ D+ timestamp layout Y m d H M S z parse from attributes timestamp field type remove field attributes timestamp when restart the collector I get the following error 2023 10 07 01 28 57 application run finished with error failed to build pipelines failed to create filelog receiver for data type logs no named capture groups in regex pattern This regex parser tested ok in regex101 is this parser allowed in signoz

Noor Ali · Answer

what is best regex pattern is allowed by signoz to run without giving the following error Error failed to build pipelines failed to create filelog receiver for data type logs no named capture groups in regex pattern 2023 10 09 16 23 51 application run finished with error failed to build pipelines failed to create filelog receiver for data type logs no named capture groups in regex pattern

Srikanth Chekuri · Answer

< nitya signoz> might be able to help

Noor Ali · Answer

Hello Nitya I am currently using this type regex parser regex S+ d+ D+ S+ d+ d+ 0 9 D+ W+ S+ + a z + W+ S+ D+ timestamp parse from attributes timestamp field layout Y m d H M S z type move from attributes body to body type remove field attributes timestamp

Noor Ali · Answer

I am getting this error filelog receiver for data type logs no named capture groups in regex pattern

Noor Ali · Answer

Per Signoz all parse are test in regex101

nitya-signoz · Answer

Since you are extracting `timestamp field` and `body` form the regex those named group should be present in your regex ` S+ d+ D+ S+ d+ d+ 0 9 D+ W+ S+ + a z + W+ S+ D+ ` <https github com StefanSchroeder Golang Regex Tutorial blob master 01 chapter2 markdown named matches>

Noor Ali · Answer

Just wondering getting closer I made few changes I got this error 2023 10 10 16 20 41 application run finished with error failed to build pipelines failed to create filelog receiver for data type logs compiling regex error parsing regexp invalid or unsupported Perl syntax ` lt `

nitya-signoz · Answer

Can you choose golang as the language in regex101 and test your regex out

Noor Ali · Answer

I currently have it setup this way filelog include tmp startServer log start at beginning operators type regex parser regex S+ d+ D+ S+ d+ d+ 0 9 D+ W+ S+ + a z + W+ S+ D+ P lt body gt | lt timestamp gt S+ | parseTimestamp field=timestamp timestamp parse from attributes timestamp field layout Y m d H M S z type move from attributes body to body type remove field attributes timestamp tcplog docker listen address 0 0 0 0 2255 operators type regex parser regex ^ P lt time gt d+ d+ d+ d 2 d+ d 2 d 3 UTC P lt body gt timestamp parse from attributes timestamp layout Y m dT H M S LZ type move from attributes body to body type remove field attributes timestamp

Noor Ali · Answer

Hello Nitya please review my word doc let me what else I need to do make it work for us thanks

Noor Ali · Answer

I need to know why it is not working for me if I am able to make this work I will be moving forward to deploy it next Thanks

nitya-signoz · Answer

can you share your config the regex in doc and the config doesn t match Also provide some sample log lines for testing it out

Noor Ali · Answer

eceivers filelog include tmp startServer log start at beginning operators type regex parser regex layout Y m d H M S z type move from attributes body to body type remove field attributes timestamp tcplog docker listen address 0 0 0 0 2255 operators type regex parser regex ^ P lt time gt d+ d+ d+ d 2 d+ d 2 d 3 UTC P lt body gt timestamp parse from attributes timestamp layout Y m dT H M S LZ type move from attributes body to body type remove field attributes timestamp

Noor Ali · Answer

This is what we or I am trying to parse 4 28 21 12 23 54 157 UTC 00000001 ManagerAdmin I TRAS0017I The startup trace state is =info 4 28 21 12 23 54 302 UTC 00000001 AdminTool A ADMU0128I Starting tool with the Custom01 profile 4 28 21 12 23 54 305 UTC 00000001 AdminTool A ADMU3100I Reading configuration for server igawas02 4 28 21 12 23 54 321 UTC 00000001 ImplFactory W WSVR0072W Ignoring undeclared override of interface com ibm websphere cluster topography DescriptionManager with implementation <http com ibm ws|com ibm ws> cluster propagation bulletinboard BBDescriptionManager 4 28 21 12 23 54 574 UTC 00000001 ModelMgr I WSVR0801I Initializing all server configuration models

Noor Ali · Answer

I have been trying to use P lt timestamp gt lt message gt P lt body gt and even lt P lt Date gt but has been showing up with error

Noor Ali · Answer

my last error now is this 2023 10 12 13 53 49 application run finished with error failed to get config cannot resolve the configuration cannot retrieve the configuration yaml line 8 mapping values are not allowed in this context

Noor Ali · Answer

I need to know what I am doing in correct so I can educate my team member in the future thank you so much for helping me on this

nitya-signoz · Answer

The regex pattern is not present and it should be enclosed in quotes The regex is not correct it doesn t work golang as language

Noor Ali · Answer

do you mean like this

Noor Ali · Answer

I have this now it is matching now in regex

Noor Ali · Answer

cape character 2023 10 12 17 10 57 application run finished with error failed to get config cannot resolve the configuration cannot retrieve the configuration yaml line 7 found unknown escape character

Noor Ali · Answer

Please do tell me where is the issue in my config now receivers filelog include tmp startServer log start at beginning operators type regex parser regex S+ d+ D+ S + d+ d+ 0 9 D+ W+ S+ a z + W+ S+ D+ P lt bod gt gt | P lt timestamp gt S+ |parseTimestamp field=timestamp layout Y m d H M S z type move from attributes body to body type remove field attributes timestamp tcplog docker listen address 0 0 0 0 2255 operators type regex parser regex ^ P lt time gt d+ d+ d+ d 2 d+ d 2 d 3 UTC P lt body gt timestamp parse from attributes timestamp layout Y m dT H M S LZ type move from attributes body to body type remove field attributes timestamp

Noor Ali · Answer

2023 10 12 17 15 02 application run finished with error failed to get config cannot resolve the configuration cannot retrieve the configuration yaml line 5 did not find expected key with this config operators type regex parser regex S+ d+ D+ S + d+ d+ 0 9 D+ W+ S+ a z + W+ S+ D+ P lt bod gt gt | P lt timestamp gt S+ |parseTimestamp field=timestamp layout Y m d H M S z type move from attributes body to body type remove field attributes timestamp

Noor Ali · Answer

Does it have to do with my pass 30 day testing with my log in id just wondering that is all