https://signoz.io logo
Powered by
#support
Title
# support
k

Krishnadas K P

08/02/2023, 1:38 PM
#support Is it possible to mask certain data from logs before injecting to Signoz Clickhouse backend ? I am using filelog receiver
v

Vishal Sharma

08/03/2023, 5:06 AM
@nitya-signoz Can you help here?
n

nitya-signoz

08/03/2023, 5:10 AM
k

Krishnadas K P

08/03/2023, 6:59 AM
Thanks for the reply @nitya-signoz. I have tried this but seems like I am doing something wrong. I am trying to mask email from the body and keys and replace with dummy value. My json response is attached. - replace_all_matches(attributes, "dummy@email.com", "[a-zA-Z0-9.]+@[a-zA-Z]+[.com]+") Do I need to replace attributes with something else ? As per the json, I am seeing attributes_string instead
n

nitya-signoz

08/03/2023, 10:38 AM
attributes will remain attributes Shouldn’t it be 
replace_all_matches(attributes, "[a-zA-Z0-9.]+@[a-zA-Z]+[.com]+", "<mailto:dummy@email.com|dummy@email.com>")
?
Is your email present in body or in attributes ?
k

Krishnadas K P

08/04/2023, 4:12 AM
Email is present inside the body and also showing as attributes_string.csusername since I am using a regex to capture keys from the original log line
@nitya-signoz When I try the order you suggested, it gives me this error application run finished with error: invalid configuration: processors:transform couldn't create function: the pattern supplied to replace_match is not a valid pattern: expected close range character Not sure why it says expected close range character, when it is already close range.
n

nitya-signoz

08/04/2023, 12:54 PM
Here you go
Copy code
transform:
    log_statements:
      - context: log
        statements:
          - replace_pattern(attributes["csusername"], "[a-zA-Z0-9.]+@[a-zA-Z]+[.com]+", "<mailto:dummy@email.com|dummy@email.com>")
k

Krishnadas K P

08/04/2023, 1:40 PM
@nitya-signoz Great. I was trying attributes.csusername and wasnt working. Now the value is replaced with dummy value. Thank you so much. Also used replace_pattern(body, "[a-zA-Z0-9.]+@[a-zA-Z]+[.com]+", "dummy@email.com") to mask content from body
@nitya-signoz I am using a regex to parse data from IIS logs and I am successfully getting these as attributes in Signoz. Is there a way to make some attributes as int instead of string ? I have some attributes like timetaken which will need operations like 'greater than' on them which is not possible with string.
n

nitya-signoz

08/31/2023, 3:54 PM
you can use convert to cast the datatype
Powered by