Hi Guys, I'm trying to redact certain sensitive in...
# general
g
Hi Guys, I'm trying to redact certain sensitive information bits that are in the log files before exporting using the Open Telemetry collector. Problem - I can't seem to figure out how to redact the value "XnHyH4QALxXnQwmvw7XtB2brs63K4pby" I have tried to use both the hash attribute processor and redaction processor without success. See my yaml config below.
Copy code
processors:
  attributes:
    actions:
      - key: body
        pattern: 'XnHyH4QALxXnQwmvw7XtB2brs63K4pby'
        action: hash

  redaction:
    blocked_values:
      - "XnHyH4QALxXnQwmvw7XtB2brs63K4pby"
See log below - I've applied json formatting to the log so i can apply json_parser on the collector config - that part is working.
Copy code
{
  "body": "Request full data: {\n    \"username\": \"user\",\n    \"key\": \"XnHyH4QALxXnQwmvw7XtB2brs63K4pby\",\n    \"operator\": \"123\",\n    \"no\": \"123\",\n    \"units\": 20\n}",
  "attributes": {
    "host_and_client": {},
    "channel": "production",
    "context": {},
    "datetime": "2023-06-28T12:51:56.962453+03:00",
    "extra": {},
    "level": 200,
    "level_name": "INFO"
  }
}
Is there anything I'm missing? How can I achieve the redaction? Thanks a lot in advance🙏
s
redaction process doesn’t support logs.
g
Hi @Srikanth Chekuri /anyone else interested, I solved this using transform processor. See below config
Copy code
processors:
  transform/redact_sensitive_info:
    error_mode: ignore
    log_statements:
    - context: log
      statements:
      - replace_pattern(attributes["message"], "XnHyH4QALxXnQwmvw7XtB2brs63K4pby", "******")
      - replace_pattern(body, "XnHyH4QALxXnQwmvw7XtB2brs63K4pby", "******")


service:
  pipelines:
    logs:
      receivers: [xyz]
      processors: [transform/redact_sensitive_info, xyz]
      exporters: [xyz]
PS - In my case, I have the log in the body section as well as in the attributes.message that's why there are two replace pattern statements. Transform processor documentation - https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/transformprocessor/README.md