https://signoz.io logo
Join Slack
Powered by
hi, we are migrating from datadog to signoz, and w...
# support
d
hi, we are migrating from datadog to signoz, and we have two backends: one in aws eks and another in heroku. we’re trying to route logs from both backends to signoz but are facing some challenges: 1. how to configure heroku drains similar to datadog? in datadog, we use the following setup: 
<https://http-intake.logs.datadoghq.com/api/v2/logs?dd-api-key=xxx&ddsource=heroku&env=staging&service=bureau-staging&host=bureau-staging>
here, we use query parameters for source, environment, etc. how can we achieve the same configuration in signoz? 2. how to handle multiline logs from heroku? we’re unsure how to handle multiline logs from heroku effectively in signoz. 3. how to change 
deploymentEnvironment
based on pod name? our eks cluster contains apps for different environments (e.g., maitred, maitred-staging, and maitred-sandbox), all running under the default namespace. is there a way to override the deploymentEnvironment field based on the pod name without changing the namespace?
we tried this configuration, but it doesn’t seem to work:
Copy code
global:
  cloud: aws
  clusterName: xxx
  deploymentEnvironment: porter
otelCollectorEndpoint: ingest.us.signoz.cloud:443
otelInsecure: false
signozApiKey: xxxx
presets:
  otlpExporter:
    enabled: true
  loggingExporter:
    enabled: false
  logsCollection:
    enabled: true
    whitelist:
      enabled: true
      signozLogs: false
      namespaces:
        - default
      pods:
        - .*
      additionalExclude:
        - namespace: default
          podRegex: ^signoz.*
      containers: []
      additionalInclude: []

otelCollector:
  config:
    processors:
      resource:
        attributes:
          actions:
            - key: deploymentEnvironment
              action: upsert
              value: sandbox
              # Apply if the pod name contains "sandbox"
              conditions:
                - key: k8s.pod.name
                  regex: .*sandbox.*
            - key: deploymentEnvironment
              action: upsert
              value: staging
              # Apply if the pod name contains "staging"
              conditions:
                - key: k8s.pod.name
                  regex: .*staging.*
            - key: deploymentEnvironment
              action: upsert
              value: prod
              # Apply if the pod name does not contain "sandbox" or "staging"
              conditions:
                - key: k8s.pod.name
                  regex: ^((?!sandbox|staging).)*$
n
Hi as of now you cannot set the environments from the params itself, but thanks for bringing this up. You can use pipelines https://signoz.io/docs/logs-pipelines/introduction/ to add different attributes based on certain keys using the add processor https://signoz.io/docs/logs-pipelines/processors/#add Also can you share the logs of otel collector after applying this change
d
Hi @nitya-signoz, sure, we will try logs pipelines. regarding on the logs from heroku, how to add 
deploymentEnvironment
?
n
Do you have any kind of field though which you can identify the the environment ?
d
we don't have some reliable field to identify the environment for heroku. is there anything similar to datadog's query param for that? like https://http-intake.logs.datadoghq.com/api/v2/logs?dd-api-key=xxx&amp;`ddsource=heroku&env=staging&service=bureau-staging&host=bureau-staging`
n
Ah, as of now we don’t have support for that , I have created an issue for the same https://github.com/SigNoz/signoz/issues/6650
13 Views
Open in Slack
PreviousNext
Powered by