```Exporting failed. Will retry the request after ...
# support
s
Copy code
Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "metrics", "name": "clickhousemetricswrite", "error": "context deadline exceeded; context deadline exceeded; context deadline exceeded; context deadline exceeded", "errorCauses": [{"error": "context deadline exceeded"}, {"error": "context deadline exceeded"}, {"error": "context deadline exceeded"}, {"error": "context deadline exceeded"}], "interval": "0s"}
getting this error in OtelCollector pods
s
There are two possible reasons. • Not enough CPU/resources • The amount of data trying to be exported is large and it times out.
s
resources: requests: cpu: 300m memory: 1500Mi limits: cpu: 500m memory: 2500Mi above are the resourses for otelcollector and there is also HPA for the above if its due to exporting large data is there any way i can fix it ? @Srikanth Chekuri
Copy code
Below are the resource for the click house
s
What is the ingestion volume and cpu and memory resource usage for otel collectors?
s
The below is for Otel collector
am not aware of this ingestion volume? what is the significance of it ?
s
What I meant by that how much data are you sending to SigNoz deployment. It's significance is, depending on the ingest volume the resource usage goes high and the collector replicas and resources for ClickHouse and otel-collectors should be adjusted.
s
am using the default values for this
Copy code
otelCollector:
  config:
    receivers:
      otlp:
        protocols:
          grpc:
            max_recv_msg_size_mib: 100

  resources:
    requests:
      cpu: 1000m
      memory: 4000Mi
    limits:
      memory: 4000Mi
  
  autoscaling:
    enabled: true
    minReplicas: 6
    maxReplicas: 15
    targetMemoryUtilizationPercentage: 70
    targetCPUUtilizationPercentage: 200

  ingress:
    enabled: true
    className: nginx
    hosts:
      - host: <http://otelcollector.mycompany.com|otelcollector.mycompany.com>
        paths:
          - path: /
            pathType: ImplementationSpecific
            port: 4318
    tls:
      - secretName: otelcollector-mycompany-backend
        hosts:
          - <http://otelcollector.mycompany.com|otelcollector.mycompany.com>
    annotations:
      <http://cert-manager.io/cluster-issuer|cert-manager.io/cluster-issuer>: letsencrypt-prod
Above is my otel collector config
164 Views