https://signoz.io logo
Join the conversationJoin Slack
Channels
contributing
contributing-frontend
general
github-activities
incidents
k8s_operator_helm
reactjs-config
signoz-alert-testing
support
testsupport
watercooler
webhook-dev
write-for-signoz
Powered by Linen
support
  • g

    Giorgio Ventura

    10/08/2022, 12:30 AM
    Hello guys
  • g

    Giorgio Ventura

    10/08/2022, 12:30 AM
    i have a question I have to deploy signoz in Linux But I want to monitor windows metrics Do you know how to do it?
  • g

    Giorgio Ventura

    10/09/2022, 7:45 AM
    Hi Team
    p
    s
    • 3
    • 12
  • c

    Chủ Huỳnh Minh

    10/10/2022, 5:02 AM
    Hi Team, I'm having issue with sending logs from logstash to signoz. I followed this guide: https://signoz.io/docs/userguide/logstash_to_signoz/ but in signoz dashboard there's still no logs at all.
    p
    n
    • 3
    • 10
  • b

    Bhavesh Patel

    10/10/2022, 6:49 AM
    Hi Team, I have tried to configure Signoz Alert Channel as google chat spaces webhook with below parameter. But, it has giving below error. Can you help me on the same? Channel Name: Test Channel Type: Webhook Webhook URL: https://chat.googleapis.com/v1/spaces/{SPACE_ID}/messages?key={Authkey}&token={Authtoken} Error: Failed to send a test message to this channel, please confirm that the parameters are set correctly CC: @Prakshal Shah
    p
    a
    s
    • 4
    • 18
  • b

    Bhavesh Patel

    10/10/2022, 11:59 AM
    Have you got any chance to see this issue?
  • v

    Vishal Goel

    10/10/2022, 12:16 PM
    I am not getting data of apm in signoz, getting this error. [ERROR] [init.py:156- export()] opentelemetry.exporter.otlp.proto.http.trace_exporter: Failed to export batch code: 405, reason: <html> i open 4317 and 4318 port but still not allowed. can someone help into this
    s
    • 2
    • 21
  • v

    Vishal Goel

    10/11/2022, 5:28 AM
    how i can send data in APM if my python application is using python version below 3.7
    p
    s
    • 3
    • 15
  • v

    Vishal Goel

    10/11/2022, 7:50 AM
    my user is not working with https but its working on http. when i created user, i am using http but later on i redirected it to https but its only working with http
  • k

    Kasim Ali

    10/11/2022, 8:10 AM
    How to get logs in signoz using fastapi python?
    p
    • 2
    • 3
  • n

    nitya-signoz

    10/11/2022, 10:42 AM
    These are your docker container logs which are automatically collected, you can disable by following this guide https://signoz.io/docs/userguide/collect_docker_logs/#disable-automatic-container-log-collection
    k
    • 2
    • 18
  • a

    Allan Li

    10/11/2022, 5:01 PM
    Hello! I just wanted to ask a questions regarding instrumentation for Apollo GraphQL. How would we get the .execute traces to display the SQL query string or any specific information regarding the GraphQL query that was made? Thanks ahead of time!
    p
    j
    • 3
    • 9
  • v

    Vishal Goel

    10/12/2022, 6:51 AM
    Doest it support java version 7?
    n
    • 2
    • 2
  • k

    Kasim Ali

    10/12/2022, 9:51 AM
    I am using manual and auto instrumentation for my fastapi python application, so when i have to instrumented and write code for app (1) before creating client / before creating gateway (2) after creating gateway
    p
    • 2
    • 1
  • d

    Devops Netapp

    10/13/2022, 2:26 PM
    Hi, After running helm install on AKS I get the following services:
    NAME                                      PORT(S)
    chi-my-release-clickhouse-cluster-0-0     8123/TCP,9000/TCP,9009/TCP
    my-release-clickhouse                     8123/TCP,9000/TCP
    my-release-clickhouse-operator-metrics    8888/TCP
    my-release-k8s-infra-otel-agent           13133/TCP,8888/TCP,4317/TCP,4318/TCP
    my-release-k8s-infra-otel-deployment      13133/TCP,8888/TCP,4317/TCP,4318/TCP
    my-release-signoz-alertmanager            9093/TCP
    my-release-signoz-alertmanager-headless   9093/TCP
    my-release-signoz-frontend                3301/TCP
    my-release-signoz-otel-collector          14250/TCP,14268/TCP,8888/TCP,4317/TCP,4318/TCP
    my-release-signoz-otel-collector-metrics  13133/TCP
    my-release-signoz-query-service           8080/TCP,8085/TCP
    my-release-zookeeper                      2181/TCP,2888/TCP,3888/TCP
    my-release-zookeeper-headless             2181/TCP,2888/TCP,3888/TCP
    Is this the correct configuration for sending the traces:
    otelcolConfig: {
            url: '<http://my-release-signoz-otel-collector.platform.svc.cluster.local:4318/v1/traces>', // URL of opentelemetry collector
          },
    When I enable traces to console I’m able to see the output but when configuring it to send to my-release-signoz-otel-collector service on port 4318 I get no results.
    s
    • 2
    • 3
  • a

    Allan Li

    10/13/2022, 3:26 PM
    Hello, is there a way to visualize and query traces by ranking them based on their GraphQL name?
    s
    • 2
    • 15
  • m

    Mark Raynsford

    10/14/2022, 6:15 PM
    hello! i wiped my installation from last week, cloned commit
    5eed384ffe05fd734bbbd8ae80b921106c505149
    (the current
    main
    ) and ran the usual
    docker-compose -f docker/clickhouse-setup/docker-compose.yaml up -d
    command to set up a fresh signoz. for whatever reason, i'm now unable to get into the UI (there are no existing accounts, i'm trying to create a new account):
    signoz.mkv
    p
    p
    • 3
    • 9
  • m

    Mark Raynsford

    10/14/2022, 6:15 PM
    don't worry about the silly password and fake email address, this is just for local testing on my dev machine. these steps worked fine a couple of weeks ago...
  • a

    Alexei Zenin

    10/14/2022, 8:25 PM
    Just wanted to flag that if you misconfigure the URL for clickhouse in the collector you get this error:
    panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x343dd12] goroutine 87 [running]: <http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousemetricsexporter.(*PrwExporter).export.func1()|github.com/SigNoz/signoz-otel-collector/exporter/clickhousemetricsexporter.(*PrwExporter).export.func1()> /src/exporter/clickhousemetricsexporter/exporter.go:280 +0xf2 created by <http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousemetricsexporter.(*PrwExporter).export|github.com/SigNoz/signoz-otel-collector/exporter/clickhousemetricsexporter.(*PrwExporter).export> /src/exporter/clickhousemetricsexporter/exporter.go:276 +0x256
    p
    a
    • 3
    • 5
  • u

    Upesh Jindal

    10/15/2022, 3:04 AM
    Hi, Is the following a known issue: "opentelemetry.sdk.metrics._internal.instrument", "funcName": "callback", "lineno": 139, "message": "Callback failed for instrument system.swap.utilization.", "exc_info": "Traceback (most recent call last):\n File \"/usr/local/lib/python3.10/site-packages/opentelemetry/sdk/metrics/_internal/instrument.py\", line 132, in callback\n for api_measurement in callback(callback_options):\n File \"/usr/local/lib/python3.10/site-packages/opentelemetry/instrumentation/system_metrics/__init__.py\", line 423, in _get_system_swap_utilization\n getattr(system_swap, metric) / system_swap.total,\nZeroDivisionError: division by zero
    s
    • 2
    • 3
  • a

    Ashish

    10/17/2022, 11:37 AM
    Hi <!channel> I am facing issue in configuring my DB monitoring, can anyone help me regarding this issue ?
  • a

    Ashish

    10/17/2022, 12:52 PM
    Hi <!channel> Does anyone know how to check if my mongodb exporter is able to communicate with signoz
    s
    s
    • 3
    • 4
  • a

    Akhilesh Bussa

    10/17/2022, 1:05 PM
    Hi Team, I have setup signoz in AWS using docker. I have configured the OTel in my django application, although I can see OTel logs, I cant see the same on signoz dashboard. I have also tried troubleshooting endpoint and it was successful, but there are no traces/data on the dashboard (Screenshots for reference) What am I missing here?
    s
    • 2
    • 16
  • a

    Akhilesh Bussa

    10/17/2022, 4:20 PM
    Hi team, I was trying to setup with Django + uWSGI with this command and the postfork decorator as mentioned in the example provided:
    OTEL_RESOURCE_ATTRIBUTES=service.name=hancock_staging OTEL_EXPORTER_OTLP_ENDPOINT="<http://MySignozIP:4318>"  opentelemetry-instrument --traces_exporter otlp_proto_http /home/hancock/env/bin/uwsgi --ini /home/hancock/config/api_uwsgi.ini
    But I get this error:
    File "/home/hancock/env/lib64/python3.7/site-packages/uwsgidecorators.py", line 10, in <module>
        import uwsgi
    ModuleNotFoundError: No module named 'uwsgi'
    Can someone help me?
  • b

    Bhavesh Patel

    10/18/2022, 3:56 AM
    Hi Team, I want to scale the Signoz infrastructure for handle the heavy load. Can you help us with the same?
    n
    v
    a
    • 4
    • 7
  • b

    Bhavesh Patel

    10/18/2022, 6:14 AM
    Also, which database we are using for storing data in signoz?
    s
    • 2
    • 2
  • k

    Kasim Ali

    10/18/2022, 8:17 AM
    currently i am trying to integrate signoz with my django app it can work properly but i have one issu with that i can get traces but receive traces are independently not with child and parent graph.
    • 1
    • 1
  • d

    Daniel Nordberg

    10/18/2022, 12:09 PM
    Am following the tutorial to get Django instrumentation running, I installed dependencies, ran
    opentelemetry-bootstrap --action=install
    , then started the server like so
    OTEL_RESOURCE_ATTRIBUTES=service.name=django_app OTEL_EXPORTER_OTLP_ENDPOINT="<http://localhost:4318>"  opentelemetry-instrument --traces_exporter otlp_proto_http python manage.py runserver --noreload
    , however I can't reach http://localhost:4318
    s
    • 2
    • 1
  • d

    Daniel Nordberg

    10/18/2022, 12:10 PM
    On another note, do I have to run the docker stack on the same network
  • ł

    Łukasz Herman

    10/18/2022, 12:16 PM
    Hi, can you help me with configuring fluentD to send correct data? Currently I'm working on two cases: • IIS Logs • Log4Net application logs With IIS I have almost everything as I would like to. The only problem I see that I would like to store some fields as numeric/int and IP address. Currently all is stored as a string. With Log4Net I need to build grok parsing and I'm lost with it... In Log4Net we are using configuration: %d [%t][%P{vRequestId}]*[%P{vSqlServer}]*[%P{vUserName}][%P{vInstance}] %-5p %c - %m%n vRootUrl:%P{vRootUrl} %n vRequestUrl:%P{vRequestUrl} %n Body:%P{vLogExtension}%n And I have no idea how to build grok filter for it.
    a
    n
    • 3
    • 20
Powered by Linen
Title
ł

Łukasz Herman

10/18/2022, 12:16 PM
Hi, can you help me with configuring fluentD to send correct data? Currently I'm working on two cases: • IIS Logs • Log4Net application logs With IIS I have almost everything as I would like to. The only problem I see that I would like to store some fields as numeric/int and IP address. Currently all is stored as a string. With Log4Net I need to build grok parsing and I'm lost with it... In Log4Net we are using configuration: %d [%t][%P{vRequestId}]*[%P{vSqlServer}]*[%P{vUserName}][%P{vInstance}] %-5p %c - %m%n vRootUrl:%P{vRootUrl} %n vRequestUrl:%P{vRequestUrl} %n Body:%P{vLogExtension}%n And I have no idea how to build grok filter for it.
a

Ankit Nayan

10/18/2022, 1:07 PM
@nitya-signoz have you worked with grok? Also, can we do the parsing at otel-collector?
n

nitya-signoz

10/18/2022, 3:55 PM
Have a little idea but would like to understand the problem better. @Łukasz Herman can you please create a GitHub issue with the configuration that you are currently using for both problems, and also share some example logs. While we work towards a solution it will help the community as well.
ł

Łukasz Herman

10/20/2022, 2:14 PM
Where should we parse the logs? Currently I understand that fluentD can use Grok to parse log and slice it into different fields. But on the SigNoz side all of them are strings. Is there any other way to parse logs (on SigNoz side)?
@nitya-signoz @Ankit Nayan Where should I parse logs? In general it will be more suitable for me to do all parsing in one central location like SigNoz. But how? And when somehow I will parse the logs, how to change datatypes (to int, IP, decimal, etc.)?
n

nitya-signoz

10/25/2022, 8:47 AM
Yes, Ideally all the parsing should be done in the otel-collector-config.yaml file. For parsing the logs there are different operators available https://signoz.io/docs/userguide/logs/#operators-for-parsing-and-manipulating-logs
If you can share some example log lines, we can help you with how to parse it.
ł

Łukasz Herman

10/25/2022, 9:05 AM
Lets focus on log like below: { "timestamp": 1666659631502000000, "id": "2FlLy8mdGBzFof0oKEZy4x4jtMg", "trace_id": "", "span_id": "", "trace_flags": 0, "severity_text": "", "severity_number": 0, "body": ""[216][638022636288199459][SQL123\\SQL2019B][System][saas] INFO Core.WebTasks.Esb.WebTaskJobProcessor - Zaczynam WebTask:AfterUpgrade (saas)\r\n vRootUrl:https://fake.org/ \r\n vRequestUrl:https://fake.org/? \r\n Body:(null)"", "resources_string": {}, "attributes_string": { "app": "app1", "environment": "UAT", "fluent_tag": "log4net", "hostname": "AP20", "logfile": "info.log", "logfilepath": "D:/Data/info.log", "module": "app1" }, "attributes_int": {}, "attributes_float": {} } I would like to parse 'body' and extract some fields like thread in "[", "]" (216), and so on.
n

nitya-signoz

10/25/2022, 9:16 AM
This is something that you have taken from the signoz UI. Can you share a raw log line from the file/source where you are getting the data, as some operators are already applied here.
ł

Łukasz Herman

10/25/2022, 9:22 AM
I'm grabbing file logs using FluentD and sending to fluentforward receiver. As I would like to parse them on SigNoz side, it doesn't matter what are source logs.
n

nitya-signoz

10/25/2022, 9:25 AM
Yeah if you are using fluentD then source logs format doesn’t matter, but the output format of fluentD matters. you can use https://docs.fluentd.org/output/stdout to check the output that is going to signoz.
ł

Łukasz Herman

10/25/2022, 10:07 AM
On Windows FluentD is working as a service. I'm not able to get standard output. I can save it to file, but it will the same as saving to fluentforward, isn't it?
n

nitya-signoz

10/25/2022, 11:41 AM
Yeah it will give me us an idea about whats happening with the logs in fluentD before it’s going to a dowstream.
ł

Łukasz Herman

10/26/2022, 7:34 AM
I was not able to send log to file (it is failing and I'm not able to figure out why) but it's logging in FluentD log showing what data it has. "2022-10-26T09:22:54+02:00\tlog4net\t{\"message\":\"[46] DEBUG - Init WindsorHttpModule\",\"logfilepath\":\"D:/Data/App1/Komunikacja/info.txt\",\"hostname\":\"AP20\",\"app\":\"App1\",\"module\":\"Komunikacja\",\"logfile\":\"info.txt\",\"environment\":\"UAT\"}\r\n"
It's more readable when I grab what is in FluentD buffer. For log4Net: 2022-10-26T09:09:39+02:00 log4net {"message":"[52] DEBUG - Dispose RequestTimerModule","logfilepath":"D:/Data/App1/Administracja/info.txt","hostname":"AP20","app":"App1","module":"Administracja","logfile":"info.txt","environment":"UAT"} For IIS: 2022-10-26T09:37:44+02:00 iis.arr {"message":"W3SVC1 ARR2 10.50.60.122 POST /App1Wcf/App1WcfService.svc X-ARR-CACHE-HIT=0&SERVER-ROUTED=AP23C&X-ARR-LOG-ID=01dcb691-a6c9-4c6d-a30f-9fa6ee484b0b&SERVER-STATUS=200 443 - 10.50.60.163 HTTP/1.1 - - App1.uat 200 0 0 3581 796 11 10.50.60.163:50137","s-sitename":"W3SVC1","s-computername":"ARR2","s-ip":"10.50.60.122","cs-method":"POST","cs-uri-stem":"/App1Wcf/App1WcfService.svc","cs-uri-query":"X-ARR-CACHE-HIT=0&amp;SERVER-ROUTED=AP23C&amp;X-ARR-LOG-ID=01dcb691-a6c9-4c6d-a30f-9fa6ee484b0b&amp;SERVER-STATUS=200","s-port":"443","cs-username":null,"c-ip":"10.50.60.163","cs-version":"HTTP/1.1","cs(User-Agent)":null,"cs(Referer)":null,"cs-host":"App1.uat","sc-status":"200","sc-substatus":"0","sc-win32-status":"0","sc-bytes":"3581","cs-bytes":"796","time-taken":"11","OriginalIP":"10.50.60.163:50137","app":"arr","type":"iis","environment":"UAT"}
n

nitya-signoz

10/26/2022, 8:45 AM
Got it, let me get back to you on the parsing steps for parsing the above logs.
@Łukasz Herman so from the data that you have shared,
message
is the actual log that is being scraped and you are parsing that in fluentD to extract different values such as
sc-bytes
,
sc-status
etc. Now since these values are parsed as a string in fluentD itself, it is stored as string in SigNoz. When you are parsing the log in fluentD you can specify the types parameter https://docs.fluentd.org/configuration/parse-section#types-parameter to parse to a specific type.
ł

Łukasz Herman

10/27/2022, 1:01 PM
Ok, thanks. What for log4net example, when I would like to parse it outside fluentD? It's more suitable for me to parse it at destination.
n

nitya-signoz

10/27/2022, 1:12 PM
The current version of signoz otel collector doesn’t support type casting. Once this issue is fixed https://github.com/SigNoz/signoz-otel-collector/issues/19#issuecomment-1293276797 , you will be able to typecast string values to int.
ł

Łukasz Herman

10/27/2022, 1:25 PM
Because of new version of Fluent Bit (2.0 looks like solving most demanding tasks to have one tool to grab all logs and metrics) I'm wondering if it will best to put few Fluent Bit servers in beetween source and destination for parsing and filtering. Then the whole stack will be http based as I can send all data in Otel format. (I like HTTP more than TCP because I can send all through our load balancers and have everything in control). Then the pipeline will be Source servers with FluentBit (many servers in many datacenters) -> (Maybe) Central local FluentBit per Environment/Datacenter -> Load balancer (entrance to main datacenter where all data will be stored) -> FluenBit farm (scailing servers easily, performing all parsing and basic filtering) -> Load balancer -> SigNoz Otel Collector I need to check If Otel input and output in FluentBit are working as I think and I they can grab data I would like to have. What do you think about that?
n

nitya-signoz

10/27/2022, 2:38 PM
Keeping the parsing stuff separate from ingestion machines will be definitely a better idea. Though you can replace the intermediate ones with otel-collectors instead of fluentbit if you are parsing the data at the source fluentBit. Since at the end when fluentD/fluentBit sends data to signoz you might have to transform a few things and doing it before it reaches the final signoz-otel-collector is a better idea.
View count: 1