https://signoz.io logo
Join Slack
Powered by
Hi all, Is anyone using gunicorn with gevent worke...
# support
s
Hi all, Is anyone using gunicorn with gevent worker class along with open telemetry auto instrumentation? We are using a django app with gunicorn Is it not yet compatible?
I was trying to monkey patch in post fork but when starting with opentelemetry-instrument. It throws a lot of assertionerrors
r
might be good to share your logs
s
Copy code
Exception in worker
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 81, in _worker
    work_item = work_queue.get(block=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/gevent/queue.py", line 381, in gevent._gevent_cqueue.SimpleQueue.get
  File "src/gevent/queue.py", line 397, in gevent._gevent_cqueue.SimpleQueue.get
  File "src/gevent/queue.py", line 373, in gevent._gevent_cqueue.SimpleQueue._SimpleQueue__get_or_peek
  File "src/gevent/_waiter.py", line 154, in gevent._gevent_c_waiter.Waiter.get
  File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
gevent.exceptions.LoopExit: This operation would block forever
        Hub: <Hub '' at 0xffff97c537e0 epoll pending=0 ref=0 fileno=11 thread_ident=0xffff97813950>
        Handles:
[]
Overriding of current TracerProvider is not allowed
Overriding of current MeterProvider is not allowed
[2025-06-07 03:26:11 +0000] [97] [INFO] Booting worker with pid: 97
[2025-06-07 03:26:11 +0000] [97] [INFO] Worker spawned (pid: 97)
Exception in worker
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 81, in _worker
    work_item = work_queue.get(block=True)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/gevent/queue.py", line 381, in gevent._gevent_cqueue.SimpleQueue.get
  File "src/gevent/queue.py", line 397, in gevent._gevent_cqueue.SimpleQueue.get
  File "src/gevent/queue.py", line 373, in gevent._gevent_cqueue.SimpleQueue._SimpleQueue__get_or_peek
  File "src/gevent/_waiter.py", line 154, in gevent._gevent_c_waiter.Waiter.get
  File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
  File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
gevent.exceptions.LoopExit: This operation would block forever
        Hub: <Hub '' at 0xffff97c53880 epoll pending=0 ref=0 fileno=10 thread_ident=0xffff9780f950>
        Handles:
[]
Overriding of current TracerProvider is not allowed
Overriding of current MeterProvider is not allowed
/usr/local/lib/python3.11/site-packages/gunicorn/workers/ggevent.py:38: MonkeyPatchWarning: Patching more than once will result in the union of all True parameters being patched
  monkey.patch_all()
/usr/local/lib/python3.11/site-packages/gunicorn/workers/ggevent.py:38: MonkeyPatchWarning: Patching more than once will result in the union of all True parameters being patched
  monkey.patch_all()
Exception in thread OtelBatchLogRecordProcessor:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.11/site-packages/opentelemetry/instrumentation/threading/__init__.py", line 129, in __wrap_threading_run
    return call_wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/export/__init__.py", line 250, in worker
    self._notify_flush_request_finished(flush_request)
  File "/usr/local/lib/python3.11/site-packages/opentelemetry/sdk/_logs/_internal/export/__init__.py", line 337, in _notify_flush_request_finished
    flush_request.event.set()
  File "src/gevent/event.py", line 116, in gevent._gevent_cevent.Event.set
  File "src/gevent/_abstract_linkable.py", line 232, in gevent._gevent_c_abstract_linkable.AbstractLinkable._check_and_notify
  File "src/gevent/_abstract_linkable.py", line 347, in gevent._gevent_c_abstract_linkable.AbstractLinkable._notify_links
  File "src/gevent/_abstract_linkable.py", line 397, in gevent._gevent_c_abstract_linkable.AbstractLinkable._handle_unswitched_notifications
  File "src/gevent/_abstract_linkable.py", line 54, in gevent._gevent_c_abstract_linkable.get_roots_and_hubs
ReferenceError: weakly-referenced object no longer exists
Exception ignored in thread started by: <bound method Thread._bootstrap of <Thread(OtelBatchLogRecordProcessor, started daemon 281473219483984)>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/threading.py", line 1002, in _bootstrap
    self._bootstrap_inner()
  File "/usr/local/lib/python3.11/threading.py", line 1049, in _bootstrap_inner
    self._delete()
  File "/usr/local/lib/python3.11/threading.py", line 1081, in _delete
    del _active[get_ident()]
        ~~~~~~~^^^^^^^^^^^^^
KeyError: 281473212345920
Like, I also want to know if anyone's using gevent with gunicorn and using open telemetry instrumentation and if it's a good idea
@Srikanth Chekuri, Maybe you can help to share your views since you have experience in this area
Also, I'm using wsgi only. Not asgi
r
that looks like a module ordering issue.. with the threads.. looks suspiciously like this : https://stackoverflow.com/a/12639040
s
Will check. Thanks
10 Views
Open in Slack
PreviousNext
Powered by