Hi,
I made my fastapi app to log data into json fo...
# supportm
Muhammed Hussein Karimi
03/01/2023, 3:01 PMHi,
I made my fastapi app to log data into json format
also I added
span_idtrace_idint Copy code
import json_logging
import logging
import sys
from config import config
from fastapi import FastAPI
app = FastAPI(title=config.app_title)
class JSONLogWebFormatter(json_logging.JSONLogFormatter):
"""
Formatter for web application log
"""
def _format_log_object(self, record, request_util):
json_log_object = super(JSONLogWebFormatter, self)._format_log_object(
record, request_util
)
current_span = trace.get_current_span()
if current_span:
json_log_object["span_id"] = current_span.get_span_context().span_id
json_log_object["trace_id"] = current_span.get_span_context().trace_id
json_log_object.update(
{
"correlation_id": request_util.get_correlation_id(
within_formatter=True
),
}
)
return json_log_object
json_logging.init_fastapi(enable_json=True, custom_formatter=JSONLogWebFormatter)
json_logging.init_request_instrument(app, custom_formatter=JSONLogWebFormatter)
# init the logger as usual
logger = logging.getLogger()
logger.setLevel(logging._nameToLevel[config.log_level.upper()])
logger.addHandler(logging.StreamHandler(sys.stdout))n
nitya-signoz
03/01/2023, 3:03 PMYou will have to use the SDK https://github.com/open-telemetry/opentelemetry-python/tree/main/docs/examples/logs
nitya-signoz
03/01/2023, 3:04 PMOr If your application is running in docker or k8s, then logs emitted to stdout will be collected by signoz
m
Muhammed Hussein Karimi
03/01/2023, 3:05 PMI'm in k8s and other logs are collecting except this app
Muhammed Hussein Karimi
03/01/2023, 3:06 PMI think something are wrong with my log structure
Muhammed Hussein Karimi
03/01/2023, 3:08 PMOr with my configuration
Muhammed Hussein Karimi
03/01/2023, 3:12 PM@nitya-signoz What can I do to fix that?
n
nitya-signoz
03/01/2023, 3:14 PMIf you logs are getting printed to stdout then it will be collected, did you try to filter in the UI
m
Muhammed Hussein Karimi
03/01/2023, 3:14 PMyes
Muhammed Hussein Karimi
03/01/2023, 3:15 PMnothing shows in the SigNoz dashboard after filtering
Muhammed Hussein Karimi
03/01/2023, 3:56 PM Copy code
{
"written_at": "2023-03-01T15:56:00.206Z",
"written_ts": 1677686160206573000,
"msg": "",
"type": "log",
"logger": "fastapi-request-logger",
"thread": "MainThread",
"level": "INFO",
"module": "implementation",
"line_no": 35,
"span_id": "176794658234006051",
"trace_id": "227587342002673840811008499138396058579",
"correlation_id": "272ba70209901ac67929a2595244ce13"
}Muhammed Hussein Karimi
03/01/2023, 3:56 PMhere is the sample log
Muhammed Hussein Karimi
03/01/2023, 4:11 PM@nitya-signoz
Muhammed Hussein Karimi
03/04/2023, 2:56 PMI have removed trace_id and span_id from logs but I'm not able to see logs in signoz
Muhammed Hussein Karimi
03/04/2023, 2:57 PM@nitya-signoz
n
nitya-signoz
03/05/2023, 3:42 PMCan you check if by any change if the namespace where you have deployed the above application is excluded in the otel collector config file receiver ?
m
Muhammed Hussein Karimi
03/05/2023, 8:55 PM Copy code
- type: filter
id: remove_some_logs
expr: 'resource["k8s.namespace.name"] matches "(^stage-.*$)" and resource["k8s.namespace.name"] != "stage-this-project"'
drop_ratio: 1.0Muhammed Hussein Karimi
03/05/2023, 8:56 PMstage-this-project is the name of the namespace
Muhammed Hussein Karimi
03/05/2023, 8:56 PMI think this filter is not making problems
n
nitya-signoz
03/06/2023, 5:18 AMIt shouldn’t cause any issue, but try removing it?
nitya-signoz
03/06/2023, 5:19 AMor if there is something else.
m
Muhammed Hussein Karimi
03/06/2023, 5:22 AMI will check
But there are no other drop filters
If I remove that my storage will become 100% full
n
nitya-signoz
03/06/2023, 5:22 AMGot it then try deploying in some other namesapce ?
m
Muhammed Hussein Karimi
03/06/2023, 6:58 AMI will test that
Muhammed Hussein Karimi
03/06/2023, 7:27 AMI changed the namespace but problem persist
Muhammed Hussein Karimi
03/07/2023, 10:56 AM@nitya-signoz
76 Views