How do you log server errors on django sites

PythonDjangoError Logging

Python Problem Overview


So, when playing with the development I can just set settings.DEBUG to True and if an error occures I can see it nicely formatted, with good stack trace and request information.

But on kind of production site I'd rather use DEBUG=False and show visitors some standard error 500 page with information that I'm working on fixing this bug at this moment ;)
At the same time I'd like to have some way of logging all those information (stack trace and request info) to a file on my server - so I can just output it to my console and watch errors scroll, email the log to me every hour or something like this.

What logging solutions would you recomend for a django-site, that would meet those simple requirements? I have the application running as fcgi server and I'm using apache web server as frontend (although thinking of going to lighttpd).

Python Solutions


Solution 1 - Python

Well, when DEBUG = False, Django will automatically mail a full traceback of any error to each person listed in the ADMINS setting, which gets you notifications pretty much for free. If you'd like more fine-grained control, you can write and add to your settings a middleware class which defines a method named process_exception(), which will have access to the exception that was raised:

http://docs.djangoproject.com/en/dev/topics/http/middleware/#process-exception

Your process_exception() method can then perform whatever type of logging you'd like: writing to console, writing to a file, etc., etc.

Edit: though it's a bit less useful, you can also listen for the got_request_exception signal, which will be sent whenever an exception is encountered during request processing:

http://docs.djangoproject.com/en/dev/ref/signals/#got-request-exception

This does not give you access to the exception object, however, so the middleware method is much easier to work with.

Solution 2 - Python

Django Sentry is a good way to go, as already mentioned, but there is a bit of work involved in setting it up properly (as a separate website). If you just want to log everything to a simple text file here's the logging configuration to put in your settings.py

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        # Include the default Django email handler for errors
        # This is what you'd get without configuring logging at all.
        'mail_admins': {
            'class': 'django.utils.log.AdminEmailHandler',
            'level': 'ERROR',
             # But the emails are plain text by default - HTML is nicer
            'include_html': True,
        },
        # Log to a text file that can be rotated by logrotate
        'logfile': {
            'class': 'logging.handlers.WatchedFileHandler',
            'filename': '/var/log/django/myapp.log'
        },
    },
    'loggers': {
        # Again, default Django configuration to email unhandled exceptions
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
        # Might as well log any errors anywhere else in Django
        'django': {
            'handlers': ['logfile'],
            'level': 'ERROR',
            'propagate': False,
        },
        # Your own app - this assumes all your logger names start with "myapp."
        'myapp': {
            'handlers': ['logfile'],
            'level': 'WARNING', # Or maybe INFO or DEBUG
            'propagate': False
        },
    },
}

Solution 3 - Python

django-db-log, mentioned in another answer, has been replaced with:

https://github.com/dcramer/django-sentry

Solution 4 - Python

Obviously James is correct, but if you wanted to log exceptions in a datastore, there are a few open source solutions already available:

  1. CrashLog is a good choice: http://code.google.com/p/django-crashlog/

  2. Db-Log is a good choice as well: http://code.google.com/p/django-db-log/

What is the difference between the two? Almost nothing that I can see, so either one will suffice.

I've used both and they work well.

Solution 5 - Python

Some time has passed since EMP's most helpful code submission. I just now implemented it, and while thrashing around with some manage.py option, to try to chase down a bug, I got a deprecation warning to the effect that with my current version of Django (1.5.?) a require_debug_false filter is now needed for the mail_admins handler.

Here is the revised code:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'filters': {
         'require_debug_false': {
             '()': 'django.utils.log.RequireDebugFalse'
         }
     },
    'handlers': {
        # Include the default Django email handler for errors
        # This is what you'd get without configuring logging at all.
        'mail_admins': {
            'class': 'django.utils.log.AdminEmailHandler',
            'level': 'ERROR',
            'filters': ['require_debug_false'],
             # But the emails are plain text by default - HTML is nicer
            'include_html': True,
        },
        # Log to a text file that can be rotated by logrotate
        'logfile': {
            'class': 'logging.handlers.WatchedFileHandler',
            'filename': '/home/username/public_html/djangoprojectname/logfilename.log'
        },
    },
    'loggers': {
        # Again, default Django configuration to email unhandled exceptions
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': True,
        },
        # Might as well log any errors anywhere else in Django
        'django': {
            'handlers': ['logfile'],
            'level': 'ERROR',
            'propagate': False,
        },
        # Your own app - this assumes all your logger names start with "myapp."
        'myapp': {
            'handlers': ['logfile'],
            'level': 'DEBUG', # Or maybe INFO or WARNING
            'propagate': False
        },
    },
}

Solution 6 - Python

I just had an annoying problem with my fcgi script. It occurred before django even started. The lack of logging is sooo painful. Anyway, redirecting stderr to a file as the very first thing helped a lot:

#!/home/user/env/bin/python
sys.stderr = open('/home/user/fcgi_errors', 'a')

Solution 7 - Python

You can use the logging library in Python, no need to pip install anything.

Replace any print() with logging.debug() but,

> Django Sentry is a good way to go

as EMP said.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionkenderView Question on Stackoverflow
Solution 1 - PythonJames BennettView Answer on Stackoverflow
Solution 2 - PythonEMPView Answer on Stackoverflow
Solution 3 - PythontobychView Answer on Stackoverflow
Solution 4 - PythonmontyloungeView Answer on Stackoverflow
Solution 5 - PythonMike O'ConnorView Answer on Stackoverflow
Solution 6 - PythonjozxyqkView Answer on Stackoverflow
Solution 7 - Pythonlam vu NguyenView Answer on Stackoverflow