WSGI extensions to zope2instance recipe/support for "mailinglogger"?

Hi there,

recently I enhanced the logger support for WSGI setups in zope2instance recipe by adding support for arbitrary logging handler (bringing back log rotation, timed log rotation).

I am now working in WSGI support for a mailing logger functionality. I have an experimental branch

which is based on the logging.handlers.SMTPHandler.

The mailinglogger functionality of ZServer is/was based on https://pypi.org/project/mailinglogger/

Question: would you like to see out of the box support for the mailinglogger in zope2instance (because it providers a more rich functionality over Python's SMTPHandler) or is supporting the logging.handlers.SMTPHandler good enough? I have some resources for doing the work but I would like to see a recommendation from you in order to get this feature merged upstream later.

-aj

1 Like

a note from me, I have patched the MaillingLogger.py locally. Because The Auth-Process fails on our Exchange, here is my Code

# Copyright (c) 2004-2007 Simplistix Ltd
# Copyright (c) 2001-2003 New Information Paradigms Ltd
#
# This Software is released under the MIT License:
# http://www.opensource.org/licenses/mit-license.html
# See license.txt for more details.

import datetime
import os
import smtplib
import socket

from email.Utils import formatdate, make_msgid
from email.MIMEText import MIMEText
from logging.handlers import SMTPHandler
from logging import LogRecord, CRITICAL
from mailinglogger.common import SubjectFormatter
from mailinglogger.common import process_ignore


this_dir = os.path.dirname(__file__)
x_mailer = 'MailingLogger '+open(os.path.join(this_dir,'version.txt')).read().strip()
flood_template = open(os.path.join(this_dir,'flood_template.txt')).read()

class MailingLogger(SMTPHandler):

    now = datetime.datetime.now
    
    def __init__(self,
                 fromaddr,
                 toaddrs,
                 mailhost='localhost',
                 subject='%(line)s',
                 send_empty_entries=False,
                 flood_level=10,
                 username=None,
                 password=None,
                 ignore=(),
                 headers=None,
                 template=None,
                 charset='utf-8',
                 content_type='text/plain'):
        SMTPHandler.__init__(self,mailhost,fromaddr,toaddrs,subject)
        self.subject_formatter = SubjectFormatter(subject)
        self.send_empty_entries = send_empty_entries
        self.flood_level = flood_level
        self.hour = self.now().hour
        self.sent = 0
        self.username = username
        self.password = password
        self.ignore = process_ignore(ignore)
        self.headers = headers or {}
        self.template = template
        self.charset = charset
        self.content_type = content_type
        if not self.mailport:
            self.mailport = smtplib.SMTP_PORT

    def getSubject(self,record):
        return self.subject_formatter.format(record)

    def emit(self,record):
        msg = record.getMessage()
        if not self.send_empty_entries and not msg.strip():
            return

        for criterion in self.ignore:
            if criterion(msg):
                return

        current_time = self.now()
        current_hour = current_time.hour
        if current_hour != self.hour:
            self.hour = current_hour
            self.sent = 0
        if self.sent == self.flood_level:
            # send critical error
            record = LogRecord(
                name = 'flood',
                level = CRITICAL,
                pathname = '',
                lineno = 0,
                msg = flood_template % (self.sent,
                                        current_time.strftime('%H:%M:%S'),
                                        current_hour+1),
                args = (),
                exc_info = None)
        elif self.flood_level and self.sent > self.flood_level:
            # do nothing, we've sent too many emails already
            return
        self.sent += 1

        # actually send the mail
        try:
            msg = self.format(record)
            if self.template is not None:
                msg = self.template % msg
            subtype = self.content_type.split('/')[-1]
            if isinstance(msg, unicode):
                email = MIMEText(msg, subtype, self.charset)
            else:
                email = MIMEText(msg, subtype)
                
            for header,value in self.headers.items():
                email[header]=value
            email['Subject']=self.getSubject(record)
            email['From']=self.fromaddr
            email['To']=', '.join(self.toaddrs)
            email['X-Mailer']=x_mailer
            email['X-Log-Level']=record.levelname
            email['Date']=formatdate()
            email['Message-ID']=make_msgid('MailingLogger')
            smtp = smtplib.SMTP(self.mailhost, self.mailport)
            if self.username and self.password:            
                # this is the patch - 3 lines code
                smtp.ehlo()
                smtp.starttls()
                smtp.ehlo()
                # end patch
                smtp.login(self.username,self.password)
            smtp.sendmail(self.fromaddr, self.toaddrs, email.as_string())
            smtp.quit()
        except:
            self.handleError(record)

It looks like @zopyx's branch was merged but documentation on how to use it was not updated in plone.recipe.zope2instance? Is that an oversight or is configuring mail logging in WSGI discouraged right now? I'm really just looking for a simple email error setup like we had with ZServer, nothing fancy like Sentry.

If there is a particular issue, please file a bugreport and assign it to me. I am happy to check this after vacation.

I looked into configuring the log rotation a while ago, this is the imho nicest setup I could come up with:

event-log-handler = logging.handlers.TimedRotatingFileHandler
event-log-args = ("${buildout:directory}/var/log/${:_buildout_section_name_}.log",)
event-log-kwargs = {"when":"W6", "backupCount":8 }

access-log-handler = logging.handlers.TimedRotatingFileHandler
access-log-args = ("${buildout:directory}/var/log/${:_buildout_section_name_}-Z2.log",)
access-log-kwargs = {"when":"W6", "backupCount":8 }

It starts the logs on monday and ends them sunday evening:

Sep 18 11:48 zeoclient-Z2.log
Aug 23 23:59 zeoclient-Z2.log.2020-08-17
Aug 30 23:59 zeoclient-Z2.log.2020-08-24
Sep  6 23:59 zeoclient-Z2.log.2020-08-31
Sep 13 23:59 zeoclient-Z2.log.2020-09-07

Only issue left is that older files are not compressed. Could add a cronjob with small bash script which finds older files in /var/log without .gz and gzips them.

The WSGI related configurations through zope2instance recipe are possibly the most simple hack you can get at this point. Providing a custom wsgi.ini template file is likely the better choice.

Any examples what you have/would put in a custom wsgi.ini to get sane logging defaults from the zeoclients?

Not really. But the generated wsgi.ini appears as a good starting point. I would modify the wsgi.ini locally as needed and then include it with the buildout.