SMTP connection timed out with OVH service

Plone Support
1

I have an error trying to set the mailer in my Plone v5.0.6.

The settings are provided by my service provider OVH (vastly used in France).

The error message is :

Connection unexpectedly closed: timed out.

in the instance.log, I have a bit more information:

 ------
2017-06-25T17:12:03 ERROR Plone Unable to send test e-mail.
Traceback (most recent call last):
  File "/opt/Plone/buildout-cache/eggs/Products.CMFPlone-5.0.6-py2.7.egg/Products/CMFPlone/controlpanel/browser/mail.py", line 82, in handle_test_action
    immediate=True)
  File "/opt/Plone/buildout-cache/eggs/Products.MailHost-2.13.2-py2.7.egg/Products/MailHost/MailHost.py", line 237, in send
    self._send(mfrom, mto, messageText, immediate)
  File "/opt/Plone/buildout-cache/eggs/Products.MailHost-2.13.2-py2.7.egg/Products/MailHost/MailHost.py", line 337, in _send
    self._makeMailer().send(mfrom, mto, messageText)
  File "/opt/Plone/buildout-cache/eggs/Products.CMFPlone-5.0.6-py2.7.egg/Products/CMFPlone/patches/sendmail.py", line 17, in _catch
    return func(*args, **kwargs)
  File "/opt/Plone/buildout-cache/eggs/zope.sendmail-3.7.5-py2.7.egg/zope/sendmail/mailer.py", line 46, in send
    connection = self.smtp(self.hostname, str(self.port))
  File "/usr/lib/python2.7/smtplib.py", line 256, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib/python2.7/smtplib.py", line 317, in connect
    (code, msg) = self.getreply()
  File "/usr/lib/python2.7/smtplib.py", line 365, in getreply
    + str(e))
SMTPServerDisconnected: Connection unexpectedly closed: timed out

I'm sure the settings are correct, and I managed to contact the SMTP server with telnet from my Plone server:

 # telnet ssl0.ovh.net 587
Trying 213.186.33.20...
Connected to ssl0.ovh.net.
Escape character is '^]'.
220-ssl0.ovh.net player770

So this leads me to some problem coming from the Plone stack

Any Idea ?

Thanks,

Jonathan

2

Port 587 sounds like MSA. Are we sure that MSA/587 is support on the Python smtplib level?
But could be a network issue in addition.

-aj

3

The only thing a bit strange is that the port is converted into an str. Apart from that there is not much, Plone/smtplib can do wrong.

I would use smtplib directly from an interactive Python session and see whether it is able to connect to your mail server -- with integer and with str port.

4

I'm quite certain I've used port 587 successfully before. Just trying now on 5.0.6 I get at least

SMTPAuthenticationError: (535, '5.7.8 Bad username or password (Authentication failed).')

(the SMTP server I'm trying seems to require 2FA now), not a connection timeout.

5

Hi,

it's great to have replies on my problem, thanks !

Her is the python test, with port as integer:

# /opt/Plone/zinstance/bin/python
Python 2.7.9 (default, Jun 29 2016, 13:08:31) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import smtplib
>>> smtp = smtplib.SMTP()
>>> smtp.connect('ssl0.ovh.net', 587)
(220, 'ssl0.ovh.net player715\nssl0.ovh.net player715')
>>> smtp.close
<bound method SMTP.close of <smtplib.SMTP instance at 0x7f11c37f0dd0>>

And with port as string, it seems to hang:

>>> smtp.connect('ssl0.ovh.net', '587')

Third test, with another SMTP server (which works in Plone):

>>> smtp.connect('mail.gmx.com', 587)
(220, 'gmx.com (mrgmx102) Nemesis ESMTP Service ready')
>>> smtp.connect('mail.gmx.com', '587')
421, 'gmx.com Service closing transmission channel - command timeout')

times out as well. So this seems interesting.

The question is why in the Plone setting, the port is passed as a string and not as an integer, while for other settings, it is ?

6

The seems to be a bug in zope.sendmail (Python AF_INET sockets expect an integer port specification).

I assume that Plone is not always using zope.sendmail (but only when you are using quued delivery).

A monkey patch for zope.sendmail.mailer.SMTPMailer replacing the class attribute smtp by a class derived from smtplib.SMTP which supports str values port specifications (not only int's) would solve your problem
(of course, a better solution would be to fix the bug in zope.sendmail).

7

Sorry, it does'nt make sense to me.

Why do I have this error in the plone control panel for the ssl0.ovh.net server and not for the mail.gmx.com server ?

Why would the port be a string for one and not for the other ?

8

I do not know. My guess has been that one is using zope.sendmail while the other does not (because Plone may use zope.sendmail only for "queued delivery" - this would explain that configuration differences may or may not cause a timeout).

I would use debugging to find out precisely what happens. You could add a code breakpoint (import pdb; pdb.set_trace()) in zope.sendmail.mailer.SMTPMailer.send (there is the bug) to check whether you come there only in the problematic case. Should you always come there, then under some conditions even an str port specification might be acceptable (but not always).

9

dieter: Python AF_INET sockets expect an integer port specification

I don't think it is correct that the port parameter has to be an integer.
The cpython code says:

if (PyLong_CheckExact(pobj)) {
    long value = PyLong_AsLong(pobj);
    if (value == -1 && PyErr_Occurred())
        goto err;
    PyOS_snprintf(pbuf, sizeof(pbuf), "%ld", value);
    pptr = pbuf;
} else if (PyUnicode_Check(pobj)) {
    pptr = PyUnicode_AsUTF8(pobj);
    if (pptr == NULL)
        goto err;
} else if (PyBytes_Check(pobj)) {
    pptr = PyBytes_AS_STRING(pobj);
} else if (pobj == Py_None) {
    pptr = (char *)NULL;
} else {
    PyErr_SetString(PyExc_OSError, "Int or String expected");
    goto err;
}

this is from
/* Python interface to getaddrinfo(host, port). */

/ARGSUSED/
static PyObject *
socket_getaddrinfo(PyObject *self, PyObject args, PyObject kwargs)
{

in the Modules/socketmodule.c file.

note the 'snprintf': the underlying api use a string (at least on Unix-like computers, I have no idea on Windows)

this is coming all from /etc/services:

x = socket.getaddrinfo('localhost','submission')
x
[(2, 1, 6, '', ('127.0.0.1', 587)), (2, 2, 17, '', ('127.0.0.1', 587))]
x2 = socket.getaddrinfo('localhost',587)
x2
[(2, 1, 6, '', ('127.0.0.1', 587)), (2, 2, 17, '', ('127.0.0.1', 587)), (2, 3, 0, '', ('127.0.0.1', 587))]
x3 = socket.getaddrinfo('localhost','587')
x3
[(2, 1, 6, '', ('127.0.0.1', 587)), (2, 2, 17, '', ('127.0.0.1', 587)), (2, 3, 0, '', ('127.0.0.1', 587))]
x4 = socket.getaddrinfo('localhost','anythingatall')
Traceback (most recent call last):
File "", line 1, in
socket.gaierror: [Errno -8] Servname not supported for ai_socktype

there is absolutely no problem IMO to changing an integer port to a string (even if it is unnecessary)

now the problem with OVH is very much specific to OVH I think.
Some testing with the control panel mail server setup seems to show that sometimes the OVH mail server does exactly what the Plone code assume to be impossible:

    # Make the timeout incredibly short. This is enough time for most mail
    # servers, wherever they may be in the world, to respond to the
    # connection request. Make sure we save the current value
    # and restore it afterward.
    timeout = socket.getdefaulttimeout()
    try:
        socket.setdefaulttimeout(3)

sometimes the OVH server replies in more that that.
Why can only be guessed at but my feeling is there a pool of available servers and the OVH dispatcher set a specific server for a client IP address and it keeps to be used for some time, and while this lasts the reply is normal (very fast). But the first time can be long (maybe it's an antispam feature)

What I am sure of is that I have seen the same code Plone with the same mail parameters fail and persist to fail, then trying again some time later succeed and keep on succeeding. When I have seen the sequence of failures I have tried with another Plone instance where I have hacked the delay to 6 seconds and this hacked instance succeeded (after a delay). Then retrying the failing Plone immediately after that succeeded quickly.
The 2 Plone instances were on the same computer, with the same public address so from the Ovh point of view it was the same client.

So a possibility is that the 'incredibly short' timeout in the CMFPlone/controlpanel/browser/mail.py file is indeed too short. At least for the strange Ovh servers.

The fact that a failing Plone instance keeps on failing may point at a problem in the code, but if true it's more subtle than a mixing of integer and string parameters. Or it may show that the Ovh dispatcher set a different server if the client IP address fails to connect - or login - (with a delay each time)