i'd like to use zope's icp-server for health checks on haproxy (to save ram compared to additional zserver threads only accessed by haproxy)
seems that icp-server does not work as expected (see below). would like to start discussion or get some pointers how others to health checks on zope instances in their setups
zserver-treads = 1
(to save RAM, second thread could load objects in cache (zodb-cache-size))
server instance1 127.0.0.1:8080 check 5s fastinter 2s downinter 2s fall 1 rise 2 maxconn 1
if you've got a long running request (longer then checkintervall + downinter * fall) on the zope innstance that occupies the only available thread, haproxy marks it as down
instead of setting
zserver-threads = 2 (so the second thread can handle the health checks) i thought using
zope's icp server would be an elegant solution. (as i can be sure not to load a lot of objects into the cache so the second thread uses RAM that is better spent elsewhere)
plone.recipe.zope2instance allows to activate it via
icp-address = 8090
configure haproxy to use the icp port
server instance1 127.0.0.1:8080 check port 8090 5s fastinter 2s downinter 2s fall 1 rise 2 maxconn 1
however, the zope instance is never recognized as up.
reason: zserver is not listening on the icp port:
on startup zope reports the server as running
2017-11-28T15:27:15 INFO ZServer HTTP server started at Tue Nov 28 15:27:15 2017 Hostname: 0.0.0.0 Port: 8080 ------ 2017-11-28T15:27:15 INFO ZServer ICP server started Address: 0.0.0.0 Port: 8090
but telnet can't connect
$ telnet 127.0.0.1 8080 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. $ telnet 127.0.0.1 8090 Trying 127.0.0.1... telnet: Unable to connect to remote host: Connection refused
if others can confirm this problem i'll write a ticket for zope2
icp doesn't work - what else to do?
i guess as long as i only check http://127.0.0.1:8080 with haproxy (zope root)
the number of objects loaded into the cache will be quite low (compared to check for a path within the plone site)
on the other hand a check on / does not really mean the plone site can be served by this zope server already.
therefore one would need to check /Plone/@@some_view
(option httpchk /Plone/@@some_view)
what do you use for health checks on your setups?
icp server (how/why does it work for you?)
zope with zserver-threads = haproxy.maxonn + 1
(/Plone, or root / for the check?)
or custom view that does not - or does - involve portal_catalog?
custom scripts outside of zope - as https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#port suggests:
Using the "port" parameter, it becomes possible to use a different port to send health-checks. On some servers, it may be desirable to dedicate a port to a specific component able to perform complex tests which are more suitable to health-checks than the application. It is common to run a simple script in inetd for instance. This parameter is ignored if the "check" parameter is not set. See also the "addr" parameter.