Moonshot for migrating ZServer to Python 3 at PLOG 2019

Events Sprints
19

I'm leaving Sorrento today, so time for a brief sprint report.

As implied by the title, my goal was not to complete ZServer Python 3 port at PLOG, but try out if I can do that and explore the possibilities allowed by the port. For that I can say that the sprint was success and I continue to complete the Python 3 port after the sprint and expect something stable around Beethoven Sprint 2019 in June (where I will migrate collective.taskqueue to support the new ZServer). If I continue to progress without issues, I hope this to be ready for Plone 6.

So, Plone 5.2 will probably ship with only WSGI option for Python 3, configuring waitress by default. But as shown above, the waitress setup may have issues when run with just a few threads on a server with a lot of big blobs (each blob download will blog a waitress worker thread until completed). But if that becomes an issue, there are other WSGI servers to try out, of course.

At the sprint I was able to:

  • replace ZServer HTTPServer with Twisted HTTP server
  • reimplement asynchronous blob streaming with Twisted FileSender
  • move ZServer WebDAV (and WebDAV source server) on top of Twisted HTTP server
  • use the same ZPublisher publish_module for HTTP as used for WSGI
  • configure Twisted to use asyncio eventloop, especially uvloop, when available
  • try out Twisted SSL support for HTTP and HTTP/2 support (based on hyper-h2)
  • try out Twisted SSH and REPL features to "SSH into Python interpreter of live Plone instance"
  • try out Autobahn with Twisted HTTP server so that WebSocket has endpoint eg. at /ws or /Plone/@@ws
  • try out Publish-Subscribe-pattern based notifications from zope.events to browsers using WebSockets and ZMQ (on Linux, with its unix domain socket support, ZMQ can be used to broadcast events across all Plone instances on the same server)

Of course, all this, with Plone 5.2rc1 on Python 3.

My changes live at my fork of ZServer at GitHub, and I'll write a more detailed blog post when it is ready to be tried out by others (currently it is a bit of a mess after all the experiments above).

My experience of Twisted was surprisingly good during the sprint. Everything just worked, documentation was good, examples were plenty and also ecosystem support looked good (ZMQ, Redis, AMQP, etc..). Therefore I am quite confident that once I get to launch our first Python 3 site later this year, I will keep using ZServer to run Plone :slight_smile:

11 Likes