The current asyncore based ZServer allows fun asynchronous features, which I have been relying a lot (collective.zamqp integrates Plone with RabbitMQ, collective.taskqueue runs simpler worker queue next to Plone without extra services and collective.futures frees Zope worker thread to server other requests while eg. waiting for a response for external service). Many may also still rely on Zope clockserver.
Obviously non of the above work with WSGI, which is fully synchronous in nature, so I should start planning replacements
Normal WSGI does not support anything async, so the only pure option probably is a middleware, which queues tasks for external workers after successful transaction. If worker need ZODB access, it probably needs to call Plone using restapi and its authentication.
Using some non-standard asynchronous WSGI server would allow async operations in middle-ware, but would require choosing the server and async solution and possible locking into some specific server (eg. uwsgi or unicorn).
Finally, there’s an option to implement ZServer on top of asyncio
Queues, and JWT authenticated external workers fetching their context and such via REST using the JWT itself as the instruction payload sounds like the thing which would cover the most use-cases.
This solves an another problem - having a dediated worker for known external dependencies (OAuth, backend based analytics or somesuch). This would also be a nice thing to have in the Plone toolbox.
This is probably the simplest solution, and should work with any WSGI server with a simple queuing middleware. Passing JWT for the user triggering the task should be enough for most use cases (to avoid managing special secrets for the workers), with some exception handling for those rare cases where JWT gets expired before the worker has execute the task
And this is how the current ZServer works. The benefit for having async worker in the same async loop with ZServer is that it could intercept responses and "hold" it until some async task have been completed. This is how collective.futures work (as long as there are unsolved futures, it will retry the request internally while browser remains waiting).
This would be most work for these options (aiohttp could replace medusa, thread executor could replace ZRendevouz, but a lot of work remains to adapt to Zope request, response, querystring demarshalling, etc...), but would allow very fancy stuff with aiohttp middlewares before and after ZPublisher.
Also, this allows async tasks to call Plone views, which are not visible for regular HTTP requests. I don't claim it's more secure than having HTTP calls to public views with proper token authentication, but it is a bit simpler for trusted code.
I agree, but this needs more thought. Anything requiring manually managet secrets would not work (eg. I wonder, how often have zope clockserver user credentials been changed in general...)
That was the second option in the first post, still under consideration, but as you said, it's "short of" and requires playing with gevent/greenlets. In my understanding async WSGI would require lock-in to a specific WSGI server (gunicorn, uwsgi, pulsar...). That makes it hard to see benefits over aiohttp based ZServer (besides a little less work).
Same question here.
I have a project on Plone 5.1 which uses collective.taskqueue on Zserver. It looks like I'll need to change how I'm doing async before upgrading to Plone 5.2.
And I am sorry that I cannot update that to WSGI based Plone. collective.taskqueue does have a branch that works with my ZServer branch on Python 3 and Plone 5.2.x. I have been running Python 3 ZServer with Plone 5.2.1 in production about a year now, but have not been using collective.taskqueue in those projects yet (but much more heavy Camunda + microservices + Plone REST API setup instead due to their requirements). That said, @gforcada has been interested in trying out ZServer with c.taskqueue + Redis...
c.taskqueue Python 3 branch works with my ZServer branch with or without Redis; the issue is that I have not been able to do and maintain released versions of those (not mentioning plone.recipe.zope2instance-support, which has no ZServer support on Python 3)
