Hello, is there multicore support in Zope 4, I know it is not the right forum, but could not find any relevant information. I would appreciate any help.
you can run Zope/Plone in a Zeoserver setup, where every zeo client is running on a different process. Together with async task addons you can have several workers for heavy background tasks.
1 Like
Python does not support running on multiple cores. You still have to use the ZEO approach as described by @MrTango.
Hello.
I have not found any detailed configuration guide about zeo multithread/multiprocessing.
How to configure zeo/zope/zodb to work together, to get a loadbalanced system?
I hope you could help me with that.
Thank you.
This thread is about Zope 4.
If you use Zope 4 with ZServer, then regarding the integration with ZEO everything remains as for Zope2.
By default, Zope 4 no longer uses ZServer but a "WSGI server" component. Nevertheless, most remains as it has been with Zope2 regarding ZEO integration. You just lose the zserver_threads configuration option. Likely, there is a corresponding option for the WSGI setup. Jens has very recently improved the Zope 4 documentation regarding WSGI. Check whether you already find something there.
I use Zope 4 with "WSGI server" component. Thank you for the link to zope.readthedocs.io site, I have found it earlier. Maybe I just missed something, but haven't found information about how to configure zope4 for loadbalancing / multiprocessing.
I would like to build up an environment with zope4, which uses all the cpu cores that I gave to the Virtual Machine that zope4 is running on.
My question is that how to do it. Can you help me with that?
Are you using Plone? Have you used ZEO before (with Plone/Zope2)? With Zope 4, you do it the same way.
ZEO has nothing to do with WSGI/ZServer. Therefore, essentially nothing changes (apart from the possibility that you may have formerly used the configuration option zserver_threads which does not exist in the WSGI setup -- but this was always optional).
I forgot: a minor thing has changed: "ZEO" is no longer part of "ZODB" but now an independent package.
1 Like
No, I am new to this area.
I have found this description about multithreading: http://www.zodb.org/en/latest/articles/old-guide/transactions.html?highlight=multithread#multithreaded-zodb-programs
Does Zope 4 do multithreading by itself (as a wsgi server component) ? If not, how to install/configure it to do so?
I would like to setup a zope4 environment, which uses all the hardware resources that I give.
I mean if there is a high load of the server, zope 4 would use all the cpu cores, not just 1. Is this possible for zope 4 somehow ?
If yes, then how ? 
Thank you
A Plone process aka single Python process can only use and saturate a single CPU core due to the limitations of Python. When you have CPU cores then you usually create a ZEO cluster setup with one ZEO server and ZEO clients. This true for all Zope versions from 1999 to 2019.
I can't figure out how to build up that loadbalanced system from zeo and zope4 from MrTango's link.
There were 4 runzeo processes run on the server, but no loadbalancing at all.
Can you help with that?
I am also would like to get answer to my other question please:
Does Zope 4 do multithreading by itself (as a wsgi server component) ? If not, how to install/configure it to do so?
Thank you.
You need to learn how to configure Plone setups using zc.buildout.
https://training.plone.org/5/mastering-plone/buildout_1.html
For a ZEO setup you need to configure a ZEO server parts and 1..N ZEO clients through buildout...
For loadbalancing you usually used haproxy for proxing an incoming HTTP from haproxy to one of the configured ZEO client..yes, you need to learn about haproxy too.
I does. Nevertheless, it usually is not able to make optimal use of multiple cores (a property of Python applications). You use a "ZEO" setup to work around this: a single "ZEO" server provides access to the shared (ZODB) database for several Zope/Plone processes (called "client"s).
Plone uses zc.buildout for the configuration of the various parts of the system -- controlled by a modular configuration with the typical entry point buildout.cfg. The complete configuration consists of sections for the various parts. The configuration for the first "client" could look like
[client1]
# Use this section to install and configure a Zope
# client instance.
# For options see http://pypi.python.org/pypi/plone.recipe.zope2instance
# Additional clients are a
recipe = plone.recipe.zope2instance
zeo-client = true
zeo-address = ${zeoserver:zeo-address}
http-address = ${buildout:http-address}
...
Further clients take over its configuration and make modifications:
[client2]
# a copy of client1, except adjusted address and var location
<= client1
http-address = ${buildout:client2-address}
...
The configuration for "ZEO" could look like:
[zeoserver]
# Use this section to install and configure a Zope
# Enterprise Objects server.
# For options see http://pypi.python.org/pypi/plone.recipe.zeoserver
recipe = plone.recipe.zeoserver
zeo-address = ${buildout:zeo-address}
zeo-log = ${buildout:directory}/var/zeoserver/zeoserver.log
pid-file = ${buildout:directory}/var/zeoserver/zeoserver.pid
socket-name = ${buildout:directory}/var/zeoserver/zeo.zdsock
blob-storage = ${buildout:directory}/var/blobstorage
Install Plone and look at concrete examples.
Note that Plone 5.2 (the version of Plone using Zope 4) is "work in progress" and some things may not yet be fully supported or work.
1 Like
Dear detier and zopyx,
Thank you for answers.
I think, I don't need plone and if it is not a need, i wouldn't install it.
How can I check or measure that my installed zope does multithreading?
How can I manage how many threads zope handles?
Can I increase this number e.g.: 100? If yes, how and where, what to write in which config file?
And how to check it if zope handles 100 threads at the same time?
Looking forward to your answer.
Thomas
Zope uses by default 4 worker threads and it is sometimes advised to run 2 server threads. Running more threads usually makes no sense performance-wise. So forget the idea running Zope with 100 threads or so.
You can go to the "Zope Management Interface" (short "ZMI"). In the top bar, you see left to the "select type to add" an icon resembling a wheel. Clicking there, brings you to the "Control Panel". One of its menu entries (left part of top bar) is Databases; select this and choose main. Near the bottom of the resulting page, you see "total number of objects in each cache". Typically, each thread has its own cache; the number of caches therefore is typically the maximal number of (Zope worker) threads which have been active concurrently so far.
If you use Zope with ZServer, then there is the configuration option zserver_threads (in zope.conf) which controls this. If you use Zope with WSGI, then this is no longer controlled by Zope itself but by the web server frontend. As mentioned earlier, Jens (Vagelpohl) has recently updated the WSGI related documentation for Zope: look there, if this issue is already covered. If not, you could contact Jens or look through the sources or pay someone to do this for you.