Porting Plone to Python 3

Development
1

Call me crazy but I just wrote a proposal to port Plone to Python 3.

Here are the essentials:

  • Python 2.7 will end in January 2020 and we need to support Python 3 to ensure a bright future for Plone (nicely put)
  • Zope and most of our dependencies now support Python 3
  • Plone 5.2 should support Python 2.7, Python 3 (3.5, 3.6 and 3.7)
  • Plone 5.2 should be released in early 2019

For specifics, deliverables, a timeline and problems please read the PLIP at

We are late to the Python 3 party and have to make a special effort. A combined push by the Plone community is needed if we want to pull this off. Porting Plone to Python 3 is a mayor undertaking and true to Plone's community spirit since the start of 2016 many people have worked to first move to Zope 4 and then start porting to Python 3. This needs to continue and even pick up speed.

Here is a screenshot that proves that we can do it. This is Plone 5.2 coredev built with the py3.cfg config (with has checkouts of 66 packages) running on Python 3.6. The site has no css/javascript but many of the features of Plone already work:

python3
python3.png964×3049 298 KB

Currently it can only run on on my laptop because I still have changes in 12 packages that I did not yet push because they are hacks and require some thought. I plan to push all of them in the next couple of days so you can give it a try too.

So far so good. But: The PLIP has a lot of deliverables and a tough timeline. Even overseeing this task is to much for one person. This is why I'm looking for a partner.

I'm looking for a co-sponsor of this PLIP who will team up with me to oversee, review and delegate all the tasks that need to be done. The idea is to have a hangout every two weeks to discuss the current state. Also it would be great to have some pair-programming sessions (remote) to work together on some blockers. Another important task is to keep the community and all relevant teams updated about the state of things.

It is not required that you attend any of the sprints mentioned in the PLIP. Going to other sprints and spreading the Python 3-fever there would actually be great. I plan on going to these:

Please ping me if you want to team up! :heart:

13 Likes
2

Status-Update: I've pushed all changes required for you to run Plone on Python 3 \o/

Use the py3.cfg config as described in https://github.com/plone/Products.CMFPlone/issues/2041#issuecomment-375194299

Please give it a try. Issues and notes are all collected in https://github.com/plone/Products.CMFPlone/projects/4

I've managed to get a couple of things working over the last two days (and nights):

  • adding and editing content (not files and images though)
  • workflow
  • versioning
  • search and indexing (for most cases)
  • portlets and viewlets

When you find one of the many things that are not yet working you can either choose to fix it or document it by creating a note in https://github.com/plone/Products.CMFPlone/projects/4 or ticket.

2 Likes
3

Hi @pbauer That looks great. :smile: There seems a branch missing

mr.developer: Cloned 'z3c.jbot' with git using branch 'python3' from 'https://github.com/zopefoundation/z3c.jbot.git'.
mr.developer: git cloning of 'z3c.jbot' failed.
mr.developer: b'fatal: Remote branch python3 not found in upstream origin\n'
m
1 Like
4

@tomgross thanks for testing. I fixed that.

5

Not really...the problem persists.

6

I have some PR you might be interested to here: https://github.com/zopefoundation/Products.CMFCore/pulls
I have also some other code to push for Products.CMFCore, probably tonite I will push that.
Also I have to rebase this one: https://github.com/plone/buildout.coredev/pull/448/files

7

@zopyx: I pushed the branch https://github.com/zopefoundation/z3c.jbot/tree/python3 so that issue should be gone. Which branch is missing for you? You also need to make sure you have a current coredev.

8

I followed the exact instructions in

In addition to that I see the following errors:

mr.developer: Cloned 'Products.ATContentTypes' with git using branch 'master' from 'https://github.com/plone/Products.ATContentTypes.git'.
mr.developer: git submodule init failed.

I had to update-git-submodules=False to the py3.cfg file.

9

@zopyx I did not know we use git submodules. Is it Products.ATContentTypes really where it is failing? Can you paste the full output somewhere?

10

Does this mean mr.developer does recursive clones per default?

11

Yes. See GitHub - fschulze/mr.developer: A zc.buildout extension to ease the development of large projects with lots of packages.

12

Full output:

ajung@dev ~/src/coredev_py3 (5.2) $  ./bin/buildout -c py3.cfg
mr.developer: Queued 'AccessControl' for checkout.
mr.developer: Queued 'DocumentTemplate' for checkout.
mr.developer: Queued 'Products.ATContentTypes' for checkout.
mr.developer: Queued 'Products.Archetypes' for checkout.
mr.developer: Queued 'Products.CMFCore' for checkout.
mr.developer: Queued 'Products.CMFDiffTool' for checkout.
mr.developer: Queued 'Products.CMFEditions' for checkout.
mr.developer: Queued 'Products.CMFFormController' for checkout.
mr.developer: Queued 'Products.CMFPlone' for checkout.
mr.developer: Queued 'Products.CMFQuickInstallerTool' for checkout.
mr.developer: Queued 'Products.CMFUid' for checkout.
mr.developer: Queued 'Products.DCWorkflow' for checkout.
mr.developer: Queued 'Products.DateRecurringIndex' for checkout.
mr.developer: Queued 'Products.ExternalMethod' for checkout.
mr.developer: Queued 'Products.GenericSetup' for checkout.
mr.developer: Queued 'Products.MimetypesRegistry' for checkout.
mr.developer: Queued 'Products.PDBDebugMode' for checkout.
mr.developer: Queued 'Products.PlonePAS' for checkout.
mr.developer: Queued 'Products.PluggableAuthService' for checkout.
mr.developer: Queued 'Products.PluginRegistry' for checkout.
mr.developer: Queued 'Products.PortalTransforms' for checkout.
mr.developer: Queued 'Products.ResourceRegistries' for checkout.
mr.developer: Queued 'Products.Sessions' for checkout.
mr.developer: Queued 'Products.SiteErrorLog' for checkout.
mr.developer: Queued 'Products.TemporaryFolder' for checkout.
mr.developer: Queued 'Products.ZopeVersionControl' for checkout.
mr.developer: Queued 'Products.statusmessages' for checkout.
mr.developer: Queued 'Zope' for checkout.
mr.developer: Queued 'archetypes.multilingual' for checkout.
mr.developer: Queued 'archetypes.referencebrowserwidget' for checkout.
mr.developer: Queued 'collective.monkeypatcher' for checkout.
mr.developer: Queued 'five.customerize' for checkout.
mr.developer: Queued 'five.intid' for checkout.
mr.developer: Queued 'five.pt' for checkout.
mr.developer: Queued 'mockup' for checkout.
mr.developer: Queued 'plone.api' for checkout.
mr.developer: Queued 'plone.app.caching' for checkout.
mr.developer: Queued 'plone.app.content' for checkout.
mr.developer: Queued 'plone.app.contentlisting' for checkout.
mr.developer: Queued 'plone.app.contentmenu' for checkout.
mr.developer: Queued 'plone.app.contenttypes' for checkout.
mr.developer: Queued 'plone.app.dexterity' for checkout.
mr.developer: Queued 'plone.app.discussion' for checkout.
mr.developer: Queued 'plone.app.event' for checkout.
mr.developer: Queued 'plone.app.folder' for checkout.
mr.developer: Queued 'plone.app.iterate' for checkout.
mr.developer: Queued 'plone.app.layout' for checkout.
mr.developer: Queued 'plone.app.linkintegrity' for checkout.
mr.developer: Queued 'plone.app.locales' for checkout.
mr.developer: Queued 'plone.app.multilingual' for checkout.
mr.developer: Queued 'plone.app.portlets' for checkout.
mr.developer: Queued 'plone.app.registry' for checkout.
mr.developer: Queued 'plone.app.relationfield' for checkout.
mr.developer: Queued 'plone.app.testing' for checkout.
mr.developer: Queued 'plone.app.textfield' for checkout.
mr.developer: Queued 'plone.app.theming' for checkout.
mr.developer: Queued 'plone.app.upgrade' for checkout.
mr.developer: Queued 'plone.app.users' for checkout.
mr.developer: Queued 'plone.app.uuid' for checkout.
mr.developer: Queued 'plone.app.versioningbehavior' for checkout.
mr.developer: Queued 'plone.app.viewletmanager' for checkout.
mr.developer: Queued 'plone.app.widgets' for checkout.
mr.developer: Queued 'plone.app.workflow' for checkout.
mr.developer: Queued 'plone.app.z3cform' for checkout.
mr.developer: Queued 'plone.autoform' for checkout.
mr.developer: Queued 'plone.dexterity' for checkout.
mr.developer: Queued 'plone.folder' for checkout.
mr.developer: Queued 'plone.formwidget.namedfile' for checkout.
mr.developer: Queued 'plone.i18n' for checkout.
mr.developer: Queued 'plone.locking' for checkout.
mr.developer: Queued 'plone.namedfile' for checkout.
mr.developer: Queued 'plone.outputfilters' for checkout.
mr.developer: Queued 'plone.protect' for checkout.
mr.developer: Queued 'plone.recipe.zope2instance' for checkout.
mr.developer: Queued 'plone.releaser' for checkout.
mr.developer: Queued 'plone.reload' for checkout.
mr.developer: Queued 'plone.resource' for checkout.
mr.developer: Queued 'plone.rfc822' for checkout.
mr.developer: Queued 'plone.scale' for checkout.
mr.developer: Queued 'plone.schemaeditor' for checkout.
mr.developer: Queued 'plone.session' for checkout.
mr.developer: Queued 'plone.stringinterp' for checkout.
mr.developer: Queued 'plone.subrequest' for checkout.
mr.developer: Queued 'plone.testing' for checkout.
mr.developer: Queued 'plone.theme' for checkout.
mr.developer: Queued 'plone.themepreview' for checkout.
mr.developer: Queued 'plone.transformchain' for checkout.
mr.developer: Queued 'plone.z3cform' for checkout.
mr.developer: Queued 'z3c.autoinclude' for checkout.
mr.developer: Queued 'z3c.caching' for checkout.
mr.developer: Queued 'z3c.jbot' for checkout.
mr.developer: Queued 'z3c.objpath' for checkout.
mr.developer: Queued 'z3c.relationfield' for checkout.
mr.developer: Queued 'zc.relation' for checkout.
mr.developer: Skipped checkout of existing package 'Products.CMFCore'.
mr.developer: Skipped checkout of existing package 'Products.ATContentTypes'.
mr.developer: Skipped checkout of existing package 'AccessControl'.
mr.developer: Skipped checkout of existing package 'DocumentTemplate'.
mr.developer: Skipped checkout of existing package 'Products.Archetypes'.
mr.developer: Skipped checkout of existing package 'Products.CMFDiffTool'.
mr.developer: Skipped checkout of existing package 'Products.CMFEditions'.
mr.developer: Skipped checkout of existing package 'Products.CMFPlone'.
mr.developer: Skipped checkout of existing package 'Products.CMFFormController'.
mr.developer: Skipped checkout of existing package 'Products.DCWorkflow'.
mr.developer: Skipped checkout of existing package 'Products.DateRecurringIndex'.
mr.developer: Skipped checkout of existing package 'Products.CMFUid'.
mr.developer: Skipped checkout of existing package 'Products.CMFQuickInstallerTool'.
mr.developer: Skipped checkout of existing package 'Products.ExternalMethod'.
mr.developer: Skipped checkout of existing package 'Products.GenericSetup'.
mr.developer: Skipped checkout of existing package 'Products.PDBDebugMode'.
mr.developer: Skipped checkout of existing package 'Products.MimetypesRegistry'.
mr.developer: Skipped checkout of existing package 'Products.PluggableAuthService'.
mr.developer: Skipped checkout of existing package 'Products.PlonePAS'.
mr.developer: Skipped checkout of existing package 'Products.PortalTransforms'.
mr.developer: Skipped checkout of existing package 'Products.ResourceRegistries'.
mr.developer: Skipped checkout of existing package 'Products.PluginRegistry'.
mr.developer: Skipped checkout of existing package 'Products.Sessions'.
mr.developer: Skipped checkout of existing package 'Products.SiteErrorLog'.
mr.developer: Skipped checkout of existing package 'Products.TemporaryFolder'.
mr.developer: Skipped checkout of existing package 'Products.ZopeVersionControl'.
mr.developer: Skipped checkout of existing package 'Zope'.
mr.developer: Skipped checkout of existing package 'Products.statusmessages'.
mr.developer: Skipped checkout of existing package 'five.customerize'.
mr.developer: Skipped checkout of existing package 'archetypes.multilingual'.
mr.developer: Skipped checkout of existing package 'archetypes.referencebrowserwidget'.
mr.developer: Skipped checkout of existing package 'collective.monkeypatcher'.
mr.developer: Skipped checkout of existing package 'five.pt'.
mr.developer: Skipped checkout of existing package 'five.intid'.
mr.developer: Skipped checkout of existing package 'mockup'.
mr.developer: Skipped checkout of existing package 'plone.api'.
mr.developer: Skipped checkout of existing package 'plone.app.content'.
mr.developer: Skipped checkout of existing package 'plone.app.contentlisting'.
mr.developer: Skipped checkout of existing package 'plone.app.caching'.
mr.developer: Skipped checkout of existing package 'plone.app.contentmenu'.
mr.developer: Skipped checkout of existing package 'plone.app.dexterity'.
mr.developer: Skipped checkout of existing package 'plone.app.folder'.
mr.developer: Skipped checkout of existing package 'plone.app.contenttypes'.
mr.developer: Skipped checkout of existing package 'plone.app.event'.
mr.developer: Skipped checkout of existing package 'plone.app.discussion'.
mr.developer: Skipped checkout of existing package 'plone.app.iterate'.
mr.developer: Skipped checkout of existing package 'plone.app.multilingual'.
mr.developer: Skipped checkout of existing package 'plone.app.portlets'.
mr.developer: Skipped checkout of existing package 'plone.app.linkintegrity'.
mr.developer: Skipped checkout of existing package 'plone.app.registry'.
mr.developer: Skipped checkout of existing package 'plone.app.testing'.
mr.developer: Skipped checkout of existing package 'plone.app.relationfield'.
mr.developer: Cloned 'plone.app.layout' with git using branch 'python3' from 'https://github.com/plone/plone.app.layout.git'.
mr.developer: git cloning of 'plone.app.layout' failed.
mr.developer: b'fatal: Remote branch python3 not found in upstream origin\n'
mr.developer: Cloned 'plone.app.textfield' with git using branch 'python3' from 'https://github.com/plone/plone.app.textfield.git'.
mr.developer: git submodule init failed.
mr.developer:
mr.developer: Cloned 'plone.app.upgrade' with git using branch 'master' from 'https://github.com/plone/plone.app.upgrade.git'.
mr.developer: git submodule init failed.
mr.developer:
mr.developer: Cloned 'plone.app.theming' with git using branch 'master' from 'https://github.com/plone/plone.app.theming.git'.
mr.developer: git submodule init failed.
mr.developer:
mr.developer: Cloned 'plone.app.locales' with git using branch 'master' from 'https://github.com/collective/plone.app.locales.git'.
mr.developer: git submodule init failed.
mr.developer:
mr.developer: There have been errors, see messages above.
```
13

plone.app.layout was the issue. Fixed.
@jensens merged that branch but I had more changes locally anyway so there is a new python3 branch now.

14

See https://github.com/plone/buildout.coredev/pull/450

15

How does it end up down that branch in the underlying git code? That's not the default and also does not exist since the beginning.

The docs or the source code of mr.developer do not refer to recursive as far as I can see. What am I missing?

16

I'm one step further. Buildout runs OK but on startup it gives me the following:

Traceback (most recent call last):
  File "bin/wsgi.py", line 241, in <module>
    wsgiapp = make_wsgi_app({}, '/home/vagrant/coredev_py3/parts/wsgi.py/etc/zope.conf')
  File "/home/vagrant/coredev_py3/src/Zope/src/Zope2/Startup/run.py", line 67, in make_wsgi_app
    starter.prepare()
  File "/home/vagrant/coredev_py3/src/Zope/src/Zope2/Startup/starter.py", line 38, in prepare
    self.startZope()
  File "/home/vagrant/coredev_py3/src/Zope/src/Zope2/Startup/starter.py", line 105, in startZope
    Zope2.startup_wsgi()
  File "/home/vagrant/coredev_py3/src/Zope/src/Zope2/__init__.py", line 49, in startup_wsgi
    _startup()
  File "/home/vagrant/coredev_py3/src/Zope/src/Zope2/App/startup.py", line 86, in startup
    OFS.Application.import_products()
  File "/home/vagrant/coredev_py3/src/Zope/src/OFS/Application.py", line 379, in import_products
    import_product(product_dir, product_name)
  File "/home/vagrant/coredev_py3/src/Zope/src/OFS/Application.py", line 389, in import_product
    global_dict, global_dict, ('__doc__', ))
  File "/home/vagrant/coredev_py3/src/Products.PlonePAS/src/Products/PlonePAS/__init__.py", line 15, in <module>
    from Products.PlonePAS.tools.groupdata import GroupDataTool
  File "/home/vagrant/coredev_py3/src/Products.PlonePAS/src/Products/PlonePAS/tools/groupdata.py", line 104, in <module>
    class GroupData(SimpleItem):
  File "/home/vagrant/coredev_py3/src/Products.PlonePAS/src/Products/PlonePAS/tools/groupdata.py", line 517, in GroupData
    canAddToGroup = MemberData.canAddToGroup.__func__
AttributeError: 'function' object has no attribute '__func__'

A similar issue is reported here: https://github.com/django-fluent/django-fluent-contents/issues/43

17

@tomgross fixed. I obviously forgot to rebase that branch. The issue is dealt with in the now updated python3-branch of PlonePAS.

18

I tried again with a fresh buildout and fixed some more issues (I had forgotten to add one checkout, two python3 branches and three python 3 branches were merged yesterday). Surry for the mess but things are moving fast.

Now it works for me except for the issue with No module named 'zc.buildout' during startup. The workaround is describes in https://github.com/plone/Products.CMFPlone/projects/4#card-8602748

19

Also: I'll be online in irc during most of the day if you have further issues or questions.

20

I can confirm Plone 5.2 dev starts up with Python 3.6 on my fedora box. I guess I'm still among the first ten seeing this. Thanks @pbauer for making this happen :smile:
I did two minor things differnetly according to the provided documentation:

  1. Buildout: I suspect the buildout issue on startup is caused by https://github.com/buildout/buildout/issues/410 (and/or https://github.com/buildout/buildout/issues/434). The workaround provided in #410 to use the bootstrap script (https://bootstrap.pypa.io/bootstrap-buildout.py) works for me. Using the bootstrap script might be not recomended but it works and has the advantage that it is automatable. I created an issue https://github.com/plone/Products.CMFPlone/issues/2374 to track this.
  2. Startup command: I use bin\wsgi.py etc/waitress.ini for startup.
1 Like