Porting Plone to Python 3

Development
61

I invest a full working day for porting my own publishing technology "Produce & Publish" to Plone 5.2 and Python 3. This included three Plone-based add-ons and three Python-only modules. After eight hours of code juggling I was able to successfully test all Plone add-ons.

Standard problems:

  • implements -> @implementer
  • conversions using unicode() or type checks against unicode -> six.text_type
  • cStringIO -> io module
  • various import fixes
  • getting rid of some Python 3 incompatible modules like BeautifulSoup 3 (replaced with lxml)
  • very few issues with reading/writing files (text vs. bytes)
  • fixed a few subtle programming patterns specific to my own coding style

I did not use any Python 2 to 3 conversion tools and I did not use any unittests (we have some real world examples that I used to test directly with production data in order to check the most important and most complicated code paths).

I did not encounter any serious backend related issues but again a bunch of UI issues (possibly also existent in Plone 5.1 and unrelated to the Python version).

6 Likes
62

Sorry for bringing up this thread. But the topic is exactly what I need. My question:
if i started the porting process for my addons, what is the standard way for dealing with 'bootstrap-buildout.py' in an Add-On, . Should i use the way with

  1. bootstrap.sh and requirements.txt

or

  1. bootstrap-buildout.py

It probably does not matter, but does anyone have a hint for me?

63

I suggest to only use requirements.txt.

64

I like the approach that @MrTango uses in PloneCLI
This is roughly based on how PloneCLI runs a build:

virtualenv --clear .
./bin/pip install -r requirements.txt
./bin/buildout

You might also want to consider specifying your version of Python in the virtualenv call.
virtualenv -p python3.7 --clear .

Essentially, I'm in agreement with what @pbauer said above.

65

The recommended setup is using requirements.txt. You can check some discussion about this subject in https://github.com/plone/Products.CMFPlone/issues/2001.

I suggest using plonecli to manage your packages and installations.

66

I like to use pyenv like so:

One time (after installing pyenv with pyenv-installer):

pyenv install 3.7.4
pyenv virtualenv 3.7.4 plone-5.2.x

Then in each buildout folder I put a requirements.txt containing (URL has to match Plone release):

-r https://dist.plone.org/release/5.2-pending/requirements.txt

Now using this:

pyenv local plone-5.2.x
pip install -r requirements.txt
buildout
67

I personally don't like repeated information such has having the setuptools and buildout versions repeated in requirements.txt and buildout.cfg.

It's a bit of a hack but this way avoids that.

virtualenv .
bin/pip install --upgrade pip zc.buildout
bin/buildout -c travis.cfg annotate | tee annotate.txt | grep -E 'setuptools= |zc.buildout= ' | sed 's/= /==/' > requirements.txt
cat annotate.txt
cat requirements.txt
bin/pip install --upgrade -r requirements.txt

I really should add a feature to buildout to get out version requirements e.g.

bin/pip install zc.buildout
bin/pip install --upgrade `bin/buildout requirements zc.buildout setuptools`
68

I guess an alternative is have buildout support the following

[buildout]
versions-file=requirements.txt
69

I wouldn't recommend to use bootstrap-buildout.py because everybody moved to use virtualenv and requirements.txt.

I moved to use Makefiles for all my projects (Plone and others). It avoids repetitive typing (like when you create your virtualenv manually) and it has everything in one file (in comparison to multiple bootstrap.sh files).

I created a self-updating Makefile that I use for all our internal projects, for Plone coredev and for all my Open Source projects:

If you have that Makefile, you just need to run "make". We use a different Makefile for JS/React projects but whatever you do, you checkout the repo and run "make". This drastically reduces the mental overhead for our devs. Feel free to use it!

5 Likes
70

+1 for that

2 Likes
71

I'm sold on things that reduce mental overhead :slight_smile: