Usage of Six when porting Plone to Python 3

I have seen some pull requests introducing conditional code for Python 2 and 3 compatibility. Can we please instead of doing this for every module use the six module https://pythonhosted.org/six/, since this is a Zope? dependency already. I don't see any shortcomings there, or did I miss something?

  • It is supported
  • It is easier to understand
  • It is easier to clean up, if we go for Python3 only
  • It is included already
3 Likes

Yes please, we have enough wheels on our own to carry around, let's avoid at least one! :smile:

1 Like

All right, let's do that. We will remove the _compat modules from the packages where we already started (see https://docs.google.com/document/d/1rYN817-9SjAQulpes36k1veTt4pABZRbl0T0BbuZrHA/edit) and use six instead.

nice! is there a tool to help us identify which imports we have to fix?

I would like to start doing this on our own add-ons.

UPDATE

I found this: http://python3porting.com/differences.html

This is also nice http://python-future.org/compatible_idioms.html

Yet, it’s good that there is now decision to prefer six.

At the conf we decide do use sixer to do this job:

how does sixer works? it's automatic or it's more like a linter? can we add it to run in CI somehow?

sixer works only under Python 3 so it's not an option for me.

LOL!
This is a topic about porting Plone to Python 3.

P.s.: I installed sixer with pipsi https://pypi.python.org/pypi/pipsi, it is a nice tool to master.

I know :slight_smile: but I want to check my code under Python 2.7 and get recommendations on what needs to be migrated; as stated in GitHub, we need more like a linter because this is going to be really hard.

I was checking Pylint again but I remembered why I just stopped trying to use it years ago: it's way too complex for beginners.

anyway, does anybody has a good .pylintrc file to begin with for Zope/Plone projects?

or, does anybody has tried to run sixer on CI? with Python 3, of course :wink:

1 Like

mmm, seems I found something usable at the end :slight_smile:

$ pylint --py3k --disable=no-absolute-import src/collective/lazysizes/
No config file found, using default configuration
************* Module collective.lazysizes.vocabularies
W: 13,22: Calling a dict.iter*() method (dict-iter-method)

-----------------------------------
Your code has been rated at 9.98/10

$ echo $?
4
$ pylint --py3k --disable=no-absolute-import src/collective/fingerpointing/
No config file found, using default configuration

------------------------------------
Your code has been rated at 10.00/10

$ echo $?
0

I'm disabling no-absolute-import because we already check for that with plone.recipe.codeanalysis, and is generating a lot of noise.

now I'm going to add this to all our packages.

Just make sure the site-packages contains the eggs for visibility into imports and such.

I usually have something along the lines of

[buildout]
extends =
    development.cfg

parts +=
    isort
    pylama

[isort]
recipe=zc.recipe.egg:scripts
eggs =
    isort

[pylama]
recipe=zc.recipe.egg:scripts
eggs =
    ${instance:eggs}
    ${test:eggs}
    pylama
    pylama-pylint
    setuptools

in my development buildouts.

thanks, I had noticed that. in fact the issue is a little bit different, if I run Pylint without the --disable=no-absolute-import I get a lot of useless warnings like these:

$ pylint --py3k src/collective/fingerpointing/
No config file found, using default configuration
************* Module collective.fingerpointing
W:  2, 0: import missing `from __future__ import absolute_import` (no-absolute-import)
...
************* Module collective.fingerpointing.upgrades.v3
W:  2, 0: import missing `from __future__ import absolute_import` (no-absolute-import)
W:  3, 0: import missing `from __future__ import absolute_import` (no-absolute-import)

-------------------------------------------------------------------
Your code has been rated at 7.79/10 (previous run: 10.00/10, -2.21)

Making sure this propagates if the use of pylint is to spread within the Plone circles. I've seen people not use pylint because it is beyond verbose upon this particular failure and that is a shame.

I recommend wrapping the config and running of pylint with pylama (or pyflakes) and starting to build a setup.cfg. Both wrappers support rather nice exclusion, inclusion etc. schemes for the configuration. Here is one of mine for loose reference and inspiration:

[pylama]
linters = pycodestyle,pydocstyle,pyflakes,pylint

ignore = C0102,C0103,C0111,C0302,C0411,D100,D102,D103,D104,D105,D200,D203,D205,D212,D213,D400,D401,D404,E121,E122,E123,E125,E126,E127,E128,E301,E0611,E1002,E1103,E1101,F0401,I0011,R0201,R0801,R0901,R0902,R0903,R0904,R0913,R0921,R0922,W0141,W0142,W0232,W0613

[pylama:pycodestyle]
max_line_length = 125

[pylama:pylint]
max_line_length = 125

[pylama:__init__.py]
linters = pycodestyle,pydocstyle,pylint

[pylama:*/bindings/*]
skip = *

[pylama:*/monkey/*]
skip = *

[pylama:*/skins/*]
skip = *

[pylama:*/tests/*]
linters = pycodestyle,pyflakes,pylint

[pylama:bootstrap.py]
skip = *

[pylama:pyxbgen.py]
skip = *

[isort]
force_alphabetical_sort_within_sections = True
force_single_line = True
line_length = 125
lines_after_imports = 2
lines_between_types = 0
no_sections = True

And you can run just one (or any subset) of the linters at a time with a --linters flag and that still respects the skip declarations and linter definitions from the config.

Since we got into this already: Have you had any fun with BBB or interface inheritance yet with pylint? It does not deal super well with old-style classes or the Zope "new style" classes (mostly stuff leaning on ExtensionClass). Sharing of all anecdotes / tips / tricks is welcome.

1 Like

well, I already implemented this in a couple of repos:

the results are mixed: locally collective.lazysizes returned a warning that should make the build fail (see above), but that's not happening and I don't know why:

https://travis-ci.org/collective/collective.lazysizes/jobs/302678106#L1698

I'll try later with more complex add-ons like collective.cover.

is working as expected in collective.cover and the build is failing:

https://travis-ci.org/collective/collective.cover/jobs/302700329#L2826

so, maybe this is a problem with dependency versions.

now we need to compile a list of messages and possible fixes for every issue.

is there already a branch for Plone 6 on the documentation?

in fact, that compilation was already made by someone else on the post mentioned by @datakurre above :slight_smile: