Flake8 version pin hell

I'm stumped on how to get past this flake8 version conflict in the https://github.com/collective/plonetheme.future_imperfect repo

https://travis-ci.com/github/collective/plonetheme.future_imperfect/builds/98073116#L752

specifically:

Got plone.recipe.codeanalysis 3.0.1.

Getting distribution for 'flake8-todo'.

Got flake8-todo 0.7.

Getting distribution for 'flake8-string-format'.

Got flake8-string-format 0.2.3.

Getting distribution for 'flake8-quotes'.

Got flake8-quotes 1.0.0.

Getting distribution for 'flake8-print'.

Got flake8-print 3.1.0.

Getting distribution for 'flake8-plone-hasattr'.

Got flake8-plone-hasattr 0.2.post0.

Getting distribution for 'flake8-plone-api'.

Got flake8-plone-api 1.4.

Getting distribution for 'flake8-pep3101'.

Got flake8-pep3101 1.2.1.

Getting distribution for 'flake8-isort'.

Got flake8-isort 2.6.0.

Getting distribution for 'flake8-deprecated'.

Got flake8-deprecated 1.3.

Getting distribution for 'flake8-debugger'.

Got flake8-debugger 3.1.0.

Getting distribution for 'flake8-commas'.

Got flake8-commas 2.0.0.

Getting distribution for 'flake8-coding'.

Got flake8-coding 1.3.1.

Getting distribution for 'flake8-blind-except'.

Got flake8-blind-except 0.1.1.

Getting distribution for 'lxml==3.5.0'.

Building lxml version 3.5.0.

Building without Cython.

Using build configuration of libxslt 1.1.28

Got lxml 3.5.0.

Getting distribution for 'flake8==2.5.5'.

Got flake8 2.5.5.

Getting distribution for 'check-manifest'.

Got check-manifest 0.37.

Getting distribution for 'pycodestyle<3.0.0,>=2.0.0'.

Got pycodestyle 2.4.0.

Getting distribution for 'six==1.10.0'.

Got six 1.10.0.

Version and requirements information containing flake8:

  [versions] constraint on flake8: 2.5.5

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-todo

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-string-format

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-quotes

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-print

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-plone-hasattr

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-plone-api

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-pep3101

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-isort

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-deprecated

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-debugger

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-commas

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-coding

  Requirement of plone.recipe.codeanalysis[recommended]: flake8-blind-except

  Requirement of plone.recipe.codeanalysis[recommended]: flake8>=2.4.0

  Requirement of flake8-todo: pycodestyle<3.0.0,>=2.0.0

  Requirement of flake8-string-format: flake8

  Requirement of flake8-quotes: flake8

  Requirement of flake8-print: pycodestyle

  Requirement of flake8-print: six

  Requirement of flake8-print: flake8>=1.5

  Requirement of flake8-plone-hasattr: flake8

  Requirement of flake8-plone-api: flake8

  Requirement of flake8-pep3101: flake8>=3.0

  Requirement of flake8-isort: testfixtures

  Requirement of flake8-isort: isort>=4.3.0

  Requirement of flake8-isort: flake8>=3.2.1

  Requirement of flake8-deprecated: flake8>=3.0.0

  Requirement of flake8-debugger: pycodestyle

  Requirement of flake8-debugger: flake8>=1.5

  Requirement of flake8-commas: flake8<4.0.0,>=2

  Requirement of flake8-coding: flake8

  Requirement of flake8-blind-except: setuptools

  Requirement of flake8>=2.4.0: pyflakes<1.1,>=0.8.1

  Requirement of flake8>=2.4.0: pep8!=1.6.0,!=1.6.1,!=1.6.2,>=1.5.7

  Requirement of flake8>=2.4.0: mccabe<0.5,>=0.2.1

While:

  Installing.

  Getting section code-analysis.

  Initializing section code-analysis.

  Installing recipe plone.recipe.codeanalysis[recommended].

Error: The requirement ('flake8>=3.0') is not allowed by your [versions] constraint (2.5.5)

The command "bin/buildout -Nc travis.cfg" failed and exited with 1 during .

I've even tried removing the code-analysis part, but somehow plone.recipe.codeanalysis[recommended] keeps getting pulled in, and when I remove that block out of the buildout.cfg https://github.com/collective/plonetheme.future_imperfect/blob/master/buildout.cfg#L26 something else keeps pulling in flake8.

I'm sure this is more complicated by trying to test using Plone 5.0.5, dating back to 2016, but I thought I'd get this working before updating to more recent Plone versions. This flake8 problem occurs just running a regular buildout locally.

Removing https://github.com/collective/plonetheme.future_imperfect/blob/d2f719b2819e0fa30bf3fa6c324cd575b3842dd8/buildout.cfg#L68 would probably fix it for this specific repo.

Or you can pin flake8-pep3101 < 1 to not get a version that requires flake8>=3.0.

1 Like

This is a constant source of pain in our projects as well tbh. You can find the version pins we use for all our open source projects (Plone 4.x - 5.x) here:

1 Like

This may be during the uninstall phase of buildout: buildout sees that you have previously had a codeanalysis part, but not anymore. So it tries to uninstall whatever was installed by this part. To uninstall this properly, it first tries to install the recipe again because it may have custom uninstall code...
Work around this, by removing the hidden .installed.cfg file and running bin/buildout again.

3 Likes

ah, I had not known why it was doing this – thanks!

I am trying this approach. I added the following pins (and kept flake8 at 2.5.5):

flake8-pep3101 = 0.6
flake8-commas = 0.1.6
flake8-isort = 1.3
flake8-deprecated = 1.0

but am now getting this weird pep517 / zipp / UNKNOWN thing:

Develop: '/Users/kimadmin/src/plonetheme.future_imperfect/.'
warning: no files found matching '*.html'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '.jshintignore' found anywhere in distribution
warning: no previously-included files matching '.DS_Store' found anywhere in distribution
Getting distribution for 'flake8-deprecated==1.0'.
warning: no previously-included files found matching '.installed.cfg'
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '*.pyc'
Got flake8-deprecated 1.0.
Getting distribution for 'zipp'.
warning: install_lib: 'build/lib' does not exist -- no Python modules to install

zip_safe flag not set; analyzing archive contents...
Installing zipp 3.1.0
Caused installation of a distribution:
UNKNOWN 0.0.0
with a different project name.
Installing zipp 3.1.0
Caused installation of a distribution:
UNKNOWN 0.0.0
with a different version.
Got None.
Version and requirements information containing zipp:
  Requirement of pep517: zipp
While:
  Installing.
  Getting section code-analysis.
  Initializing section code-analysis.
  Installing recipe plone.recipe.codeanalysis[recommended].
Error: There is a version conflict.
We already have: UNKNOWN 0.0.0
but pep517 0.8.2 requires 'zipp'.

running buildout -vvv gives me this (hopefully relevant) bit:

Getting required 'testfixtures'
  required by flake8-isort 1.3.
We have the best distribution that satisfies 'testfixtures'.
Picked: testfixtures = 6.14.1
Getting required 'isort'
  required by flake8-isort 1.3.
We have the best distribution that satisfies 'isort'.
Picked: isort = 5.2.0
Getting required 'pep8'
  required by flake8-commas 0.1.6.
We have the best distribution that satisfies 'pep8'.
Picked: pep8 = 1.7.1
Getting required 'mccabe<0.5,>=0.2.1'
  required by flake8 2.5.5.
We have the best distribution that satisfies 'mccabe<0.5,>=0.2.1'.
Picked: mccabe = 0.4.0
Getting required 'pyflakes<1.1,>=0.8.1'
  required by flake8 2.5.5.
We have the best distribution that satisfies 'pyflakes<1.1,>=0.8.1'.
Picked: pyflakes = 1.0.0
Getting required 'toml'
  required by check-manifest 0.42.
We have the best distribution that satisfies 'toml'.
Picked: toml = 0.10.1
Getting required 'pep517'
  required by check-manifest 0.42.
We have the best distribution that satisfies 'pep517'.
Picked: pep517 = 0.8.2
Getting required 'zipp'
  required by pep517 0.8.2.
We have no distributions for zipp that satisfies 'zipp'.
Getting distribution for 'zipp'.
Running easy_install:
"/Users/kimadmin/src/plonetheme.future_imperfect/bin/python" "-c" "import sys; sys.path[0:0] = ['/Users/kimadmin/src/plonetheme.future_imperfect/eggs/setuptools-24.3.0-py2.7.egg']; from setuptools.command.easy_install import main; main()" "-mZUNxd" "/Users/kimadmin/src/plonetheme.future_imperfect/eggs/tmpzcCD4d" "-v" "/var/folders/lx/6chcd57j7w16ycjwnn_x9xg00000gx/T/tmpXps9tEget_dist/zipp-3.1.0.tar.gz"
path=/Users/kimadmin/src/plonetheme.future_imperfect/eggs/setuptools-24.3.0-py2.7.egg

Processing zipp-3.1.0.tar.gz

Now I'm looking for why pep517 is being pulled in... it didn't exist in 2016. https://pypi.org/project/pep517/#history

How come having everything pinned (which I think was the case when this repo was put together, similarly so for Plone 5.0.5) doesn't actually let me easily rerun buildout without having all this scary **** happen?

@tkimnguyen zipp >= 2.0.0 is not compatible with Python 2. See:

@tkimnguyen I recommend you simply remove this amount of pinns. It worked for me. See:

@mauritsvanrees by the way. You can remove the pycodestyle pinn in Plone 5.0 coredev?

This package always conflicts with the pinned version on flake8:

I believe this pinout isn't necessary in coredev. In Plone 5.2 this package isn't pinned.

I think not everything was pinned :wink:
I used allow-picked-versions = false to make sure I have everything pinned.

When creating the initial versions list I set show-picked-versions = true and set allow-picked-versions = true once, and add that to a local dumpversions.cfg with a [versions] local - your project's KGS. Add that to git and make your buildout.cfg extend it.

Now, your dependencies can still be changed on PyPI, so new unexpected sub-dependencies are possible. I recommend building your project, from scratsch, every day using something like Jenkins to monitor that.

I also used devpi to have a company wide cache of upstream packages based on an allowlist, so I never really run into this kind of issues.

Technically I can. But there won't ever be a new Plone 5.0 release, so it is no use.
See https://github.com/plone/buildout.coredev/blob/5.0/DO-NOT-USE-THIS-BRANCH.txt

OK. Thanks!

Thanks, but it's worse than that: zipp was not released until 2018, and I was looking for pins from late 2016.

I'm not sure I understand how this would help. I have these:

setuptools = 24.3.0
zc.buildout = 2.5.2
plone.testing = 5.0.0

but I do not have pins for configparser, docker, elasticsearch, nor pycodestyle.

This is what I decided to try next. I had to help buildout by pip install Pillow==3.2.0 and it completed!

The pins I had to add:

zest.releaser = 6.8
twine = 1.8.1

And the output was:

Versions had to be automatically picked.
The following part definition lists the versions picked:
[versions]
pathtools = 0.1.2
pkginfo = 1.5.0.1
requests-toolbelt = 0.9.1
watchdog = 0.10.3

# Required by:
# clint==0.5.1
args = 0.1.0

# Required by:
# twine==1.8.1
clint = 0.5.1

# Required by:
# zest.releaser==6.8
colorama = 0.4.3

# Required by:
# plonetheme.future-imperfect==1.0.2.dev0
plone.app.themingplugins = 1.1

# Required by:
# plone.app.themingplugins==1.1
# plonetheme.future-imperfect==1.0.2.dev0
z3c.jbot = 0.8

Thank you all! I will push these changes to the repo :slight_smile:

Is there a specific reason why you want to use the 2016 versions? Can't you just use more current versions?

Plone Foundation Code of Conduct