Buildout not working / Couldn't find index page for / Couldn't find a distribution / Disabling non HTTPS access to APIs on PyPI

hvelarde · October 27, 2017, 1:03pm

no, pypi.python.org is behind Fastly CDN:

$ host pypi.python.org
pypi.python.org is an alias for prod.python.map.fastly.net.
prod.python.map.fastly.net has address 151.101.92.223

pigeonflight · October 27, 2017, 1:04pm

I had a similar issue this morning:

attempted to run

docker run -p 8080:8080 -e PLONE_ADDONS="rapido.plone" plone fg

and got the following output:

Getting distribution for 'mr.developer==1.35'.
Got mr.developer 1.35.
Getting distribution for 'collective.xmltestreport==1.3.4'.
warning: no previously-included files matching '*.pyc' found anywhere in distribution
Got collective.xmltestreport 1.3.4.
Uninstalling precompiler.
Uninstalling zopepy.
Uninstalling instance.
Installing _mr.developer.
Generated script '/plone/instance/bin/develop'.
Updating zeoserver.
Installing instance.
/plone/instance/lib/python2.7/site-packages/pkg_resources/__init__.py:193: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.
  stacklevel=1,
Couldn't find index page for 'rapido.plone' (maybe misspelled?)
Getting distribution for 'rapido.plone'.
Couldn't find index page for 'rapido.plone' (maybe misspelled?)
While:
  Installing instance.
  Getting distribution for 'rapido.plone'.
Error: Couldn't find a distribution for 'rapido.plone'.

without addons
docker run -p 8080:8080 plone fg

works as expected.

hvelarde · October 27, 2017, 1:11pm

seems like a setuptools/buildout issue; I added some debug in out Travis build and it seems buildout is not downloading wheels:

UnsupportedWheel: zc.recipe.egg is in an unsupported or invalid wheel

https://travis-ci.org/plonegovbr/portal.buildout/jobs/293662255

seems a known issue:

the workaround is:

$pip install zc.recipe.egg --no-cache-dir

ngoeddel · October 27, 2017, 1:13pm

Hi. I just figured out how to solve the issue. Go into your Plone directory, then lib/python2.7/site-packages/zc/buildout and change http to https in the file easy_install.py. Now my buildout is able to find packages again.

rodfersou · October 27, 2017, 1:16pm

Need to update zc.buildout to 2.9.5 version to fix it

idgserpro · October 27, 2017, 1:36pm

Well, it seems that all buildouts will fail if you don't already have your eggs downloaded in a cache folder and are using zc.buildout <= 2.9.5. That's why you won't see this error if you already have an environment but it's in CI.

Assuming your python is already a virtualenv, you can run for example:

git clone https://github.com/collective/minimalplone4 && cd minimalplone4 && wget https://bootstrap.pypa.io/bootstrap-buildout.py && python bootstrap-buildout.py && mkdir eggs && bin/buildout buildout:eggs-directory="eggs" buildout:download-cache="eggs" versions:zc.buildout=1.7.1

You will get the error:

Couldn't find index page for 'plone.recipe.zope2instance' (maybe misspelled?)
Getting distribution for 'plone.recipe.zope2instance==4.3'.
Couldn't find index page for 'plone.recipe.zope2instance' (maybe misspelled?)
While:
  Installing.
  Getting section instance.
  Initializing section instance.
  Installing recipe plone.recipe.zope2instance.
  Getting distribution for 'plone.recipe.zope2instance==4.3'.
Error: Couldn't find a distribution for 'plone.recipe.zope2instance==4.3'.

Trying from 1.7.1 to all new buildout versions fail, it works only when we are in 2.9.5. Try for yourself:

git clone https://github.com/collective/minimalplone4 && cd minimalplone4 && wget https://bootstrap.pypa.io/bootstrap-buildout.py && python bootstrap-buildout.py && mkdir eggs && bin/buildout buildout:eggs-directory="eggs" buildout:download-cache="eggs" versions:zc.buildout=2.9.5

The magic is this commit: https://github.com/buildout/buildout/commit/fac9979f516fe1b080258f9b367401a8cc1832e8#diff-db23dcd814354c954091a9b90dbfd92a funny though that if I add index = https://pypi.python.org/simple in my buildout it doesn't work, I need to upgrade zc.buildout to 2.9.5.

FYI @hvelarde and @mauritsvanrees
. I think a change in https://github.com/collective/buildout.plonetest pinning zc.buildout is needed with this justification. More implications in Plone will come, that's why I'm posting it here. Maybe https://github.com/plone/Products.CMFPlone/issues/1502 will need to be revisited.

fredvd · October 27, 2017, 3:08pm

The easiest fix without updating zc.buildout for more recent zc.buildout and setuptools versions is to create/modify your buildout config or the default.cfg (on unixes in ~/.buildout/default.cfg) and add under the [buildout] section:

index = https://pypi.python.org/simple

So that https is used for index access.

idgserpro · October 27, 2017, 3:12pm

Adding this to my buildout didn't work.

warning: no previously-included files matching '*.pyc' found anywhere in distribution
Not found: 
https://pypi.python.org/simple/plone.recipe.zope2instance/
Not found: 
https://pypi.python.org/simple/plone.recipe.zope2instance/
Couldn't find index page for 'plone.recipe.zope2instance' (maybe misspelled?)
Not found: 
https://pypi.python.org/simple/
Getting distribution for 'plone.recipe.zope2instance==4.2.20'.
While:
  Installing.
  Getting section instance.
  Initializing section instance.
  Installing recipe plone.recipe.zope2instance.
  Getting distribution for 'plone.recipe.zope2instance==4.2.20'.
Error: Couldn't find a distribution for 'plone.recipe.zope2instance==4.2.20'.

Do you have shared eggs directories? Did you try to add to a CI configuration?

fredvd · October 27, 2017, 3:18pm

@idgserpro I just spotted your line where you noticed adding index = doesn't work, for me it does, but with zc.buildout 2.8.0 in my buildout(s). Weird indeed.

Jenkins build jobs will start to fail now as well where bootstrap.py is run.

mauritsvanrees · October 27, 2017, 3:46pm

If you use a bootstrap.py for zc.buildout version 1.x, you probably have a wrong ez_setup.py url. For coredev 4.3 I have fixed it here:

idgserpro · October 27, 2017, 4:11pm

https://bootstrap.pypa.io/bootstrap-buildout.py is already using https://bootstrap.pypa.io/ez_setup.py. At least in the Plone 4 tests I made above, I don't think it's related.

mauritsvanrees · October 27, 2017, 4:16pm

Coredev 4.3 uses a bootstrap.py that installs zc.buildout 1.x. It is this one, which has the old ez_setup url:

http://downloads.buildout.org/1/bootstrap.py

I made a pull request for updating it at https://github.com/buildout/buildout/pull/419

mauritsvanrees · October 27, 2017, 4:22pm

Oh, for those who cannot reprocuce the bootstrap error locally, try using a virtualenv without setuptools, otherwise the failing part gets ignored:

  virtualenv --no-setuptools --no-pip --no-wheel --python=python2.7 .

idgserpro · October 27, 2017, 4:33pm

Using your commit in https://github.com/plone/buildout.coredev/commit/316da91ced540273bc161987b43423955ee5d8c7:

git clone https://github.com/collective/minimalplone4 && cd minimalplone4 && wget https://raw.githubusercontent.com/plone/buildout.coredev/316da91ced540273bc161987b43423955ee5d8c7/bootstrap.py && python bootstrap.py && mkdir eggs && bin/buildout buildout:eggs-directory="eggs" buildout:download-cache="eggs" versions:zc.buildout=1.7.1

Gives the same error:

Couldn't find index page for 'plone.recipe.zope2instance' (maybe misspelled?)
Getting distribution for 'plone.recipe.zope2instance==4.3'.
While:
  Installing.
  Getting section instance.
  Initializing section instance.
  Installing recipe plone.recipe.zope2instance.
  Getting distribution for 'plone.recipe.zope2instance==4.3'.
Error: Couldn't find a distribution for 'plone.recipe.zope2instance==4.3'.

But we'll try to reproduce like you said in Buildout not working / Couldn't find index page for / Couldn't find a distribution / Disabling non HTTPS access to APIs on PyPI.

idgserpro · October 27, 2017, 4:42pm

If you already has cached eggs, there's no error, that's why we're using mkdir eggs && bin/buildout buildout:eggs-directory="eggs" buildout:download-cache="eggs", to force it to always download the eggs.

@mauritsvanrees using the virtualenv and different bootstrap (for buildout 1 and 2) we still get the same error if zc.buildout is pinned to any version different from 2.9.5.

mauritsvanrees · October 27, 2017, 10:18pm

@idgserpro: I need the updated bootstrap.py because otherwise the bootstrap call gives an error. After that finishes correctly, and I ignore the eggs cache, I get the same error as you. It works for me when I add this to your last command:

buildout:index=https://pypi.python.org/simple/

tkimnguyen · October 28, 2017, 12:32am

This topic is now a banner. It will appear at the top of every page until it is dismissed by the user.

tkimnguyen · October 28, 2017, 12:33am

This topic is no longer a banner. It will no longer appear at the top of every page.

tkimnguyen · October 28, 2017, 12:34am

h2o · October 28, 2017, 3:15am

I confirm that this solution works for me. I replaced my bootstrap.py with the one at:

https://bootstrap.pypa.io/bootstrap-buildout.py

and added the line in my default.cfg:

index=https://pypi.python.org/simple/

Putting the above line in the [buildout] section of buildout.cfg should work as well.