Buildout fails: download.zope.org not reachable

Hi everyone,

I am working on a rather large Plone 4 project and trying to refresh my build using buildout I recently noticed a time-out on http://download.zope.org/.

Is this a known issue? Is there a mirror?

Kind regards,
Markus

Indeed, we can't reach http://download.zope.org/ in the browser as well.

Yes, since yesterday: Zope.org breakdown and old Plone's versions.cfg

Maybe @jfroche or @gotcha can help again.

@esteele, would it be possible to adjust the different version.cfg files on dist.plone.org and add links to local files (I could help with that if you can tell me where to change it)? And do we have backups of the linked files from download.zope.org somewhere?

@derdritte do you have the complete error traceback on your terminal? More people may have this problem and it would be nice to have this indexed in google.

I tested with a legacy buildout here and got:

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/buildout.py", line 1995, in main
    getattr(buildout, command)(args)
  File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/buildout.py", line 666, in install
    installed_files = self[part]._call(recipe.install)
  File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/buildout.py", line 1410, in _call
    return f()
  File "/home/user/instances/buildout-cache/eggs/plone.recipe.zope2instance-4.2.21-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 113, in install
    installed.extend(self.install_scripts())
  File "/home/user/instances/buildout-cache/eggs/plone.recipe.zope2instance-4.2.21-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 617, in install_scripts
    requirements, ws = self.egg.working_set(['plone.recipe.zope2instance'])
  File "/home/user/instances/buildout-cache/eggs/zc.recipe.egg-1.3.2-py2.7.egg/zc/recipe/egg/egg.py", line 101, in working_set
    **kw)
  File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 859, in install
    return installer.install(specs, working_set)
  File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 643, in install
    for_buildout_run=for_buildout_run):
  File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 488, in _get_dist
    dist, avail = self._satisfied(requirement)
  File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 253, in _satisfied
    return None, self._obtain(req, source)
  File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 413, in _obtain
    if index.obtain(requirement) is None:
  File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 489, in obtain
    self.prescan()
  File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 530, in prescan
    list(map(self.scan_url, self.to_scan))
  File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 806, in scan_url
    self.process_url(url, True)
  File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 338, in process_url
    f = self.open_url(url, tmpl % url)
  File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 745, in open_url
    return open_with_auth(url, self.opener)
  File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 946, in _socket_timeout
    return func(*args, **kwargs)
  File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 1064, in open_with_auth
    fp = opener(request)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1076, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 444, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 400, in _read_status
    line = self.fp.readline(_MAXLINE + 1)
  File "/usr/lib/python2.7/socket.py", line 476, in readline
    data = self._sock.recv(self._rbufsize)
timeout: timed out

Apologies, I do indeed have a trace.
I might add that this isn't exactly the most useful trace.
You may start buildout with the -D flat to get a post-mortem debugger, which is a bit more informative.

 % bin/buildout -N
While:
  Initializing.

An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1682, in main
    user_defaults, windows_restart, command)
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 198, in __init__
    data['buildout'].copy(), override))
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1358, in _open
    _update(eresult, _open(base, fname, seen, dl_options, override))
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
    eresult = _open(base, extends.pop(0), seen, dl_options, override)
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1306, in _open
    path, is_temp = download(filename)
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/download.py", line 96, in __call__
    local_path, is_temp = self.download(url, md5sum, path)
  File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/download.py", line 173, in download
    tmp_path, headers = urllib.urlretrieve(url, tmp_path)
  File "/usr/lib/python2.7/urllib.py", line 94, in urlretrieve
    return _urlopener.retrieve(url, filename, reporthook, data)
  File "/usr/lib/python2.7/urllib.py", line 240, in retrieve
    fp = self.open(url, data)
  File "/usr/lib/python2.7/urllib.py", line 208, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.7/urllib.py", line 345, in open_http
    h.endheaders(data)
  File "/usr/lib/python2.7/httplib.py", line 1013, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 864, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 826, in send
    self.connect()
  File "/usr/lib/python2.7/httplib.py", line 807, in connect
    self.timeout, self.source_address)
  File "/usr/lib/python2.7/socket.py", line 571, in create_connection
    raise err
IOError: [Errno socket error] [Errno 110] Connection timed out

Anyone able to get the site online again?

maybe try https://web.archive.org/web/*/http://download.zope.org ?

All the unreachable versions are available at http://dist.plone.org/versions/.

I adjusted the buildout.plonetest files to use custom versions and link to the ones from dist.plone.org. So all test setups extending buildout.plonetest should now be working again.

It would be nice if we can adjust the versions.cfg's for several 4.0.x, 4.1.x, 4.2.x and 4.3.x (latest is working) to include the Zope versions from dist.plone.org. Any hint on how to do this or who to ping is appreciated.

I understand this is an emergency but please don't do this kind of changes in master branch without further discussion; IMO, this issue must be fixed on versions.cfg files in http://dist.plone.org/ instead.

I'll work on updating the files on dist.plone.org.

2 Likes

FYI: zope.org is reachable again.

1 Like

Just to follow up on this:
Will recipes on the (more stable) dist.plone.org be updated to no longer reference zope.org? Will zope.org be migrated or management be taken over? It seems this will become an issue again.

I sent you the files long time ago to update dist.plone.org. Can we please have this as download.zope.org is down again. BTW the files are available at my GitHub.

Does anybody know what's up with @esteele? Is he currently unavailable? Can someone ping him maybe on other channels?

I've been asking that for weeks:

Still alive at least on Github. I'm not worried yet, I trust clarity will ensue over time.

seems he's busy working on some private projects.

AFAIK he is not employed by the Plone Foundation to be the release manager, so yes: work, family and live(TM) is something that every one of us is enjoying as well.

Since two Sorrento's sprints (i.e. 1.5 years) we have a release team:

https://github.com/orgs/plone/teams/release-team

So rather than calling individuals, and fingerpoint to a specific person, it is always better to point to a group :slight_smile:

There are still some parts of the release management that he is keeping for himself, we (the other release team members) should push him a bit to get us in there as well, so we effectively could do all the jobs required.

As for this specific case (download.zope.org down) it would be helpful if someone has the archives ready to be pushed there, as the infra team has also access to dist.plone.org I guess.

Regarding your other issue (newere 4.3.x release) with the latest security hotfix, we usually block the next release until the patches are merged, it does not make sense that a security patch releases in November and a new release made in December still has to carry that hotfix. I know that we could do that, but that would be just even more confusing for non-insiders.

1 Like

That's good to know. Anyhow, I tried to ping Eric when the server was down for the first time to help him with the versions. No response. I talked to him at the conference and we agreed that I will send him the files, so I don't need access to the server directly. I did this but nothing happened and now the server is down again.

I think most of us rely on the version.cfg's from download.zope.org for some older projects and/or testing, because they are linked from dist.plone.org. They are already available there, just not linked from the older version.cfg's. So it's "just" a simple copy/paste of the files in https://github.com/tmassman/dist.plone.org_versions. If someone is in doubt about the correctness, there are 2 commits, first with the original versions, and then with the new links.

So, if someone from the "infra team" please could help out here would be great.