Hi everyone,
I am working on a rather large Plone 4 project and trying to refresh my build using buildout I recently noticed a time-out on http://download.zope.org/.
Is this a known issue? Is there a mirror?
Kind regards,
Markus
@esteele, would it be possible to adjust the different version.cfg files on dist.plone.org and add links to local files (I could help with that if you can tell me where to change it)? And do we have backups of the linked files from download.zope.org somewhere?
@derdritte do you have the complete error traceback on your terminal? More people may have this problem and it would be nice to have this indexed in google.
I tested with a legacy buildout here and got:
An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/buildout.py", line 1995, in main
getattr(buildout, command)(args)
File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/buildout.py", line 666, in install
installed_files = self[part]._call(recipe.install)
File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/buildout.py", line 1410, in _call
return f()
File "/home/user/instances/buildout-cache/eggs/plone.recipe.zope2instance-4.2.21-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 113, in install
installed.extend(self.install_scripts())
File "/home/user/instances/buildout-cache/eggs/plone.recipe.zope2instance-4.2.21-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 617, in install_scripts
requirements, ws = self.egg.working_set(['plone.recipe.zope2instance'])
File "/home/user/instances/buildout-cache/eggs/zc.recipe.egg-1.3.2-py2.7.egg/zc/recipe/egg/egg.py", line 101, in working_set
**kw)
File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 859, in install
return installer.install(specs, working_set)
File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 643, in install
for_buildout_run=for_buildout_run):
File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 488, in _get_dist
dist, avail = self._satisfied(requirement)
File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 253, in _satisfied
return None, self._obtain(req, source)
File "/home/user/instances/buildout-cache/eggs/zc.buildout-2.5.3-py2.7.egg/zc/buildout/easy_install.py", line 413, in _obtain
if index.obtain(requirement) is None:
File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 489, in obtain
self.prescan()
File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 530, in prescan
list(map(self.scan_url, self.to_scan))
File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 806, in scan_url
self.process_url(url, True)
File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 338, in process_url
f = self.open_url(url, tmpl % url)
File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 745, in open_url
return open_with_auth(url, self.opener)
File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 946, in _socket_timeout
return func(*args, **kwargs)
File "/home/user/instances/buildout-cache/eggs/setuptools-27.3.0-py2.7.egg/setuptools/package_index.py", line 1064, in open_with_auth
fp = opener(request)
File "/usr/lib/python2.7/urllib2.py", line 400, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 418, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open
r = h.getresponse(buffering=True)
File "/usr/lib/python2.7/httplib.py", line 1076, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 444, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 400, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 476, in readline
data = self._sock.recv(self._rbufsize)
timeout: timed out
Apologies, I do indeed have a trace.
I might add that this isn't exactly the most useful trace.
You may start buildout with the -D flat to get a post-mortem debugger, which is a bit more informative.
% bin/buildout -N
While:
Initializing.
An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1682, in main
user_defaults, windows_restart, command)
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 198, in __init__
data['buildout'].copy(), override))
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
eresult = _open(base, extends.pop(0), seen, dl_options, override)
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
eresult = _open(base, extends.pop(0), seen, dl_options, override)
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
eresult = _open(base, extends.pop(0), seen, dl_options, override)
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1358, in _open
_update(eresult, _open(base, fname, seen, dl_options, override))
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
eresult = _open(base, extends.pop(0), seen, dl_options, override)
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1356, in _open
eresult = _open(base, extends.pop(0), seen, dl_options, override)
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/buildout.py", line 1306, in _open
path, is_temp = download(filename)
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/download.py", line 96, in __call__
local_path, is_temp = self.download(url, md5sum, path)
File "/home/vagrant/wieninfo-buildout/eggs/zc.buildout-1.4.4-py2.7.egg/zc/buildout/download.py", line 173, in download
tmp_path, headers = urllib.urlretrieve(url, tmp_path)
File "/usr/lib/python2.7/urllib.py", line 94, in urlretrieve
return _urlopener.retrieve(url, filename, reporthook, data)
File "/usr/lib/python2.7/urllib.py", line 240, in retrieve
fp = self.open(url, data)
File "/usr/lib/python2.7/urllib.py", line 208, in open
return getattr(self, name)(url)
File "/usr/lib/python2.7/urllib.py", line 345, in open_http
h.endheaders(data)
File "/usr/lib/python2.7/httplib.py", line 1013, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 864, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 826, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 807, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
IOError: [Errno socket error] [Errno 110] Connection timed outAnyone able to get the site online again?
All the unreachable versions are available at http://dist.plone.org/versions/.
I adjusted the buildout.plonetest files to use custom versions and link to the ones from dist.plone.org. So all test setups extending buildout.plonetest should now be working again.
It would be nice if we can adjust the versions.cfg's for several 4.0.x, 4.1.x, 4.2.x and 4.3.x (latest is working) to include the Zope versions from dist.plone.org. Any hint on how to do this or who to ping is appreciated.
I understand this is an emergency but please don't do this kind of changes in master branch without further discussion; IMO, this issue must be fixed on versions.cfg files in http://dist.plone.org/ instead.
Just to follow up on this:
Will recipes on the (more stable) dist.plone.org be updated to no longer reference zope.org? Will zope.org be migrated or management be taken over? It seems this will become an issue again.
I sent you the files long time ago to update dist.plone.org. Can we please have this as download.zope.org is down again. BTW the files are available at my GitHub.
Does anybody know what's up with @esteele? Is he currently unavailable? Can someone ping him maybe on other channels?
I've been asking that for weeks:
Still alive at least on Github. I'm not worried yet, I trust clarity will ensue over time.
seems he's busy working on some private projects.
AFAIK he is not employed by the Plone Foundation to be the release manager, so yes: work, family and live(TM) is something that every one of us is enjoying as well.
Since two Sorrento's sprints (i.e. 1.5 years) we have a release team:
https://github.com/orgs/plone/teams/release-team
So rather than calling individuals, and fingerpoint to a specific person, it is always better to point to a group 
There are still some parts of the release management that he is keeping for himself, we (the other release team members) should push him a bit to get us in there as well, so we effectively could do all the jobs required.
As for this specific case (download.zope.org down) it would be helpful if someone has the archives ready to be pushed there, as the infra team has also access to dist.plone.org I guess.
Regarding your other issue (newere 4.3.x release) with the latest security hotfix, we usually block the next release until the patches are merged, it does not make sense that a security patch releases in November and a new release made in December still has to carry that hotfix. I know that we could do that, but that would be just even more confusing for non-insiders.
1 Like
That's good to know. Anyhow, I tried to ping Eric when the server was down for the first time to help him with the versions. No response. I talked to him at the conference and we agreed that I will send him the files, so I don't need access to the server directly. I did this but nothing happened and now the server is down again.
I think most of us rely on the version.cfg's from download.zope.org for some older projects and/or testing, because they are linked from dist.plone.org. They are already available there, just not linked from the older version.cfg's. So it's "just" a simple copy/paste of the files in GitHub - thomasmassmann/dist.plone.org_versions: Corrected version.cfg's for dist.plone.org. If someone is in doubt about the correctness, there are 2 commits, first with the original versions, and then with the new links.
So, if someone from the "infra team" please could help out here would be great.