I noticed that the Universal installer for 5.2.2 installs Python3.6 if you state --build-python=3
Is it better to use 3.6 (than never versions) … for any reason ?
No, all 3.6 to 3.8 works. Probably it is better to use 3.8, since its the most recent (and most performant) version. No idea why the installer do not use it. Maybe there is a reason @tkimnguyen knows?
[update]
Given the flaky Robot tests on 3.6 only I tend to not recommend 3.6 anymore if someone asks me...
But this completely intuitively.
[/update]
The most immediate reason is that I have not looked into it. I wanted to get a working 5.2.2 installer released as a first priority, followed by installers for 4.3 and 5.1. I will next update the Vagrant, then would look into the Heroku trial setup (or remove it from plone.org/download if it isn’t easily updatable).
But it’s easy to issue new versions of the unified installer so there’s hope yet.
I will add: I hesitate to change things if they don’t need to be changed. Before I would change the downloaded Python to 3.7 or 3.8, I would like to know if Plone has been fully tested with each of those versions. I know from experience that it seems that 3.7 is fine with Plone, but can someone say definitively that all tests pass? Same for 3.8.
And: it would be best if the —build-python option remained a choice of last resort, since it is almost always better to have Python installed via a system package; that way, it will receive updates. A locally source-built Python will not get updated unless someone takes explicit (and time consuming, effort-requiring) action to do so manually.
All my 5.2.2 deployments are running on 3.8 latest w/o any problems.
Jenkins says yes: https://jenkins.plone.org/
Just for Python 3.6 we sometimes get flaky Robot tests.
So the opposite might be true.
Btw., did you test the installer on Windows? I am just curious...
I asked @mauritsvanrees the same question two or three weeks ago for an upcoming upgrade. Ubuntu 18.04 LTS official package management stops at Python 3.7.5, latest is 3.7.9 . But the half-official deadsnakes extras repo will give you the latest & greatest Python 3.8.5 for 18.04 and intentionally does not provide the latest OS distributed python version so it can't break os dependencies. And I don't fancy Ubuntu 20.04 LTS yet for different reasons, so I wanted to know if 3.8 was/is safe.
Supported Python version is when all available tests pass for that Python. Python 3.7 is supported as Jen(s)kins points out. The issue with Python3.8 as Maurits told me is that a few Zope ZEO tests are flaky/failing, but it is more likely if this is because of an already fragile test setup or really an issue with Python3.8. But I can't find the link from The Github issue at the moment which Maurits sent me. 
For failing ZEO tests on 3.8, see https://github.com/zopefoundation/ZEO/pull/159#issuecomment-663960043
I did not, because I knew it did not work correctly for the 5.2.1 unified installer. I tested it with macOS 10.15, and Ubuntu 18 LTS and 20 LTS.
The next time I touch the unified installer I will switch it to use Python 3.7
Would it be possible / could it be an idea to be able to choose python version?
--build-python="3.7" (or --build-python="3.6" / "3.8" ) ?
1 Like
Hell, the idea of Nils and my work was to make it work on Win 10. Its almost done, So I expected it to work on Win 10 if published. We need automated tests here, otherwise this is a roulette every single release. With GitHub Actions we have all three environments at our fingertips, so, it should not be that hard.
As of the master branch, the Unified Installer now builds Python 3.8.5.
I have looked into the specification of arbitrary Python versions already, and there is a pull request for it.
Yes, it supports something like --build-python=3.7 (using the currently best 3.7 version, 3.7.9), and as well --python-prefix=/opt/ (will usually need to be run as root) to create it as /opt/Python-3.7.
We need to stabilize what’s going on in that repo. A process followed by everyone for tagging a release or creating a branch. We can’t have multiple people merging changes into master without this, especially without proper and consistent testing prior to merging.
Agreed. That's why I didn't simply merge my bigger changes there but made a pull request.
But I couldn't find a branch for the 5.2.x releases, and I couldn't find documentation about releasing and the like; so all I could do was refer to the master branch.
We should certainly be able to compile a recent Python interpreter, rather than one which has already reached security-only stage; especially since the Plone appararently runs nicely with Python 3.7 and 3.8. This is tested elsewhere.
The current installer script certainly (and dearly) needs refactoring (and more than I did already):
- If something doesn't work, you are lost because of the insufficient debug information. (My little
loggedfunction helps there, for a start, because you can see where something failed). - It is a hell of spaghetti code and can quite well use some structure, established by shell functions.
- Those pairs of variables (
PYTHON_URL,PYTHON_MD5,PYTHON_TB,PYTHON_DIR,WANT_PYTHON... all in Python 2 and Python 3 versions) can really be replaced by something cleaner.
My improved main_install_script.sh uses array variables which are, admittedly, not available in the Bourne shell. We might need to support non-array shells; but we already have an install.sh in front which detects bash, and we could easily generate a non-array version for sh when running buildme.sh.
Thanks @tobiasherp
Yes, there should have been a 5.2 branch, but I didn't realize that until this week. I need to merge a couple of remaining outstanding things from my initial release of the 5.2.2 installer (mea culpa), and then resolve to do my work only locally so that multiple people can contribute to this repo. When I accepted to take responsibility for this from Steve McMahon, I had assumed that I'd be the only one working on it, as he had mostly been.
You're never alone with Plone.
My being "benevolent dictator until retirement" for the installer was probably not great for the community. But what you're mainly running into is an integration and testing problem. The UI needs to be tested on about five platforms, not just one, so we never developed a continuous testing and integration scheme that could protect against pull requests that broke things.
(By the way, it's actually much worse than five platforms if you count having to work with both custom built and platform versions of Python. And the testing load doubled with the need to work with both 2.7.x and 3.x. All this meant that a simple #.#.# update might require a couple of days of testing on three different machines [Linux, Windows, Mac] and multiple virtual environments [to cover recent versions of Red Hat and Debian flavors].)
1 Like
This wasn’t meant as a complaint, more as an explanation why the process was less defined than it should be if the repo were to be contributed to by multiple people.
I've created a 5.2 branch from master today.
@tobiasherp I am looking at this commit https://github.com/plone/Installers-UnifiedInstaller/commit/91dad5ec8f16fc6415ce0232dcae736e51e84c42 ... can you tell me where versions-prod.cfg gets created? Is that the file name we would use instead of, say, Zope-releases-4.5.1-versions.cfg?
The file name is set by the fetch_versions.py script:
$ ../fetch_versions.py 5.2.2
requirements.txt
Zope-releases-4.5.1-versions-prod.cfg
Zope-releases-4.5.1-versions.cfg
release-5.2.2-versions.cfgMaybe I'm misunderstanding here but do we want to be adding to this repo these version files? The process of creating the unified installer for, say, 5.2.2, fetches these and packages them into the installer, so adding them to the repo seems overkill.