After some Twitter posts this weekend I promised @tisto to write a post here on how I got Plone 5.2/Python3 running on my new M1 (arm based) mac mini. I wasn't home though and then a PloneConf started. And Timo reported he got things running, so the pressure was off. But I promised....
This is only for tinkering, and fun for now, many projects we depend on to get a Plone stack running on this new architecture are still working on 'official' support. Myself I'm still doing my daily work on the previous OS.X version (never upgrade to a .0 ) on an Intel laptop.
My goal was to get everything running 'natively' with ARM instructions, so not invoking the Rosetta2 transpiling feature. This is one of the main 'tricks' to get up and running: make a copy of the Terminal.app activate 'Rosetta2' on it, start that Terminal.app and install all you want.
I'm doing this writeup from memory and partly from my .zsh_history because I pulled this of 10-12 days ago. YMMV. Some details could be missing, this is not a literal howto.
Soo... The challenge is largely that our stack has a lot of lower level dependencies. Both Python and Plone require compiled libraries installed. I've been using homebrew for this. A large part of homebrew already works on the M1, but you have to install all packages from source and Homebrew advises you to install it not in the standard directory, so use /opt/homebrew. Homebrew advises this on their installation page for now for arm macs until official support is there (Installation — Homebrew Documentation) . I've collected my own knowledge also from pages like these:
Also I was a bit stubborn and I wanted to install Python 3.8.6. And not the Python 3.8.2 that Apple gives you pre-installed. Still, reading how I got Python to compile from source could give you tips for other things you want to get installed.
Before you start installing homebrew, if you have the bandwidth and patience, download and install Xcode 2.3beta. It contains some essential resources we need later. Other online sources say you should have enough by downloading the XCode command line tools, which also contains a compact version of the macosX SDK. (the full xcode 2.3 also contains it) and it has git and other command line goodies, but then you have to select/switch between the two using xcode-select. (xcode-select -install adds the command line tools, but Xcode 2.2 will add 2.2. command line tools, which is not what you want). I got stuck later on with just the command line tools 2.3beta downloaded from Apple's developer site, so I removed XCode 2.2/2.3beta commandline tools and installed the full XCode 2.3 beta.
Building Python later didn't work for me the first 20 trial and error attempts because early in the process it would get stuck on the zlib dependency. This library is also in this MacosX.sdk. And I honestly don't know anymore if I solved this by passing homebrews zlib into LDFLAGS/CFLAGS, or if the build still picks zlib from the MacOSX.sdk. According to other resources the 2.3 beta commandline tools should have worked as well. If you get stuck: go for and try the full Xcode 2.3 beta.
So Xcode 2.3beta, install homebrew to /opt/homebrew , add /opt/homebrew/bin to your path and you can install multiple libraries like zlib2, bzip2, libjep automake, autoconf, openssl, sqlite, etc. using brew install.
Oh and do check you are adding your path to .zshrc, as zsh is the default shell on BigSur. (you could install bash through homebrew and continue with a fresh bash version happily ever after).
At the time of tinkering only Python3.9 was working out of the box with homebrew, but as I learned from Twitter 2-3 weeks ago (thank you Joni & Roel) homebrew only cares about Python because of their other packages depending on a python version. So on my intel mac all my Python3.7 got broken after a 'brew upgrade' because the maintainers decided that 3.9 was shiny enough and python3.7 got removed. Python3.8 is a 'kegg only' release, not meant for consumption by other homebrew packages, so python3.8 from homebrew stays installed. But “brews install -s python@3.8” is not supporter yet.
pyenv to the rescue. (Check the installation info for pyenv because it needs some extra stuff in your path to work properly. ) You can install it with homebrew on your M1 and install Python. Right? Nope.
You installed homebrew not in /usr/local/, so you need to pass 'ALL' locations to LDFLAGS and CFLAGS for those low level libraries that the python interpreter needs. And, Python 3.8.X needs some ARM patches. Fortunately the homebrew project already created those, so install the libs in the incantation below and this will get you a working Python 3.8.6 binary:
PYTHON_CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl)" \ CFLAGS="-I$(brew --prefix zlib)/include -I$(brew --prefix sqlite)/include -I$(brew --prefix bzip2)/include" \ LDFLAGS="-L$(brew --prefix zlib)/lib -L$(brew --prefix sqlite)/lib -L$(brew --prefix bzip2)/lib" \ pyenv install --patch 3.8.6 <<(curl -sSL https://raw.githubusercontent.com/Homebrew/formula-patches/113aa84/python/3.8.3.patch\?full_index\=1)
Well, almost, because pyenv on my machine got a bit hazy, or I mistook the installation instructions. pyenv 'saw' my python 3.8.6, but wouldn't activate it when I said pyenv local 3.8.6 in a directory. Put the ~/.pyenv/shims directory on your path instead of ~/.pyenv/bin and then it worked for me. Jay!.
Now you can run python3 -m -venv . and you have a nice virtualenv and a python 3.8.6.
Fetch an existing Plone 5.2 project, bin/pip install -r requirements.xt ; bin/buildout, and go.... Ehm, not yet.
There's Pillow, which needs some libraries like png to exist. Pillow is very install friendly and searches those libraries on macs with homebrew under /usr/local/homebrew, but not under /opt/homebrew. When your bin/buildout starts to install the python packages and Pillow is built, it lists what it has found. You can use the same trick as above to extend LDFLAGS and CFLAGS before calling buildout. Another route is to have pkg-config from homebrew on your path, which will give Pillow another tool to request the installation paths of libpng and a few other libraries.
Last hurdle I remember having, but my .zsh_history no longer tells me is that I had to manually install the cffi python package in my virtualenv before buildout was able to compile Zope's python C-extensions. But the error message will sort of point you to that.
And that's how I got Plone 5.2 running for the moment on Python 3.8.6 .
I also tried to get Python2 running, but "here be dragons". There are no 'arm' patches yet to compile python2 that I found. But you don't need to compile python2 because Apple also provides a system Python 2.7.16.
But that's nothing to cheer, because the real problem with Python2 and Plone development is that you want to use virtualenv. And virtualenv breaks with python2. The problem seems to be that virtualenv modifies the python executable in the installed env, and Big Sur has new codesign measures that kill the process on startup because the binary has been modified. On the other hand people report it doesn't seem to happen with Big Sur on Intel macs so something else is going on as well. Got this info I think from a pyenv GitHub issue, but can't re-find it at the moment.
( python3's -m venv or virtualenv don't have this Python2 issue as Python3 is relocatable through symlinks or paths or other less nasty things than rewriting the ExecutablePath in the virtualenv binary)
