Just wanted to point out that there is new tool "uv". "uv" is a pip replacement, written in Rust, blazing fast.
uv pip install Plone
...takes less than one second for installing Plone from cached downloads and just a few seconds with download all 1XX packages.
Also, "uv" can be used as a drop-in replacement for venv.
uv venv -p python3.12 <target_dir>
is doing the same as
python3.12 -m venv <target_dir>
...but blazing fast.
Give it a try.
(test) yurj@mypc:/tmp/test$ uv pip install Plone
Resolved 236 packages in 5.61s
Built plone-app-locales==6.0.20
Built repoze-xmliter==0.6.1
Built sgmllib3k==1.0.0 Downloaded 235 packages in 6.88s
Installed 236 packages in 347ms
+ accesscontrol==6.3
[...]
+ zope-viewlet==5.0
(test1) yurj@mypc:/tmp/test1$ uv pip install Plone
Resolved 236 packages in 121ms
Installed 236 packages in 258ms
+ accesscontrol==6.3
[...]
+ zope-viewlet==5.0
I was trying it some days ago and it failed for me 
I think the problem is that extras of non-direct dependencies are not resolved so far.
It looks promising though 
mxmake with uv (branch) install Plone in 4s 
(wheels in cache, uv preinstalled globally):
(venv) ~/ws/sandbox/cplone$ make clean
(venv) ~/ws/sandbox/cplone$ time make install
Setup Python Virtual Environment using package 'uv' at 'venv'
Using Python 3.12.2 interpreter at /home/jensens/.pyenv/versions/3.12.2/bin/python3
Creating virtualenv at: venv
+ pip==24.0
Activate with: source venv/bin/activate
Resolved 3 packages in 1ms
Installed 2 packages in 13ms
+ setuptools==69.1.1
+ wheel==0.42.0
Install/Update MXStack Python packages
Resolved 13 packages in 2ms
Installed 12 packages in 5ms
+ blessed==1.20.0
+ editor==1.6.6
+ inquirer==3.2.4
+ jinja2==3.1.3
+ markupsafe==2.1.5
+ mxdev==3.1.0
+ mxmake==1.0a3
+ readchar==4.0.5
+ runs==1.2.2
+ six==1.16.0
+ wcwidth==0.2.13
+ xmod==1.8.1
Install zpretty
Resolved 5 packages in 3ms
Installed 4 packages in 26ms
+ beautifulsoup4==4.12.3
+ lxml==5.1.0
+ soupsieve==2.5
+ zpretty==3.1.0
Install Ruff
Resolved 1 package in 6ms
Installed 1 package in 16ms
+ ruff==0.2.2
Install pyupgrade
Resolved 2 packages in 1ms
Installed 2 packages in 2ms
+ pyupgrade==3.15.1
+ tokenize-rt==5.2.0
Install isort
Resolved 1 package in 0ms
Installed 1 package in 3ms
+ isort==5.13.2
Install Black
Resolved 6 packages in 1ms
Installed 6 packages in 10ms
+ black==24.2.0
+ click==8.1.7
+ mypy-extensions==1.0.0
+ packaging==23.2
+ pathspec==0.12.1
+ platformdirs==4.2.0
Checkout project sources
###############################################################################
mxmake: hook initialized
# Load configuration
Can not parse override:
###############################################################################
# Read infiles
Read [r]: requirements.txt
Read [c]: constraints.txt
Read [c]: https://dist.plone.org/release/6.0.9/constraints.txt
###############################################################################
# No sources configured!
###############################################################################
# Write outfiles
Write [c]: constraints-mxdev.txt
Write [r]: requirements-mxdev.txt
🎂 You are now ready for: pip install -r requirements-mxdev.txt
(path to pip may vary dependent on your installation method)
Create project files
###############################################################################
mxmake: hook initialized
# Load configuration
Can not parse override:
###############################################################################
# Read infiles
Read [r]: requirements.txt
Read [c]: constraints.txt
Read [c]: https://dist.plone.org/release/6.0.9/constraints.txt
###############################################################################
# Write outfiles
Write [c]: constraints-mxdev.txt
Write [r]: requirements-mxdev.txt
🎂 You are now ready for: pip install -r requirements-mxdev.txt
(path to pip may vary dependent on your installation method)
Install python packages
Resolved 239 packages in 38ms
Installed 237 packages in 415ms
+ accesscontrol==6.3
+ acquisition==5.1
+ attrs==23.1.0
+ authencoding==5.0
- beautifulsoup4==4.12.3
+ beautifulsoup4==4.12.2
+ borg-localrole==3.1.11
+ btrees==5.1
+ certifi==2023.11.17
+ cffi==1.16.0
+ chameleon==4.2.0
+ charset-normalizer==3.3.2
+ collective-monkeypatcher==1.2.1
+ cssselect==1.2.0
+ datetime==5.3
+ decorator==5.1.1
+ diazo==1.5.0
+ documenttemplate==4.6
+ docutils==0.18.1
+ extensionclass==5.1
+ feedparser==6.0.10
+ five-customerize==2.1.0
+ five-intid==2.0.0
+ five-localsitemanager==4.0
+ future==0.18.3
+ icalendar==5.0.11
+ idna==3.4
+ jsonschema==4.20.0
+ jsonschema-specifications==2023.11.2
- lxml==5.1.0
+ lxml==4.9.3
+ markdown==3.5.1
+ missing==5.0
+ multimapping==5.0
+ multipart==0.2.4
+ paste==3.7.1
+ pastedeploy==3.1.0
+ persistence==4.1
+ persistent==5.1
+ piexif==1.1.3
+ pillow==9.5.0
+ plone==6.0.9
+ plone-alterego==2.0.0
+ plone-api==2.0.8
+ plone-app-caching==3.1.3
+ plone-app-content==4.1.0
+ plone-app-contentlisting==3.0.4
+ plone-app-contentmenu==3.0.3
+ plone-app-contentrules==5.0.3
+ plone-app-contenttypes==3.0.5
+ plone-app-customerize==2.0.1
+ plone-app-dexterity==3.2.0
+ plone-app-discussion==4.0.2
+ plone-app-event==5.1.1
+ plone-app-i18n==4.0.1
+ plone-app-intid==2.0.0
+ plone-app-iterate==5.0.2
+ plone-app-layout==4.0.7
+ plone-app-linkintegrity==4.0.3
+ plone-app-locales==6.0.18
+ plone-app-lockingbehavior==2.0.0
+ plone-app-multilingual==7.0.3
+ plone-app-portlets==5.0.6
+ plone-app-querystring==2.1.1
+ plone-app-redirector==3.0.1
+ plone-app-registry==2.0.4
+ plone-app-relationfield==3.0.3
+ plone-app-textfield==2.0.0
+ plone-app-theming==5.0.6
+ plone-app-upgrade==3.1.1
+ plone-app-users==3.0.4
+ plone-app-uuid==2.2.2
+ plone-app-versioningbehavior==2.0.1
+ plone-app-viewletmanager==4.0.3
+ plone-app-vocabularies==5.0.4
+ plone-app-widgets==5.0.0
+ plone-app-workflow==5.0.3
+ plone-app-z3cform==4.3.0
+ plone-autoform==2.0.1
+ plone-autoinclude==1.0.1
+ plone-base==1.2.0
+ plone-batching==2.0.6
+ plone-behavior==2.0.0
+ plone-browserlayer==3.0.2
+ plone-cachepurging==3.0.2
+ plone-caching==2.0.1
+ plone-contentrules==3.0.1
+ plone-dexterity==3.0.3
+ plone-event==2.0.1
+ plone-folder==4.0.0
+ plone-formwidget-namedfile==3.0.3
+ plone-formwidget-recurrence==3.0.3
+ plone-i18n==5.0.2
+ plone-indexer==2.0.0
+ plone-intelligenttext==4.0.1
+ plone-keyring==4.0.0
+ plone-locking==3.0.0
+ plone-memoize==3.0.2
+ plone-namedfile==6.2.3
+ plone-outputfilters==5.0.4
+ plone-portlet-collection==4.0.3
+ plone-portlet-static==4.0.1
+ plone-portlets==3.0.0
+ plone-protect==5.0.0
+ plone-registry==2.0.0
+ plone-resource==3.0.0
+ plone-resourceeditor==4.0.1
+ plone-rest==4.1.2
+ plone-restapi==9.2.1
+ plone-rfc822==3.0.0
+ plone-scale==4.1.0
+ plone-schema==2.0.1
+ plone-schemaeditor==4.0.5
+ plone-session==4.0.4
+ plone-staticresources==2.1.9
+ plone-stringinterp==2.0.1
+ plone-subrequest==2.0.4
+ plone-supermodel==2.0.3
+ plone-theme==4.0.1
+ plone-transformchain==3.0.0
+ plone-uuid==2.0.0
+ plone-volto==4.2.0
+ plone-z3cform==2.0.3
+ plonetheme-barceloneta==3.1.7
+ products-btreefolder2==5.1
+ products-cmfcore==3.2
+ products-cmfdifftool==4.0.1
+ products-cmfdynamicviewfti==7.0.2
+ products-cmfeditions==4.0.2
+ products-cmfplacefulworkflow==3.0.3
+ products-cmfplone==6.0.9
+ products-cmfuid==4.1
+ products-daterecurringindex==3.0.1
+ products-dcworkflow==3.0
+ products-extendedpathindex==4.0.1
+ products-externalmethod==5.0
+ products-genericsetup==3.0.1
+ products-isurlinportal==2.0.1
+ products-mailhost==5.0
+ products-mimetypesregistry==3.0.1
+ products-plonepas==8.0.2
+ products-pluggableauthservice==2.8.1
+ products-pluginregistry==2.0
+ products-portaltransforms==4.0.1
+ products-pythonscripts==5.0
+ products-sessions==5.0
+ products-siteerrorlog==6.0
+ products-standardcachemanagers==5.0
+ products-statusmessages==5.0.6
+ products-zcatalog==7.0
+ products-zopeversioncontrol==4.1
+ pycparser==2.21
+ pyjwt==2.8.0
+ python-dateutil==2.8.2
+ python-gettext==5.0
+ pytz==2023.3.post1
+ record==4.0
+ referencing==0.31.1
+ repoze-xmliter==0.6.1
+ requests==2.31.0
+ restrictedpython==7.0
+ roman==4.1
+ rpds-py==0.13.2
- setuptools==69.1.1
+ setuptools==69.0.2
+ sgmllib3k==1.0.0
+ simplejson==3.19.2
+ transaction==4.0
+ unidecode==1.3.7
+ urllib3==2.1.0
+ waitress==2.1.2
+ waitress-fastlisten==1.0.0rc1
+ webob==1.8.7
+ webresource==1.2
+ webtest==3.0.0
+ wsgiproxy2==0.5.1
+ z3c-caching==3.0
+ z3c-form==4.3
+ z3c-formwidget-query==2.0.0
+ z3c-objpath==2.0
+ z3c-pt==4.0
+ z3c-relationfield==1.0
+ z3c-zcmlhook==2.0
+ zc-lockfile==3.0.post1
+ zc-relation==2.0
+ zconfig==4.0
+ zexceptions==5.0
+ zodb==5.8.1
+ zodbpickle==3.1
+ zope==5.9
+ zope-annotation==5.0
+ zope-app-locales==5.0
+ zope-browser==3.0
+ zope-browsermenu==5.0
+ zope-browserpage==5.0
+ zope-browserresource==5.1
+ zope-cachedescriptors==5.0
+ zope-component==6.0
+ zope-componentvocabulary==2.3.0
+ zope-configuration==5.0
+ zope-container==5.2
+ zope-contentprovider==5.0
+ zope-contenttype==5.1
+ zope-copy==4.3
+ zope-datetime==5.0.0
+ zope-deferredimport==5.0
+ zope-deprecation==5.0
+ zope-dottedname==6.0
+ zope-event==5.0
+ zope-exceptions==5.0.1
+ zope-filerepresentation==6.0
+ zope-globalrequest==2.0
+ zope-hookable==6.0
+ zope-i18n==5.1
+ zope-i18nmessageid==6.1.0
+ zope-interface==6.1
+ zope-intid==5.0
+ zope-keyreference==6.0
+ zope-lifecycleevent==5.0
+ zope-location==5.0
+ zope-pagetemplate==5.0
+ zope-processlifetime==3.0
+ zope-proxy==5.1
+ zope-ptresource==5.0
+ zope-publisher==7.0
+ zope-ramcache==3.0
+ zope-schema==7.0.1
+ zope-security==6.2
+ zope-sendmail==6.0
+ zope-sequencesort==5.0
+ zope-site==5.0
+ zope-size==5.0
+ zope-structuredtext==5.0
+ zope-tal==5.0.1
+ zope-tales==6.0
+ zope-testbrowser==6.0
+ zope-testing==5.0.1
+ zope-traversing==5.0
+ zope-viewlet==5.0
+ zope2==4.0
Install pytest
Resolved 4 packages in 1ms
Installed 3 packages in 5ms
+ iniconfig==2.0.0
+ pluggy==1.4.0
+ pytest==8.0.2
Install Coverage
Resolved 1 package in 5ms
Installed 1 package in 4ms
+ coverage==7.4.3
Install mypy
Resolved 4 packages in 2ms
Installed 3 packages in 82ms
+ mypy==1.8.0
+ types-setuptools==69.1.0.20240223
+ typing-extensions==4.10.0
Install cookiecutter
Resolved 22 packages in 3ms
Installed 12 packages in 19ms
+ arrow==1.3.0
+ binaryornot==0.4.4
+ chardet==5.2.0
+ cookiecutter==2.6.0
+ markdown-it-py==3.0.0
+ mdurl==0.1.2
+ pygments==2.17.2
+ python-slugify==8.0.4
+ pyyaml==6.0.1
+ rich==13.7.0
+ text-unidecode==1.3
+ types-python-dateutil==2.8.19.20240106
Create Plone/Zope configuration from https://github.com/plone/cookiecutter-zope-instance to ./instance
Warning: initial user's password is insecure, it should be at least 10 characters long!
real 0m4,055s
user 0m2,162s
sys 0m2,183s
I saw it too this week, with Python Weekly linking to a blog post
venv is fast, less than a second:
$ uv venv -p python3.12 plone6010uv
$. bin/activate
uv pip cannot handle remote constraints yet:
$ time uv pip install -c https://dist.plone.org/release/6.0.10/constraints.txt Plone
**error**: failed to open file `https://dist.plone.org/release/6.0.10/constraints.txt`
So download it with curl and try again. Latest Plone 6.0.10 (today) has a beta release of one package, and you need to allow that, otherwise you get an error:
$ time uv pip install -c constraints.txt Plone
× No solution found when resolving dependencies:
╰─▶ Because there is no version of repoze-xmliter==1.0b1 and plone-protect==5.0.1 depends on repoze-xmliter==1.0b1, we can conclude that plone-protect==5.0.1 cannot be used.
And because plone-app-caching==3.1.4 depends on plone-protect==5.0.1, we can conclude that plone-app-caching==3.1.4 cannot be used.
And because plone==6.0.10 depends on plone-app-caching==3.1.4 and you require plone==6.0.10, we can conclude that the requirements are unsatisfiable.
hint: repoze-xmliter was requested with a pre-release marker (e.g., repoze-xmliter==1.0b1), but pre-releases weren't enabled (try: `--prerelease=allow`)
$ time uv pip install --prerelease=allow -c constraints.txt Plone
Resolved 235 packages in 1.07s
Built plone-app-locales==6.0.20
Built sgmllib3k==1.0.0
Built pillow==9.5.0 Downloaded 235 packages in 25.87s
Installed 235 packages in 1.01s
...
real 0m28.296s
user 0m21.820s
sys 0m15.971s
Most time seems to be spent on building Pillow on my Mac.
Afterwards, installing the same versions in the same venv is very fast:
$ uv pip install --prerelease=allow -c constraints.txt Plone
Audited 1 package in 126ms
I throw away the venv, and recreate and it is extremely fast now:
$ time uv pip install --prerelease=allow -c constraints.txt Plone
Resolved 235 packages in 66ms
Installed 235 packages in 1.00s
real 0m1.413s
user 0m0.336s
sys 0m1.384s
When I download the Plone 6.0.9 constraints in this existing venv and ask to install it, it takes 12 seconds, needing to compile and download some more packages.
Then updating to the 6.0.10 constraints again, takes 2 seconds.
So indeed really fast.
Could be interesting to start using this in tests. It could help to cache the uv cache directory then:
$ uv cache dir
/Users/maurits/Library/Caches/uv
In my case, after the commands I ran above, this cache already contains almost 800 MB.
I read somewhere that uv uses hard links. I hope this means that for example a .tox directory with several virtualenvs does not take up so much real hard disk space. In some plone repositories, I have a tox directory of 2 GB.
All in all: promising.
Does anyone know if it solves pip's problem of not being able to "overwrite" the version of a package in a constraint.txt?
yes, mxdev solves it, but uv can handle this too to a certain degree as it seems. GitHub - astral-sh/uv: An extremely fast Python package installer and resolver, written in Rust.
Interesting, because I use mxdev, which fetches it for me, I never run into this one.
I took a customer project as real world example and had some issues with sources in mxdev, so now a new version 4.0.0 of mxdev is out.
mxmake (unreleased, main branch) now generates a Makefile working perfectly fine to install Python packages with defined requirements and constraints using mxdev and so installs Plone with Git-sources.
And its really fast.
Steps to get there:
git clone https://github.com/mxstack/mxmake.git
pip install -e ./mxmake
I will release mxmake soon so the clone part is no longer needed.
then create a requirements.txt with
-c constraints.txt
Plone
and a constaints.txt with
-c https://dist.plone.org/release/6.0.9/constraints.txt
and add a cookiecutter-zope-instance instance.yaml file:
default_context:
wsgi_listen: 0.0.0.0:8080
initial_user_name: admin
initial_user_password: verysecretpasswordhere
debug_mode: on
db_storage: direct
Run mxmake init and choose "application" and "core"
further down choose domains applications.cookiecutter, applications.zope, and all core.* and do not edit any settings.
now edit the Makefile and in the header find the setting PYTHON_PACKAGE_INSTALLER?=pip and set it to
PYTHON_PACKAGE_INSTALLER?=uv
Now you are ready to to run make:
make zope-start
That is it.
If you want the venv generated by uv, it need to be available before (global).
Then set in the Makefile MXENV_UV_GLOBAL?=true.
Now its again a bit faster (spares 4-8 seconds).
This is now merged (not released though) Support remote `https://` requirements files (#1332) by jannisko · Pull Request #2081 · astral-sh/uv · GitHub