I would like to use Plone 6 with the Volto frontend for a personal website, and have got the nginx, Frontend, Backend container example working behind TLS/SSL as I describe here. I’m having more difficulty than I expected changing that recipe in order to mix in several community-developed addons. Specifying particular addons using an ADDONS environment variable doesn’t seem to work.
I realize the alternative is to Install Plone from its packages, and have explored that enough to try several addons. I am experiencing two problems with this approach.
First, the solutions I found for getting the containers-based site working behind TLS/SSL are not working, and it’s a whole new system to learn to solve them anew.
More daunting than that, I want to continue to use docker compose to build and deploy my system and I haven’t got a start on what it would take to containerize the packages-based build.
The combination of these two lead me to hope I just missed a way to include Volto addons in the container build.
So here are the questions with which I’d appreciate help:
If not, is there a simple way to change the plone/plone-frontend machinery to enable easy addition of Volto addons? (I see ”--skip-addons in the project Dockerfile. Would omitting that get the ADDONS environment variable to have the effect I want? If so, is there a reason to not remove it? If not, I will be happy to submit a PR. I’m going to experiment with that while waiting for guidance.)
I also realized that wiring the addons into the image makes a lot more sense than mixing them in every time I start the site. It provides assurance against occasions when access to specific addons or specific versions of an addon is unavailable.
So I guess using those --addon clauses in the GitHub - plone/plone-frontend: Plone frontend Docker images using Node 16.Dockerfile is a good way to go. Once I have the image I can incorporate it in the docker compose scheme for running the site behind https/TLS, which I describe here. I welcome suggestions about ways to improve this arrangement, but think I’m ready to actually build and start populating my site!
I have successfully built a Docker image with various addons mixed in. (Here's a version of the Dockerfile that does that.) But I haven't figured out how to configure the image build so the resulting image looks for the ADDONS environment variable, or takes an "--addon" argument, or anything like that. I thought that the plone/plone-frontend was supposed to do that, but haven't gotten it to work.
Being able to mix the addons into my Docker image is adequate for my purposes. It would be handy to be able to specify more addons when invoking the container in order to expedite trying new ones, so I'm quite interested in how to solve that as well. But I'm happy I can move forward with building my site, including whatever addons I need, and love that it's all packaged as containers and container images, so I can deploy it all using Docker.
Now you probably do not need this any longer.
For the records, there is Cookiecutter Plone Starter which helps getting your environment bootstrapped - including a custom frontend container build.
As far i have understood the example docker-compose files provides a running instance but without any possibiity to use addons. To be able to use addons it's necessary to create an own docker image, the recommended tool to create the project is Cookiecutter, which provides also new generated docker images backend and frontend.
Did i miss something or is there a way to reuse already existing docker images and extend them with addons?
I believe that's right. At least, I have not found a way to make an existing docker image use addons of your choice. I use cookiecutter to make a build environment which I configure with the addons I choose to include and then build a new frontend docker image.
I still would like to find a way to make an existing Volto frontend docker image include arbitrary frontend addons (the frontend build configuration and build process is elaborate, even with cookiecutter), but I suppose that Volto is not organized for that. If so, maybe the documentation could be made more clear about that fact?
It's possible to do it, but we have some conceptual problems:
We start with "Volto team publishes a volto docker image"
docker images are supposed to be pretty much "immutable", to be a "ready to use" delivered solution. When you docker run an image, you expect that it will be up and running as soon as it has finished downloading
but a Volto addon is Javascript package. It needs to be installed and the whole bundle (the static js / css files) need to be rebuilt to include that code
the Javascript ecosystem relies on yarn.lock files to describe the node_modules layout and be able to rebuild it consistently. Lately this hasn't been such a big issue, but in the past ensuring a consistent node_modules was a challenge. This is due to the fact that, unlike Python, in javascript it's possible to have multiple versions of the same library and there's a "hoisting" process where the libraries are all lifted in the top-level node_modules (if you browse the inner folders of your node_modules, you'll find other node_modules folders inside them, that's how JS solves the multiple versions problem)
Each time you add a JS package, you regenerate the yarn.lock file so in effect, you'll be mutating something that's supposed to be shipped with the docker image.
I would assume to get this solved by using a combination of component shadowing and persistence storage with volumes. I tried to use src/customizations to override components but i struggled as well. Maybe it's a similar root cause. But your answer helps me to focus a bit more on building own images instead trying to use existing ones (for the moment).
It sounds like not having a docker image change through operation is part of reliability and security assurances. In that sense it may not be appropriate to have such fundamental functionality changes that addons can entail be available just through some settings that you supply when running an image – maybe it makes sense for inclusion of different addons to necessarily result in a different docker image.
If so, it would still be quite useful to be able to specify a Volto frontend docker image and some addons to include (and others to remove) and get a new Volto frontend docker image. And maybe aiming to produce a new image would alleviate some of the fundamental challenges?
While cookiecutter helps, I feel that the complexity of concocting a frontend docker image is still limiting Plone + Volto uptake.
You can't override with docker volumes some shadowed file (src/customizations) because that file is only used when developing.
Let's get the basics straight:
you start Volto (in development mode) with yarn start. This compiles on the fly the files, and it uses 2 ports: 3000, 3001. The second port is used to auto-reload the code changes that you do while developing.
When you deploy, even if you don't use docker, you need to do yarn build which creates static js files and then you can copy those generated files + the node_modules folder and then you can run node build/server.js and you'll have Volto http server running. It's running standalone, as a nodejs app. It does not have any connection with the src/ folder, none whatsoever. Everything is included in those static files that the Volto http server includes in its generated HTML, and they get loaded in the browsers of the visitors.
So, to somehow shadow a file with a docker image, you need to rebuild the whole static resources bundle, which is an operation that you do typically when you initially build the docker image, not "on demand".
That doesn't mean that it's not possible to create a Dockerfile that would automatically rebuild the bundle if it detects something. But really, it's not that difficult to create your own docker image. Just grab a Dockerfile, then run docker build . -t myname/myproject, then you can upload that docker image to dockerhub: docker login and docker push myname/myproject (be aware that the docker image will be public, unless you pay an anual subscription of 60USD).
I know this is a sidetrack, but there's a small, somewhat obscure refinement to something you mentioned that I have found extremely valuable:
You can preface the yarn start with a port designation in order to start yarn on that port (with the second port incremented accordingly). So, eg:
PORT=4000 yarn start
Will run the development instance on 4000 and 4001.
The beauty of this is you can run it at the same time as a standard instance is running on 3000 and 3001. Updates to the src will show in browser visits to port 4000 while they do not show in visits to 3000.
Often my changes are in an addon I'm working on. Once I've got them settled I increment the addon's version, npm publish the module, then in the frontend implementation package I yarn add the new version of the module and do a new make build of the frontend image. And, as you suggest, I can docker push the new docker image or test first using the image on my host. I use docker compose to run my system, so I might adjust my docker-compose.yml to run the changed complement of images, whether it's the one I pushed to docker hub or the new one I have on my host.
That you can specify the port may be obvious to some, but I didn't know about it until I got a hint that it may be possible, and then it was not easy to discover how.
I neglected to mention that the frontend instance running on port 4000 (or whatever) will connect to the same backend as the one running on 3000, without any conflict. You have full access to your site instance through the changed frontend code.
I'm looking at the plone-frontend and the custom image examples.
Wondering why Dockerfile.prod seems to build from node rather than inheriting from from plone/plone-frontend
# syntax=docker/dockerfile:1
FROM node:16-bullseye-slim
LABEL maintainer="Plone Community <dev@plone.org>" \
org.label-schema.name="frontend-prod-config" \
org.label-schema.description="Slim image for Plone frontend deployments" \
org.label-schema.vendor="Plone Foundation"
# Install busybox and wget
RUN <<EOT
set -e
apt update
apt install -y --no-install-recommends busybox wget
busybox --install -s
rm -rf /var/lib/apt/lists/*
mkdir /app
chown -R node:node /app
EOT
# Run the image with user node
USER node
# Set working directory to /app
WORKDIR /app
# Expose default Express port
EXPOSE 3000
# Set healthcheck to port 3000
HEALTHCHECK --interval=10s --timeout=5s --start-period=30s CMD [ -n "$LISTEN_PORT" ] || LISTEN_PORT=3000 ; wget -q http://127.0.0.1:"$LISTEN_PORT" -O - || exit 1
# Entrypoint would be yarn
ENTRYPOINT [ "yarn" ]
# And the image will run in production mode
CMD ["start:prod"]
Thanks @davisagli digging deeper it is starting to make sense.
My goal is to build a custom frontend in CI/CD which I can use in deployment.
Without the luxury of time (other projects), I've had to delay until I have time to plug away at the task again.
I suppose, I should read how it is done for plone-frontend and modify it for my use case.
