GSoC 2018 Ideas: Plone CLI docker support

This is one post in a series to begin focused discussion about ideas that came out of our 2018 GSoC Brainstorm.

Please use this post as a place to begin to discuss this idea more in depth.

This idea was proposed by @pigeonflight, and he is willing to serve as a mentor and thinks @MrTango would also be a great mentor

The description:

"The PloneCLI can be extended to streamline development using docker based containers"

Go for it, community! Make this one shine!

1 Like

@pigeonflight can you please describe more what you have in mind here?

Sorry for the delay in responding. Still putting my ideas together.
We recently deployed two customer plone sites using docker. We had to do a lot of reading and figuring out.
The idea is to "bundle" the best practices and knowledge into a plonecli command.
It would be based on plone.docker and most likely make use of docker compose.

A developer would be able to use the plonecli to bootstrap and manage docker based development and deployment.

Imagine being able to enter a buildout folder and type:

plonecli dockerize

the resulting output might look like this:

Your buildout has been dockerized.
It is now possible to launch your Plone site on any docker ready
server using the following command:

   docker-compose up -d

You can also deploy your Plone site to platforms that support docker, such as:
 "heroku, platform.sh and elastic beanstalk"

In case there are copyright issues we could use a command like plonecli containerize.

This will require a lot more ground work before it is ready for plonecli. This work will have to at least include the following:

  • A docker ready buildout template file (maybe called it buildout-docker.cfg or docker.cfg). This is needed because the standard buildout.cfg has too many "extras" that don't play well with plone.docker.
  • a useful docker-compose.yml file
  • sensible defaults for integrating with Continuous Delivery (gitlab is an easy target for this).

Stuff I'm looking at as examples or references:





https://docs.plone.org/manage/docker/docs/scaling/docker_compose.html

We more or less have that already in collective.hostout docker branch.

bin/hostout app dockerfile

will produce a working dockerfile. I guess it could produce a sample compose.

But I think internally we are only using a small part of this now. What I'd like to do is strip hostout back to just a pip installable command that takes a buildout, a dumped versions and can package up that environment into a tar useful for docker. However with a good set of .dockerignore files you might not even need that.

I would advise against making too many magic cli that hide too much stuff. As soon as your want something somewhat different you are then back to learning docker. You have to get the right balance. Perhaps just having some well documented examples to cut and paste is enough?

@pigeonflight I like the idea and can help from the cli or mrbob side.
@djay as the cli just uses bobtemplates and fires some commands, I think it should be fine.
We should print every command we run and the user can see the files in the directory too.
So it's just a way to save copy&paste and some commands.

The order of task to get this done is probably the the following:

  1. Create the buildout config
  2. Create the docker files
  3. Put everything into a bobtemplate maybe directly in bobtemplate.plone and register it for the plonecli
  4. Extend plonecli with needed docker commands

But if you want to dockerize an existing buildout, there is more to do. Because, you want customize the existing files and use them. Maybe you can write down what steps are needed here.

We could also build this as a plugin for plonecli, instead of implementing it direcly in it.

Put a Plone installation into a Docker container is

  • error-prone
  • time-consuming
  • specific for each project (unless you deal with bare Plone buildouts only)

I see very little practical use for such a project in order to have a general benefit for most of us.

-aj

@zopyx,
There was a bit of a (error-prone, time-consuming) learning curve but I've made good progress with Docker and Plone. The trickiest part is using Docker for Plone development, both eea.docker.plone and plone.docker have approaches to this. I haven't tried eea.docker.plone (their approach is documented here: https://github.com/eea/eea.docker.plone#developing-plone-add-ons),
but I've found the plone.docker approach (documented here: https://docs.plone.org/manage/docker/docs/usage/addons.html#add-ons ) a bit too "fiddly" for me.

The benefit that I see for this project would be in reducing the error and time to being productive for new developers.
If we can pull this off, the biggest win is for the consumer of the containerized installation.
For the primary benefit to be realized it should be as close to trivially easy for more experienced developers to containerize a working environment.

These discussions are important because they help in clarifying the "why" behind a project. Hopefully this brings more clarity about the purpose of the project.

2 Likes

Ok @pigeonflight if you are being the mentor, I'll support you with the plonecli/plugin part and will be also available to test the things out. Just that I got it right, we are doing this to creat docker container which will be dillivert than elsewhere, right? Because plonecli is not a deployment tool :wink:

Not clear on the question.
But I'm clear that the vision for plonecli is not deployment.

It sounds to me like this one is approaching something like a well-defined state. @pigeonflight would you be able to work on a draft of a full-on GSoC proposal in the spirit of those at https://plone.org/community/gsoc/ideas/ ?

Sure. I guess I could place it in a shared Google Doc for now.

no the c.hostout way is completely automated.

Just putting a few questions out there before I push this GSoC proposal forward.
Is this duplicating effort? Should we focus on making hostout more visible in the community?
Or is it reasonable to want to offer a way to distribute development environments with docker and plonecli?

Additionally the scope of this may be too small for a full GSoC project. Spoke to @MrTango off the air about this and it looks like bundling it in with the rest of the plonecli work may make the most sense.

@pigeonflight, did you manage to get a google doc written up with a proposal? I'm trying to add ideas to our published ideas list and can't find any work you might have done on this. Could you leave a link here?

Just wrote it up:


I'm still wondering if it is enough for a full GSoC project.

I think combining it with the bobtemplates work Mail proposed will make it solid. I’ll merge the two. Will that work for you?

c

That's fine with me.

@pigeonflight @MrTango What's your opinion on, should the GSOC focus on building on top of current plone.docker -project or should be try pip based docker builds already?

The official buildout based docker project is pretty stable and complete, but different from any other python based project.

Pip based build still needs some tweaks (custom version of z3c.autoinclude and replacement for plone.recipe.zope2instance), but we do have a sprint (Sauna Sprint) scheduledon July to work on those issues. The main benefit of pip based docker is that it is trivial to extend (by just installing new packages with pip).

Existing pip based Plone docker examples:

1 Like