Best practices for CI in Gitlab

Plone Support Add-on Development
, ,
1

Hi there,

I want to create a Dockerfile for Gitlab's CI/CD pipeline. For example I have a repository with a simple Plone addon and some test cases which I want to run on every commit. May do you have an example configuration for me which installs Plone 5.2.1, installs my addon and runs my test cases on it?

This is the first time I am doing this. I know how Dockerfiles work but I never did any CI/CD with it. There are example templates for all kinds of programming languages but there is none for Plone addon testing. Does it make sens to use a build from Docker Hub as a base image? Or am I on the wrong path here?

Can you give me some hints or can you point me to a resource which can help me with that, specifically for Plone?

Thanks.

2

I don't have a best practice, because we are running the testing of our plone addons in the project repositories. Our Gitlab config triggers the project builds on every change in the addon repositories. Maybe this could be a workaround to set up a plone repository for testing your addons.

3

So you have an already installed Plone running on a different machine and then you trigger a buildout -c develop.cfg or something similar with every commit which in turn tests the new code on that machine?
Or what do you mean with a project repository?

4

plonecli/ bobtemplates.plone generates a .gitlabci.yml like so

https://github.com/ploneF/bobtemplates.plone/blob/master/bobtemplates/plone/addon/.gitlab-ci.yml

with some basic setup.

5

I know this template. But it always runs into errors. I guess the first reason is that is uses Python 2.7 and not Python 3.8. But maybe I can try to fix that.

6

I created a Docker image for running CI tests in gitlab using this guide http://www.cosent.nl/en/blog/plone-gitlab-continuous-integration that was invaluable. For my purposes I didn't bother to use a pre-defined Plone-specific image, I just used centos7 as a base because of our particular environments and installed Plone and dependencies on it, and from that created a buildout using all of our eggs. All of my projects on my company's internal gitlab use this with a .gitlab-ci.yml file to run the tests and a simple buildout config gitlab-ci.cfg. The Docker container itself already has a fully built buildout cache from creating the image, so when you are running tests automatically after a commit it will very quickly create a new buildout using the cached eggs location.

The cfg file is very minimal. The extends location it refers to is defined in the image

[buildout]
extends = /plone5/instance/buildout.cfg
develop = .

[versions]
# this tells buildout to _not_ use a released version of this package
ims.webdev =

The yml file is also fairly simple. This is the relevant snippet

  - echo "Running buildout"
  - buildout -c gitlab-ci.cfg
  - echo "buildout successful"
  - echo "running test on version:"
  - python setup.py --version
  - /builds/$CI_PROJECT_PATH/bin/test -v -s ims.webdev
1 Like
7

You can try https://hub.docker.com/r/itspirit4docker/plone-testing-base (repository is at https://gitlab.com/it-spirit/plone-testing-base). It is based on the bobtemplates.plone configuration, but has a lot of dependencies pre-installed (so the docker image is bigger, but that is ok for testing). The default uses Python 3.7, but it supports 2.7 as well. It also has geckodriver installed for robot tests (which always resulted in errors for me when installing on the fly).

1 Like
8

Wow. That script is gold. It just works.
But there is a small inconvenience.

The command "bin/code-analysis" exited with 1 in 0.396s.
ERROR: Job failed: exit code 1

Is this normal if there are any small issues regarding the coding conventions? Can I just ignore some of these warnings? These are some of the warnings I want to ignore:

E302 expected 2 blank lines, found 1
W293 blank line contains whitespace
E251 unexpected spaces around keyword / parameter equals
E203 whitespace before ':'

Sorry for that dumb question but for now this topic is relatively new to me and I still have a lot to learn.

9

Glad it works for you. You could check thoses errors before you commit/push (recommended), or disable the return-status-codes in your .gitlabci.yml:

  - buildout -n -c buildout.cfg download-cache=downloads code-analysis:return-status-codes=False
10

I'm sorry I bumped up old threads .
But I thought this is better, as create a new one - for nearly the same....

Can anyone share a working buildout-setting for a gitlab-CI Pipeline with Gitlab's Package Registry - or give me a hint how I get it done?

I tried it for 2days now, but i can't get it...
Publishing within a CI-Pipeline works, but pulling/download within buildout.cfg won't...
If I open my "find-links"-URL, i get a beautiful working page with all my published addons - the only thing I noticed, the dot in "my.addon" in the package a-href is now a hypen, but the title is correct <a href="..../my-addon">my.addon</a>

Package downloading with pip install my.addon --index-url https://gitlab.example.at/api/v4/groups/81/-/packages/pypi/simple --no-deps is working!

Buildout Traceback

We have no distributions for my.addon that satisfies 'my.addon'.
Couldn't find index page for 'my.addon' (maybe misspelled?)
Getting distribution for 'my.addon'.
Couldn't find index page for 'my.addon' (maybe misspelled?)
While:
  Installing instance1.
  Getting distribution for 'my.addon'.
Error: Couldn't find a distribution for 'my.addon'.

This are my files:

buildout.cfg

[buildout]
newest = false
allow-hosts = *
allow-picked-versions = true
prefer-final = false
find-links = https://gitlab.example.at/api/v4/groups/81/-/packages/pypi/simple

[instance1]
eggs +=
    ${buildout:eggs}
    my.addon

.pypirc

[distutils]
index-servers =
    gitlab
    gitlab-pypi

[gitlab]
## for CI Pipeline
repository = https://gitlab.example.at/api/v4/projects/${env.CI_PROJECT_ID}/packages/pypi
username = gitlab-ci-token
password = ${env.CI_JOB_TOKEN}

[gitlab-pypi]
## for buildout, pip,....
repository = https://gitlab.example.at/api/v4/groups/81/-/packages/pypi/simple
username = __token__
password = glpat-1233456

pip.conf

[global]
extra-index-url = https://gitlab.example.at/api/v4/groups/81/-/packages/pypi/simple

.pydistutils.cfg

[easy_install]
# repository = https://gitlab.example.at
index_url = https://gitlab.example.at/api/v4/groups/81/-/packages/pypi/simple
find_links = https://gitlab.example.at/api/v4/groups/81/-/packages/pypi/simple

And a thousand thanks for your help :slight_smile:

Versions:

  • setuptools 44.1.1
  • wheel 0.37.1
  • pip 20.3.4
  • zc.buildout 3.0.1