Automatic Test Content for development and CI

Looking for architecture help plz:

As a developer,
I want to install test content into a clean buildout
So I don't have to create it manually TTW

As a tester
I want to save my test content and maintain it
so I can replicate tests easily

As Jenkins
I want to do a ci build with test content
and sick selenium on it
so CSS errors can break the build.


The way we handle this in our current system is to make "test content" the "backup from X days ago". Basically restoring production data onto the test or development server.

Drawbacks:

  1. creating new test content for very specific issues can't be saved - it gets thrown away when a more recent backup is restored.
    eg: a story with an image floating left and a video floating right renders wrong in facebook instant

  2. the test content is big to export/import. Copying it to a local instance is grounds for a vacation day, and starting / stopping the instance can be very annoying.


The part that I'm struggling with is the format and method for export/import.
-- Transmogrifier and jsonmigrator would be perfect, but it's really overly complicated for this kind of thing.
-- genericsetup.content doesn't seem to save blobs
-- zexp exports don't seem to like my dexterity types.

Is anybody doing this kind of test-content imports for testing builds?
Any opinions on the best tools to create this process with?

For repeatedly creating content:
Add an additional profile to your packages that will create the content for you.
See https://docs.plone.org/develop/addons/components/genericsetup.html for more on that.

For automated tests:
Create the content in a testing layer or the setUp of your testcase , and use either python (unit tests/integration test) or robot tests.
https://docs.plone.org/develop/testing/index.html should contains some pointers on how to run all this.
Some examples can be found here: https://docs.plone.org/external/plone.app.dexterity/docs/testing/integration-tests.html

I think the mr.bob templating stuff also creates the boilerplate code for this. You can take a look there as well.

I already have a QA profile with an import step. The implementation of that step is what I’m asking about.

I would like to use GenericSetup.content, but it appears to not save blobs. Please correct me if I’m wrong.

This is a functional or regression test. Maybe if I said ‘I want to test my templates and CSS’ it would be more clear.

This is not something that bin/test is designed for.

You do bring up a good point with the GS profile tho. I’m thinking of pros/cons between a GS profile or a separate package that my QA buildout installs. And it just occurred to me that recipe.plonesite might also be useful here.... not sure

There is an option somewhere to use an existing ZODB in test runs. See Is it possible to run robot tests using a Data.fs from production instead of creating it on the fly?

1 Like

This is actually a really good idea. I think it also sets us up well for a process to create additional content as needed. Once new test content is created TTW there is no extra step to export and version it.