Running Plone Robot Tests with Github Actions?

Development Testing and Automation
1

Dear Lazy Web, has anyone here Plone robot tests running on Github actions or know an example setup? Thx, a lazy developer.

1 Like
2

I think this runs robot tests: collective.collectionfilter/main.yml at master · collective/collective.collectionfilter · GitHub

2 Likes
3

Also this one does that collective.workspace/tests.yml at f7565a64d8b8f74a420adb515728c17a368b7abe · collective/collective.workspace · GitHub

1 Like
4

I followed the example of collective.collectionfilter to run them for Add support for Python 3 in Plone 5.2 and 6 by pbauer · Pull Request #12 · collective/Products.PasswordStrength · GitHub but cannot get it to work. All tests fail with

ElementClickInterceptedException: Message: element click intercepted: Element is not clickable at point (162, 927)
/home/runner/buildout-cache/eggs/robotframework-3.1.2-py3.9.egg/robot/running/builder.py:61: DeprecationWarning: Option 'warn_on_skipped' is deprecated and has no effect.
  (Session info: chrome=100.0.4896.60)

See more · collective/Products.PasswordStrength@b295764 · GitHub

What am I doing wrong?

5

Maybe the element is covered by another one?
Did you try changing the viewport of the tests?
Maybe Xvfb -ac :99 -screen 0 1280x1024x24 should be something bigger.
Also I would consider validating the visibility of the element before clicking:

Wait Until Element Is visible  ...

Hard to say more without diving deep in to the PR.

6

We have a working robottest setup now in collectionfilter. See Refactoring Project, use pip install method by 1letter · Pull Request #174 · collective/collective.collectionfilter · GitHub

I took the idea from here GitHub - nanasess/setup-chromedriver: ChromeDriver for use in GitHub Actions and added it here Refactoring Project, use pip install method by 1letter · Pull Request #174 · collective/collective.collectionfilter · GitHub

The test-layer-ignore-warnings in the Makefile is specific to collectionfilter to test the different layers separately Refactoring Project, use pip install method by 1letter · Pull Request #174 · collective/collective.collectionfilter · GitHub

(now we can start fixin the tests :face_with_peeking_eye:)

Friday 13th UPDATE: All robottests are fixed now in collective.collectionfilter ... for convenience I'll paste the GHA config here:

name: CI

on:
  push:
    branches-ignore:
      - "releases/**"
      - "master"
jobs:
  build:
    runs-on: ubuntu-20.04
    strategy:
      fail-fast: false
      matrix:
        python:
          - "3.8"
          - "3.9"
          - "3.10"
          - "3.11"
        plone:
          - "6.0-latest"
        # see testing.py for layer names
        test_layer:
          - "IntegrationTesting"
          - "AcceptanceTestingPortlet_AjaxDisabled"
          - "AcceptanceTestingPortlet_AjaxEnabled"
          - "AcceptanceTesting_Tiles"
    steps:
      - uses: actions/checkout@v3

      - name: Setup Plone ${{ matrix.plone }} with Python ${{ matrix.python }}
        id: setup
        uses: plone/setup-plone@v1.0.0
        with:
          python-version: ${{ matrix.python }}
          plone-version: ${{ matrix.plone }}

      - name: Install package
        run: |
          make VENV=off install

      - name: Run Lint
        run: |
          make VENV=off lint

      - name: Install Chromedriver
        uses: nanasess/setup-chromedriver@v1

      - name: Start Browser
        run: |
          export DISPLAY=:99
          chromedriver --url-base=/wd/hub &
          sleep 5  # make sure chromedriver is running
          sudo Xvfb -ac :99 -screen 0 1920x1280x24 > /dev/null 2>&1 &

      - name: Run tests
        run: |
          VENV=off TEST_LAYER=${{ matrix.test_layer}} ROBOT_BROWSER=headlesschrome make test-layer-ignore-warnings

      - name: Save Robotframework Log
        uses: actions/upload-artifact@v3
        # safe artifacts only for failing tests
        if: ${{ failure() }}
        with:
          name: robottest-logfiles-${{ matrix.test_layer }}-${{ matrix.plone }}-${{ matrix.python }}
          path: |
            test_*
            robot_*
1 Like
7

I'd recommend use ROBOT_BROWSER=headlesschrome ... this solved every "Element is not clickable..." error for me.

1 Like
8

Last year we had a lot of problems with the same group of failing tests in coredev that had to do with the viewport and/or elements that were not reachable/selectable. Especially in the updated Control Panels the save/confirm button often failed.

One of the things we discovered IIRC was to add a focus to the element before you test it further. Then the viewport is less limiting.

We first added these in the tests themselves but a few new library statements were created and @mauritsvanrees moved them in this PR to plone.app.robotframework: