Test coverage of robot tests

Hello guys, I have a BrowserView full of methods, which I am covering almost at a 100% with Robot tests, however the coverage reports 22% coverage...

I have looked around, and cannot find anywhere, any information about there being a difference in coverage report, whether the code was executed by robot tests or unit tests...

This is a Plone 4.3.11 site, with the following versions:

  • zope.testing-3.9.7
  • plone.testing-4.1.1
  • plone.app.testing-4.2.5
  • zope.testrunner-4.4.4
  • unittest2-0.5.1
  • plone.app.robotframework-1.0
  • robotframework_selenium2library-1.7.4
  • robotsuite-1.7.0
  • robotframework-3.0

And this is how my buildout parts look like

[test]
recipe = zc.recipe.testrunner
defaults = ['-s', 'custom.package', '--auto-color', '--auto-progress']
eggs =
    Plone
    Pillow
    plone.app.upgrade
    custom.package [test]

[coverage]
recipe = zc.recipe.egg
eggs = coverage

[test-coverage]
recipe = collective.recipe.template
input = inline:
    #!/bin/bash
    ${buildout:directory}/bin/coverage run --source=${buildout:directory}/src/custom/package bin/test --all
    ${buildout:directory}/bin/coverage html
    ${buildout:directory}/bin/coverage report -m # --fail-under=50
    # Fail (exit status 1) if coverage returns exit status 2 (this happens
    # when test coverage is below 100%.
output = ${buildout:directory}/bin/test-coverage
mode = 755

After running ./bin/test-coverage, I get all tests executed, both unit and robot, but it would seem I only get coverage report for unit tests...

Any suggestion?

Not sure why your way is not working but I made the change here https://github.com/collective/collective.collectionfilter/blob/master/.travis.yml to ensure robot tests were included in coverage and it worked

Got annoyed by github giving me a red cross because coverage was going down when adding new features.

BTW, that popt program I put in that .travis file is super useful to give you terminal output from your robot tests when they fail on travis but not locally. I'm not sure why travis doesn't have the test artifacts feature CircleCI does that makes it easy to save the robot test output.