Namespace package pip installation and zope-testrunner

I have a bunch of packages under the namespace "ims" and some nested namespaces like "ims.policies". I wanted to convert all of them to PEP 420 – Implicit Namespace Packages | style packages, removing the from namespace directories and setting up a pyproject.toml. I was able to pip install everything this way but with a big problem - zope-testrunner is unable to discover any of my tests.

I also tried going back to more of a legacy style by putting a in every directory with


And my pyproject.toml has

include = ["ims"]

This leads to strange behavior. zope-testrunner is able to discover all of my test again, but one of my packages cannot be imported. Everything on a single level can be imported and I can import ims.policies.checkout but not ims.policies.portal. I have double checked that everything has the same I tried changing the include to ["ims", "ims.policies"] but that didn't matter.

Based on this section Package Discovery and Namespace Packages - setuptools 68.1.2.post20230823 documentation I'm wondering if these legacy setups just cannot be counted on to work with pyproject.toml and instead require That would suck to go back to that, since I came so close with everything else.

Alternatively, is there something I can do to tell zope-testrunner how to find tests in pep-420 style packages?

I think I found the issue in zope.testrunner This section avoids descending down directories that do not have a While my tests/ directory does have an it never gets there in pep 420 namespace packages because the namespace directories do not.

Anyone with more expertise have thoughts on removing this if block? I'm guessing the benefit of avoiding dead ends is small compared to the benefit of being able to use this with pep 420 style packages. And users can still configure tests to explicitly ignore dirs.

See also Test discovery with implicit namespace packages · Issue #11 · zopefoundation/zope.testrunner · GitHub

An idea could be to modify the zope.testrunner.find.contains_init_py(options, fnamelist) passing to it a runner.option with hints to find PEP420 namespace packages.

A similar way is used by setuptools.

setuptools uses a specialized PackageFinder: PEP420PackageFinder for PEP420 namespace packages when declared as find_namespace: ([tool.setuptools.packages.find] in toml).

The PEP420PackageFinder._looks_like_package() returns simply True for every package declared with find_namespace:. Compare it to PackageFinder._looks_like_package():

Testrunner has the option --package. See its use in Knitting in extra package directories

Interesting. I won't lie, the reasoning on this in the docs seems a bit arcane to me but I was able to run

python -m zope.testrunner --test-path=sources\ims.policies.portal --package .\sources\ims.policies.portal\ims\policies\portal\tests\

without any changes to zope.testrunner.