Strange catalog behavior on Plone 5.1

[hvelarde]

I'm working on a PR to fix an issue on an add-on and I'm facing a weird behavior in the catalog of latest Plone 5.1 branch:

Fix Canonical URL updater form

part of the fix involves reindexing the catalog in order to reflect a new behavior that was just added on a previous version; the code to test this upgrade step looks like this:

from sc.social.like.behaviors import ISocialMedia
from sc.social.like.tests.utils import enable_social_media_behavior
with api.env.adopt_roles(['Manager']):
    for i in xrange(0, 10):
        api.content.create(self.portal, 'News Item', str(i))

# break the catalog by deleting an object without notifying
self.portal._delObject('0', suppress_events=True)
self.assertNotIn('0', self.portal)
enable_social_media_behavior()
results = api.content.find(object_provides=ISocialMedia.__identifier__)
self.assertEqual(len(results), 0)

as you can see I'm creating 10 instances of a content type, after that I'm applying a behavior to it; this change must not be reflected in the catalog because enabling the behavior should not fire an object reindex at all.

the assertion pass normally in Plone 5.0 (ZODB 3.10.7) and fails in Plone 5.1 (ZODB 5.2.4).

any hints?

BTW, this is the code for enable_social_media_behavior():

def enable_social_media_behavior():
    fti = getUtility(IDexterityFTI, name='News Item')
    behaviors = list(fti.behaviors)
    behaviors.append(ISocialMedia.__identifier__)
    fti.behaviors = tuple(behaviors)
    # invalidate schema cache
    notify(SchemaInvalidatedEvent('News Item'))

[hvelarde]

Hanno Schlichting pointed out in the ZODB list this could be instead related with this PLIP:

PLIP: assimilate collective.indexing

here is his reply:

On 07/21/2017 12:34 PM, Hanno Schlichting wrote:
It's pretty unlikely changes in the ZODB would result in a single test
failure in a Plone catalog.

I'd say the more likely culprit is the inclusion of collective.indexing.
I think its code got merged into the CMF/Plone catalog tools for Plone
5.1.

Maybe you could try to get a hold of the index queue and peek into it,
and see what that gives you
(Products.CMFCore/Products/CMFCore/indexing.py at 2.3 · zopefoundation/Products.CMFCore · GitHub).

@gforcada @MrTango do you mind to take a look at this issue?

[gforcada]

You could try to use the environment variable described here:

plone/documentation/blob/5.1/develop/plone/searching_and_indexing/external-catalogs.rst

i.e.

CATALOG_OPTIMIZATION_DISABLED=y ./bin/test

Or, you could manually process the queue:

from Products.CMFCore.indexing import processQueue
...
do some operations that will trigger catalog indexing operations
...
processQueue()
check that the operations took place

I hope this helps

[hvelarde]

thanks! that made the trick and tests are now passing.