Some objects not making it into the UUID index?

flipmcf · August 19, 2020, 10:19pm

We had a missing image in one of our stories and I just spent a few hours dealing with a UUID I just couldn't find in the index, no matter how hard I tried:

>>> story = app.RFA.english.news.china['indonesia-death-07092020180018.html']
>>> image = story['indonesia-death.jpg']
>>> from plone.uuid.interfaces import IUUID
>>> uuid = IUUID(image, None)
>>> uuid
'8a4e1f1a-00f4-4913-927e-7b8fe894e3a1'

ok.... then....

>>> from plone.app.uuid.utils import uuidToObject
>>> obj = uuidToObject(uuid)
>>> obj

huh?

>>> from Products.CMFCore.utils import getToolByName
>>> catalog = getToolByName(app.RFA, 'portal_catalog')
>>> catalog.unrestrictedSearchResults(UID=uuid)
[]

Wha..?

I cleared and reindexed the UUID index and this specific case was resolved, but I'm noticing that my Path Index has 83973 values, but the UUID index has 80058 values. That seems a bit suspicious to me. I expect I'll be revisiting a bug where another UUID lookup is failing.

What am I doing wrong?
Should I be suspicious?
Anyone else seeing something like this?
Where can I debug deeper? Like raise a pdb if the indexer fails to put an object's UID into the UID index? I want to investigate this further.

dieter · August 20, 2020, 4:30am

You need to be careful with the interpretation of the term "index value": the number of index values is typically not the number of indexed documents. The "values" for a PathIndex are for example not paths but id components in those paths.

jaroel · August 20, 2020, 6:37am

Be sure that plone.app.referenceablebehavior is enabled

mauritsvanrees · March 12, 2021, 12:12pm

Old topic, but I just bumped into a similar problem reported by a customer this week.
Plone 5.2, Python 3. I could reproduce the problem locally like this (slightly adapted, but this should do the trick):

Create Folder A and within it Page A.
Create Folder B.
Make a zexp of Page A.
Import the zexp in Folder B.

Result: both copies of Page A have the same UUID. The UID index can only handle unique values, so it does not return anything when querying for this UUID.
On the live site, resolveuid/uid-of-page-a gave a 404. Locally it picked the original. So my reproduction is not exact. But probably close enough.

Solution: remove the _plone.uuid attribute from the copy and store a new uuid on it.

I have created a script to find such problems and fix them, including a clear and rebuild of the UID index:

github.com

zestsoftware/plonescripts/blob/master/fix_uid_index.py

# Rebuild the UID index by clearing and reindexing.
# And try to fix duplicate UIDs.
# This can at least happen when you import a zexp twice, in different folders.
#
# For updates and more such scripts, see https://github.com/zestsoftware/plonescripts
#
# Run this with:
# bin/instance run scripts/fix_uid_index.py
#
# Note: this script only works on Python 3!
# But this is only because of f-strings, so should be easily fixable.
# Tested on Plone 5.2.

import sys
import transaction
from plone.uuid.handlers import addAttributeUUID
from plone.uuid.interfaces import ATTRIBUTE_NAME
from zope.component import getUtility
from zope.component.hooks import setSite
from zope.intid.interfaces import IIntIds

This file has been truncated. show original

Change the site id. Try it out on a copy of the database first please...
And it is tested in Plone 5.2 Python 3 only. Should work on older versions as well, but the script prints a few lines using f-strings, which are Python 3 only. Should be easily fixable.