I am working on a synchronization script for taking the data from the API and create an object.
My script was able to create the objects successfully in a newly created instance, but when I tried it in the live instance (which already has many more imported objects), it gave error. The number of the objects that I want to import is about 6000 and there are multiple images for each of them.
The current error is about creating the images files but it also raising the same errors for creating the artwork objects (custom content type). It raises the following error when I run the script:
Module Products.PluginIndexes.unindex, line 213, in insertForwardIndexEntry
TypeError: '<' not supported between instances of 'int' and 'str'
The error seems to be related to the indexing of the database. Even though I deleted the previously created objects (the same custom type), there seems to be a conflict which causes it to give errors.
When i try to clean and rebuild the catalog using the Plone UI, it causes the Plone to restart itself. When I try to reindex, it also causes it to crush.
I tried re-cataloging with the debug mode: but it keeps giving the same warning for all of the objects:
WARNING:plone.app.contenttypes. indexers:Lookup of PrimaryField failed for Plone/nl/collectie-onderzoek/collectie/kunstwerken/nathan-phillips-square-a-winters-night-skating/456c0b48d32bfd7c01ade92b46b54a9c558efe7b-jpg If renaming or importing please reindex!
Has anyone seen this error before? If it is indeed about the indexing: Is there a better way to fix the indexing of the database other than using Plone UI? Here is the script I use for importing the images:
def import_images(container, images):
MAX_RETRIES = 3
DELAY_SECONDS = 5
# Delete the existing images inside the container
for obj in api.content.find(context=container, portal_type='Image'):
api.content.delete(obj=obj.getObject())
for image in images:
primaryDisplay = image.get('PrimaryDisplay')
retries = 0
success = False
# Tries MAX_RETRIES times and then raise exception
while retries < MAX_RETRIES:
try:
with requests.get(
url=f"{IMAGE_BASE_URL}/{image.text}", stream=True, verify=False, headers=HEADERS
) as req: # noqa
req.raise_for_status()
data = req.raw.read()
if "DOCTYP" in str(data[:10]):
continue
imagefield = NamedBlobImage(
# TODO: are all images jpegs?
data=data,
contentType="image/jpeg",
filename=image.text,
)
image = api.content.create(
type="Image",
title=image.text,
image=imagefield,
container=container,
)
if primaryDisplay == '1':
ordering = IExplicitOrdering(container)
ordering.moveObjectsToTop([image.getId()])
success = True
break
except RequestException as e:
retries += 1
if retries < MAX_RETRIES:
time.sleep(DELAY_SECONDS)
else:
print(f"Failed to fetch image {image.text} after {MAX_RETRIES} attempts: {e}")
if not success:
print(f"Skipped image {image.text} due to repeated fetch failures.")
return f"Images {images} created successfully"
Here is the full version of the errors:
Versions:
- Plone 6.0.2 (6013)
- Volto 16.21.0
- CMF 2.7.0
- Zope 5.8
- Python 3.11.5 (main, Aug 24 2023, 12:23:19) [Clang 15.0.0 (clang-1500.0.40.1)]
- PIL 9.4.0 (Pillow)
- WSGI: On
- Server: waitress 2.1.2