Huge Data.fs, mostly by plone.app.imaging.scale.ImageScale

Plone Support Administering Plone sites
,
1

Hi,

I have some Plone 4.3.x web sites with a very huge Data.fs (> 30Go), after analysing it, it figures that the most part of the volume is taken by images.

I thought that images where stored into the blobstorage, no ?

For example :

plone.app.imaging.scale.ImageScale 57.3%
OFS.Image.Pdata 39.1%

Is there is a way to reduce the Data.fs volume ?

Thanks

2

Did you try to pack it, maybe locally using:


?

3

We use this script to purge all image scales as part of a migration:

import transaction
from zope.annotation import IAnnotations
from zope.component.hooks import setSite

KEY = 'plone.scale'

site = app.plone_portal
setSite(site)
catalog = site.portal_catalog

query = dict(portal_type=['Image'])

for brain in catalog(**query):
    obj = brain.getObject()

    annos = IAnnotations(obj)
    if KEY in annos:
        print(obj.absolute_url(), 'SCALE REMOVED')
        del annos[KEY]
    else:

        print(obj.absolute_url(), 'no SCALES found')

transaction.commit()
1 Like
4

I dont think they always do (there are different 'field-types' used).

Personally, I consider it a bug if it is possible to have 'Image' saved as blob, but scales as part of Data.fs

Is there any use for that ? Maybe the 'icons and thumbs ???

5

I believe a newer version does store scales as blobs.

6

This is correct (at least in Plone 5.2).
Plone 4 clearly used to store scales within the ZODB. But I do not recall at which point the storage of scales switched to blobs.

7

This looks like some very old cruft....

8

Yes, they are (at least on Plone 4.3+). But the objects which references the actual image blob are kept Data.fs. How many instances of ImageScale do you have?

Also, you can use collective.zodbdebug to see if the image scale blobs are in fact stored in the blobstorage.

And, for reference you may also check these issues:

1 Like
9

Hi,

Thanks for all your replies.

Finally, we have found another solution : removing orphaned revisions.
This leads to reduce from 30Go to 3Go the Data.fs !

4 Likes
10

How did you do that? I might have to do this too.

11

You can remove orphaned revisions manually in portal_historiesstorage I think. But it is easier with collective.revisionmanager, install this as add-on and you have a nice control panel.

Also, for image scales, this script with bin/instance run on the command line can help.

Note that when you do any of this, you will not see a Data.fs or blobstorage size decrease yet. You first have to pack, keeping zero days.

2 Likes