What is the best way to remove members folder that are empty?

I have about 9000 Members folder, most of then are empty, some have important information in then. what is the best way to remove the folder that are empty?

Iterate over the member folders, check their contents (catalog search using 'path' index) and remove the related folder if empty. Any problems?

-aj

not sure about this, but maybe you can just use:
folder.getFolderContents()

and if it does not return any items, delete the folder ?

Use the catalog to find them, then send them away to the great beyond of your undo history:


catalog = site.portal_catalog
members = site['Members']

empty = map(
    lambda brain: brain._unrestrictedGetObject(),
    [
        brain for brain in catalog.unrestrictedSearchResults(
            {
                'portal_type':'Folder',
                'path': '/Members'
            })
        if len(catalog.unrestrictedSearchResults({'path': brain.getPath()})) == 1
    ])

members.manage_delObjects(map(lambda folder: folder.getId(), empty))
1 Like

I am trying to delete all empty members with the following which does not work.

            members = list(context.portal_catalog.queryCatalog({
                    "path":"/yc1/Members/" }))
            for b in members:
                     #folderid=str(b.getId())
                     #f.write('folder id is' + folderid + ' \n')
                    items = len(b.getFolderContents())
                    if items==1:
                        count=count+1
                        #deletedobjs.append(b.getId())
                        #f.write('obj deleted \n')
                        members.manage_delObjects([b.getId()]) #delete the folder

I decided to try the code provided by above and could not get that to work either
I am using
site=getSite()

to provide the value for the variable site. Can anyone point out what I am doing incorrectly. Thanks

I get the following error when I use the code provided by user seanupton

catalog = site.portal_catalog
AttributeError: 'NoneType' object has no attribute 'portal_catalog'

This is the code:
site=getSite()
catalog = site.portal_catalog
members = site['Members']

            empty = map(
                lambda brain: brain._unrestrictedGetObject(),
                [
                    brain for brain in catalog.unrestrictedSearchResults(
                        {
                            'portal_type':'Folder',
                            'path': '/Members'
                        })
                    if len(catalog.unrestrictedSearchResults({'path': brain.getPath()})) == 1
                ])

            members.manage_delObjects(map(lambda folder: folder.getId(), empty))

You cannot call b.getFolderContents() on a brain. Maybe b.getObject().getFolderContents() will work.

from plone import api
catalog = api.portal.get_tool('portal_catalog')

The most likely reason for not getting the site, is when you're running bin/instance debug without properly setting the threadlocal for site, like `bin/instance -O yc1 debug'.

See http://docs.plone.org/develop/plone.api/docs/api/exceptions.html#plone.api.exc.CannotGetPortalError

Additionally, getFolderContents() is security filtered so you may start deleting member folders that aren't exactly empty. Better to use the unrestrictedSearch solution.

In a script, you do not have the same environment setup as in a web request. That's why "getSite()" returns "None". In my scripts, I typically use "zope.component.hooks.setSite" to define which portal should be the current "site". Another contribution in this discussion suggests, that "bin/instance" could have a parameter ("-O") to do something similar from the command line.

I have been plugging away on this on and off for a while and I still can't get it to work. I have 2 applications written.

Application One code:

            members = list(context.portal_catalog.queryCatalog({
                    "portal_type":("Folder"),
                    "path":"/yc1/Members/"}))
                    #"sort_on" : "getId" }))
            for b in members:
                try:
                    obj = b.getObject()
                    #if obj.getId=='alias':
                    #   next
                    #f.write("got to before len statement \n")
                    #if len(obj.getFolderContents()) == 1:
                    #if len(b.getFolderContents()) == 1 or len(b.getFolderContents()) == 0:
                    if (len(b.getFolderContents())==0):
                         #f.write("got here \n")
                         f.write('deleting folder with url= ' + obj.absolute_url() + ' ' + str(obj.created())+' \n')
                         obj.aq_parent.manage_delObjects([obj.getId()])
                         #folder.manage_delObjects([obj.getId()])
                         count=count+1
                         deletedobjs.append(obj.absolute_url())
                         transaction.commit()

                except Exception:
                    f.write('exception on this url= ' + obj.absolute_url() + ' ' + str(obj.created())+' \n')
                    f.write (str(Exception))
                    pass

The above seems to work, but doesn't. It still leaves folders that are empty intact and sometimes delete folders with stuff in it.

Application 2 code(taken from this site):
site=getSite()
catalog = site.portal_catalog
members = site['Members']
empty = map(
lambda brain: brain._unrestrictedGetObject(),
[
brain for brain in catalog.unrestrictedSearchResults(
{
'portal_type':'Folder',
'path': '/Members'
})
if len(catalog.unrestrictedSearchResults({'path': brain.getPath()})) == 1
])

            members.manage_delObjects(map(lambda folder: folder.getId(), empty))
            transaction.commit()

I call the above code by issuing this command

/usr/local/Plone/zeocluster/bin/client1 -O yc1 run /usr/local/Plone/zeocluster/products/FacultyCV/rundeleteMembers.py

I get the following errors:

<five:implements
    class="Products.ATContentTypes.content.folder.ATFolder"
    interface=".interfaces.ICSVReplicable"
  />

is deprecated. Please use the directive instead.
actions = self.handler(context, **args)
/usr/local/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.7.egg/zope/configuration/config.py:706: DeprecationWarning: Using <five:implements /> in File "/usr/local/Plone/zeocluster/src/Products.csvreplicata/Products/csvreplicata/configure.zcml", line 31.2-34.6
<five:implements
class="Products.ATContentTypes.content.topic.ATTopic"
interface=".interfaces.ICSVReplicable"
/>
is deprecated. Please use the directive instead.
actions = self.handler(context, **args)
/usr/local/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.7.egg/zope/configuration/config.py:706: DeprecationWarning: Using <five:implements /> in File "/usr/local/Plone/zeocluster/src/Products.csvreplicata/Products/csvreplicata/configure.zcml", line 38.2-41.6
<five:implements
class="Products.csvreplicata.adapters.CSVReplicataObjectSearcherAbstract"
interface=".interfaces.ICSVReplicataObjectsSearcher"
/>
is deprecated. Please use the directive instead.
actions = self.handler(context, **args)
/usr/local/Plone/buildout-cache/eggs/zope.configuration-3.7.4-py2.7.egg/zope/configuration/config.py:706: DeprecationWarning: Using <five:implements /> in File "/usr/local/Plone/zeocluster/src/Products.csvreplicata/Products/csvreplicata/configure.zcml", line 42.2-45.6
<five:implements
class="Products.csvreplicata.adapters.CSVReplicataExportImportPluginAbstract"
interface=".interfaces.ICSVReplicataExportImportPlugin"
/>
is deprecated. Please use the directive instead.
actions = self.handler(context, **args)
2016-10-13 11:17:50 WARNING Init Class Products.Five.metaclass.RedirectsView has a security declaration for nonexistent method 'errors'
2016-10-13 11:17:50 WARNING Init Class Products.Five.metaclass.Utils has a security declaration for nonexistent method 'convert'
2016-10-13 11:17:52 WARNING SecurityInfo Conflicting security declarations for "setId"
2016-10-13 11:17:52 WARNING SecurityInfo Class "AuthoringContentFolder" had conflicting security declarations
2016-10-13 11:17:55 WARNING Init Class Products.Five.metaclass.p25wrapper_is_locked_for_current_user has a security declaration for nonexistent method 'is_locked_for_current_user'
/usr/local/Plone/buildout-cache/eggs/Zope2-2.13.24-py2.7.egg/Zope2/App/ClassFactory.py:22: DeprecationWarning: PlacelessTranslationService: The PlacelessTranslationService itself is deprecated and will be removed in the next major version of PTS.
return getattr(m, name)
/usr/local/Plone/buildout-cache/eggs/Products.ZSyncer-1.0.3-py2.7.egg/Products/ZSyncer/init.py:44: DeprecationWarning: The product_name parameter of ToolInit is now ignored
icon='zsyncer_icon.gif', # Must be in this dir.

What you assume as errors for your application 2 are in fact warnings only (either deprecation warnings at Python level or warning log messages).

The "path" in application 2's "unrestrictedSearchResults" looks wrong. It should be the full path to "Members".

Application 1 looks much more natural than application 2. In order to be able to use such standard code in scripts, I usually login as a Zope "Manager". The corresponding function is "AccessControl.SecurityManagement.newSecurityManager". It is called with 2 arguments "None" and a user object describing the user to be logged in. To get such a user object, I use the "getUser" method of the appropriate "acl_users" object.

Thank you for your response. Can you think of any reason why application one would delete some members that are empty but not others. It also deletes some members that have stuff in them.

I am getting this error on some of the member folders that are not being deleted. I have rebuilt the catalog and re-indexed but am still getting the error

2016-10-14 12:41:42,668 - plone.protect - DEBUG - object you attempted to mark safe does not have an oid
2016-10-14 12:41:42,791 - txn.140259554453248 - DEBUG - new transaction
2016-10-14 12:41:42,857 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:42,877 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:42,887 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:42,954 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:42,962 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:43,058 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:43,062 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:43,098 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:43,196 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523
2016-10-14 12:41:43,225 - Zope.KeywordIndex - DEBUG - Attempt to unindex nonexistent document id -1878693523

Catalog searches do not return the objects themselves but proxies (also called "brain"s). Apparently, in your current version of application one, you are calling "getFolderContents" on such a proxy (rather than the object itself). I am surprised that this does not give an exception but in any case, I would not trust the result. Call "getFolderContents" on the object.

Depending on rights of the "current user", "getFolderContents" may not see all content objects. Thus, if the current user lacks full rights, you may think you have an empty folder which in fact is not empty.

Apparently, the catalog data structures are not completely consistent. Usually, this is not a big problem (you are looking at DEBUG log entries not WARNING or ERROR or even FATAL).

I have often seen similar log entries (even ERROR log entries) (usually during deletion of something) without any other visible problems with the sites. Likely, Plone does occasionally something wrong leading to rare but usually harmless catalog inconsistencies. As the inconsistencies are mostly harmless, nobody seems to have yet spent time to analyse the causes.

I have changed it to call getFolderContents on the object to no avail.

            members = list(context.portal_catalog.queryCatalog({
                    "portal_type":("Folder"),
                    "path":"/yc1/Members/"}))
                    #"sort_on" : "getId" }))
            for b in members:
                try:
                    obj = b.getObject()
                    if (len(obj.getFolderContents())==0):
                         obj.aq_parent.manage_delObjects([obj.getId()])
                         count=count+1
                         deletedobjs.append(obj.absolute_url())
                         transaction.commit()

This looks good.

When a piece of code which looks good nevertheless does not behave as expected, debugging may be necessary to understand what goes wrong.

I assume that still sometimes a folder which should be deleted is not and at other times, a folder is deleted that should not be deleted. I would set up a copy of your site (I would copy "Data.fs"); manually delete all folders with the exception of some examples for the problems above. Then step through your code above with a debugger and find out where things do not behave as expected.

It may help to avoid the "try: ... except:" in the code above in order to understand the behaviour: "try: .. except:" can easily hide problems.

Thank you for your response. I am in the process of debugging.