You should fast (enough) iterating over all brains and building a mapping portal_type -> counter.
This should be fast even for some thousands of brains. Eventually itertools.groupby() might be helpful here.
This is a raw count from inspecting the internals of the index. As such, it requires loading the entire index from the ZODB, and won't work if you also need to apply other indexes to limit the results. But it should be faster than doing a query and looping over the results.
The catalog search result is in fact a LazyMap, i.e. a sequence (of catalog record ids) and a function. On access to a sequence component, the function is transparently applied to the base component (which gives you the catalog proxy/brain). Using the above internal implementation details, you can access the raw search result (the set of catalog record ids) and "and" it with the sets for the various values in the index your are interested in to obtain the count values.
Whether this is faster than looping over the proxies depends on the size of the result set and the size of the index.
# This takes the catalog record ids from the result set (_seq) and
# compares them with the records in the catalog (_index[portal_type])
portal_catalog = getToolByName(portal, 'portal_catalog')
_index = portal_catalog.Indexes['portal_type']._index
# We get a list of tuples when searching.
_seq = set(
x[1] if isinstance(x, (list, tuple)) else x
for x in self.lazy_resultset._seq
)
counter = {}
for portal_type in _index:
index_values = set(_index[portal_type])
counter[portal_type] = len(_seq & index_values)
I think this should perform better for large result sets, though I haven't benchmarked this
@jaroel There is a builtin function which gives you exactly this efficiently.
Almost completly undocumented but we make use of it in production. We do reports by combining different facets into a combined field and then walking over the histogram of those unique values to give grouping totals.
Sorry I missed that. If you want to count a filtered search result then that method doesn't work. What you would need to do is build that into the index you are counting. What I did was create a special groupby index with values like "Male|USA|White" etc and only had values on items I wanted to count. Then I iterate over the histogram rather than the over the whole DB to count things like number of Males or number of white males. So if you have much less combined groupings than records this works well. The tradeoff is if you need to add in another facet then is a huge reindex.
Not sure if this helps your usecase though.