State of profiling under Zope 4/ Python 3?

It seems my beloved Products.ZopeProfiler did not make it into Python 3 land.

What is the state of profiling under Zope 4? What to use, best practices?

@thet posted a screenshot and a link to


a couple of weeks ago.

I haven't used it much, but it looks promising and you can attach the profiler to a running process!

I was looking more for some kind of integration of the stdlibs profile (cProfile) module. Something where I can profile a single request through Zope, write the result to a file and start analyzing using the stdlibs pstats module in order to see callers, callees, cumulative time, number of calls.

If there is some nice UI on top of it, ok. Downloading a the requests profile would be enough. There are tools like snakeviz and others helping with analysis.

You could use repoze.profile, a profiling WSGI middleware. It uses cProfile under the hood.
See https://repozeprofile.readthedocs.io/en/latest/

1 Like

I usually use profilehooks for profiling together with snakeviz for visualization. It is a pity that CacheManager and DebugManager were removed from Zope. It's a but manual but it makes it easy to profile specific methods/views/tests.

@dieter: Are you planning to port Products.ZopeProfiler to Zope4/Python3?

@jensens: Please report about your experiences with repoze.profile and/or py-spy.

Philip Bauer via Plone Community wrote at 2019-9-12 07:35 +0000:

...
@dieter: Are you planning to port Products.ZopeProfiler to Zope4/Python3?

The main feature Products.ZopeProfiler provided over other
profiling solutions has been its "high level" (i.e. Zope rather
than Python level) profile support.
This was important when most logic was in ZODB objects
("Script Python", "DocumentTemplate", "PageTemplate");
nowadays, it resides mostly in views and therefore is profilable at
the Python level.
Therefore, I do not know whether it is important to have
Products.ZopeProfiler in a modern environment.

py-spy is not what I am looking for, so I did not test it deeper. I think @thet collected already some experience, maybe he can report here?

I tested repoze.profile and yeah, it just works!

Lazy as I am, I wrote a quick integration for plone.recipe.zope2instance here as a draft PR with a TODO-list (help welcome, I have no idea when I have time to finish it). See its README for available options. Lazy, because this took me including trying out, fiddling with KCacheGrind and this post) only 2 hours of work!

TL;DR: use branch, in [instance] set profile = on.

Experience: It gives access to the profile of the last request executed on an instance. There is a web-interface, the raw-data is written to a file and - a big plus - the data is written in callgrind/cachegrind format and ready for further analysis with a tool supporting this general format (if the additional package is installed). I used KCachegrind (also available as QCacheGrind for Windows/OSX) and its complete easy to get good results here.

3 Likes

Plone Foundation Code of Conduct