Interactive shell for debugging with Plone 6 docker-compose: The wsgi equivalent of bin/instance debug

[pigeonflight (David Bain (Will Theme Plone Sites))]

Today I wanted to do some interactive debugging.

Old way

The old way before wsgi tended to look like

bin/instance debug

You'll get an interactive prompt. The root of your Zope will be called app and, most likely your Plone site will be called Plone

>>> plonesite = app.Plone

and then do what you want on the interactive prompt.

New way

For the default docker-compose setup this works:

docker compose exec backend ./docker-entrypoint.sh console

and then work the prompt as expected

>>> plonesite = app.Plone

Background Research and round-about way

DON'T DO IT THIS WAY. I'VE LEFT THIS HERE FOR REFERENCE PURPOSES
I found some documentation on using zconsole for debugging

And got some help over in the Plone Slack channel from @tiberiuichim
He pointed me to The Big List of Small Volto Rules · Issue #2810 · plone/volto · GitHub
With some further guidance from @tiberiuichim I finally came to this as the solution:
This way involves modifing docker-compose.yml to set bash as the default command.
This prevents the instance from starting as normal.

version: "3"

services:

  backend:
    #build: .
    image: <the image>
    command: bash
    tty: true

Once running (typically launched with docker compose up -d)

The following series of commands launch an interactive shell

docker compose exec backend bash
./docker-entrypoint.sh bash
bin/zconsole debug etc/relstorage.conf

You'll see the prompt:
and it works as expected

>>>
>>> plonesite = app.Plone

Note the use of relstorage.conf
If you use zeo storage use zeo.conf instead
if you use file storage (Data.fs), use zope.conf instead

It looks in the folder where the for the conf files
I was successful with the plone-backend docker image.

[pigeonflight (David Bain (Will Theme Plone Sites))]

There's a problem with this approach.

It does not allow the use of the plone.api.
When I attempt to use the plone.api, I get a
plone.api.exc.CannotGetPortalError error.

Starting debugger (the name "app" is bound to the top-level Zope object)
>>> from plone import api
>>> api.content.find(path="/resources",portal_type="File")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/app/lib/python3.11/site-packages/plone/api/content.py", line 649, in find
    catalog = portal.get_tool("portal_catalog")
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/validation.py", line 73, in wrapped
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/portal.py", line 105, in get_tool
    return getToolByName(get(), name)
                         ^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/portal.py", line 68, in get
    raise CannotGetPortalError(
plone.api.exc.CannotGetPortalError: Unable to get the portal object. More info on https://docs.plone.org/develop/plone.api/docs/api/exceptions.html#plone.api.exc.CannotGetPortalError

The old way to resolve this was to pass the portal_id using the -O option:

for example bin/instance -O Plone debug.

see: plone.api.exc – Backend – <code class="docutils literal notranslate"><span class="pre">plone.api</span></code> – API methods and descriptions — Plone Documentation v6.0

Using the docker-entrypoint.sh approach, I'm not sure how to pass the portal_id.
I tried the following:
docker compose exec backend ./docker-entrypoint.sh -O Plone console
But got the following error output:

./docker-entrypoint.sh: line 161: exec: -O: invalid option
exec: usage: exec [-cl] [-a name] [command [argument ...]] [redirection ...]

[pigeonflight (David Bain (Will Theme Plone Sites))]

In the absence of being able to pass the option at the command line, I've found a way to get the portal object to the api, I don't like it, but it works.

I had to explicitly set the portal_id using setSite in conjuction with makerequest BEFORE being able to use the plone api in the interactive console.
So a session looks like this:

from Testing.makerequest import makerequest
from zope.component.hooks import setSite
app = makerequest(app)
setSite(app["Plone"]) # Plone is the portal_id
# Now I can use the Plone api in my interactive session
from plone import api
api.content.find(path="/Plone/",portal_type="File") # I need to use the path to the portal, "/Plone" otherwise it searches all portals in Zope

I've noticed that when I use the plone.api in the interactive console it treats the path root as the root of Zope rather than the root of the portal. This means that in my interactive console, for a portal named "Plone", /Plone/resources is equivalent to /resources in my normal usage of the api.

[pigeonflight (David Bain (Will Theme Plone Sites))]

To avoid an AccessControl.unauthorized.Unauthorized: error when trying to create content from the interactive console with plone.api
I'm using adopt_user

portal = api.portal.get()
title = "my title"
description = "my file"
with api.env.adopt_user(username="admin"):
    obj = api.content.create(container=portal,type="File",title=title,description=description)

I'm getting an error: `AttributeError: 'NoneType' object has no attribute 'URL'
here's the traceback:

>>> with api.env.adopt_user(username="admin"):
...     api.content.create(container=portal,type="File",title=title,description=description)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/app/lib/python3.11/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/validation.py", line 73, in wrapped
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/validation.py", line 149, in wrapped
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/content.py", line 106, in create
    content.aq_parent.manage_renameObject(content_id, new_id)
  File "/app/lib/python3.11/site-packages/plone/folder/ordered.py", line 194, in manage_renameObject
    result = super().manage_renameObject(id, new_id, REQUEST)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/OFS/CopySupport.py", line 394, in manage_renameObject
    notify(ObjectMovedEvent(ob, self, id, self, new_id))
  File "/app/lib/python3.11/site-packages/zope/event/__init__.py", line 33, in notify
    subscriber(event)
  File "/app/lib/python3.11/site-packages/zope/component/event.py", line 27, in dispatch
    component_subscribers(event, None)
  File "/app/lib/python3.11/site-packages/zope/component/_api.py", line 146, in subscribers
    return sitemanager.subscribers(objects, interface)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/zope/interface/registry.py", line 445, in subscribers
    return self.adapters.subscribers(objects, provided)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/zope/interface/adapter.py", line 896, in subscribers
    subscription(*objects)
  File "/app/lib/python3.11/site-packages/zope/component/event.py", line 37, in objectEventNotify
    component_subscribers((event.object, event), None)
  File "/app/lib/python3.11/site-packages/zope/component/_api.py", line 146, in subscribers
    return sitemanager.subscribers(objects, interface)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/zope/interface/registry.py", line 445, in subscribers
    return self.adapters.subscribers(objects, provided)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/zope/interface/adapter.py", line 896, in subscribers
    subscription(*objects)
  File "/app/lib/python3.11/site-packages/plone/app/caching/purge.py", line 250, in purgeOnMovedOrRemoved
    if isPurged(object) and "portal_factory" not in request.URL:
                                                    ^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'URL'
>>>     obj = api.content.create(container=portal,type="File",title=title,description=description)
  File "<stdin>", line 1
    obj = api.content.create(container=portal,type="File",title=title,description=description)
IndentationError: unexpected indent
>>> with api.env.adopt_user(username="admin"):
...     obj = api.content.create(container=portal,type="File",title=title,description=description)
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/app/lib/python3.11/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/validation.py", line 73, in wrapped
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/decorator.py", line 232, in fun
    return caller(func, *(extras + args), **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/validation.py", line 149, in wrapped
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/plone/api/content.py", line 106, in create
    content.aq_parent.manage_renameObject(content_id, new_id)
  File "/app/lib/python3.11/site-packages/plone/folder/ordered.py", line 194, in manage_renameObject
    result = super().manage_renameObject(id, new_id, REQUEST)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/OFS/CopySupport.py", line 394, in manage_renameObject
    notify(ObjectMovedEvent(ob, self, id, self, new_id))
  File "/app/lib/python3.11/site-packages/zope/event/__init__.py", line 33, in notify
    subscriber(event)
  File "/app/lib/python3.11/site-packages/zope/component/event.py", line 27, in dispatch
    component_subscribers(event, None)
  File "/app/lib/python3.11/site-packages/zope/component/_api.py", line 146, in subscribers
    return sitemanager.subscribers(objects, interface)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/zope/interface/registry.py", line 445, in subscribers
    return self.adapters.subscribers(objects, provided)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/zope/interface/adapter.py", line 896, in subscribers
    subscription(*objects)
  File "/app/lib/python3.11/site-packages/zope/component/event.py", line 37, in objectEventNotify
    component_subscribers((event.object, event), None)
  File "/app/lib/python3.11/site-packages/zope/component/_api.py", line 146, in subscribers
    return sitemanager.subscribers(objects, interface)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/zope/interface/registry.py", line 445, in subscribers
    return self.adapters.subscribers(objects, provided)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/zope/interface/adapter.py", line 896, in subscribers
    subscription(*objects)
  File "/app/lib/python3.11/site-packages/plone/app/caching/purge.py", line 250, in purgeOnMovedOrRemoved
    if isPurged(object) and "portal_factory" not in request.URL:
                                                    ^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'URL'

[davisagli (David Glick)]

It would be a nice improvement to the docker-entrypoint.sh script to pass the -O flag here: plone-backend/skeleton/docker-entrypoint.sh at 6.0.x · plone/plone-backend · GitHub

It should pass the value of the SITE_ID environment variable, with a fallback to Plone as the default.

[pigeonflight (David Bain (Will Theme Plone Sites))]

@davisagli

It would be a nice improvement to the docker-entrypoint.sh script to pass the -O flag

100% agree.
Additionally, I need clarity on why api.content.create(...) is giving me AttributeError: 'NoneType' object has no attribute 'URL' in a the container.

[pigeonflight (David Bain (Will Theme Plone Sites))]

It could be an environment variable.
SITE_ID="Plone"
if the variable is set, add the -O option.

[davisagli (David Glick)]

The script in the docker image which creates a new site already uses the SITE_ID environment variable if it is set, or "Plone" as a default (plone-backend/skeleton/scripts/create_site.py at 6.0.x · plone/plone-backend · GitHub). I was trying to propose that the entrypoint console command should use the same logic.

[davisagli (David Glick)]

@pigeonflight

Additionally, I need clarity on why api.content.create(...) is giving me AttributeError: 'NoneType' object has no attribute 'URL' in a the container.

You might need to do:

from zope.globalrequest import setRequest
setRequest(app.REQUEST)

to make sure that the request is fully set up for any code that uses zope.globalrequest.getRequest

[pigeonflight (David Bain (Will Theme Plone Sites))]

@davisagli

You might need to do:

from zope.globalrequest import setRequest
setRequest(app.REQUEST)

That works! Thanks @davisagli

[yurj (Yuri)]

@pigeonflight

The following series of commands launch an interactive shell

docker compose exec backend bash
./docker-entrypoint.sh bash
bin/zconsole debug etc/relstorage.conf

Thanks for this precious help! Here just to add you can do also directly: docker-compose exec backend ./docker-entrypoint.sh console

# docker-compose exec backend ./docker-entrypoint.sh console
Starting debugger (the name "app" is bound to the top-level Zope object)
>>> 
>>> 
>>> 

To insert breakpoints in your code to do debug you can then just run

# docker-compose exec backend ./docker-entrypoint.sh start
Using ZEO configuration
2024-04-03 14:53:19 INFO [ZEO.ClientStorage:270][MainThread] zeostorage ClientStorage (pid=3783) created RW/normal for storage: '1'
[...]