PyCharm usage tips for developing Plone? (fixing global module imports)

I somehow got the debugging to work... I enabled buildout support, and gave it the instance so it could find paths to eggs (I have omelette in this buildout but I don't know if it was needed).

The debugging in PyCharm is really very nice.

1 Like

Did you also get the popup in PyCharm to install native debuggng extensions in your projects Python? This speeds up the debugging python process a lot.

omelette is not necessary for debugging afaik, but adding parts/omelette to the paths that PyCharm scans for source files does help with code completion, searching fof symbols etc. as long aa the buildout support in PyCharm is not fixed.

Here are instructions to use PyCharm without undermaintained Buildout support:

At first, add part for interpreter named "python" into buildout.cfg:

[buildout]
parts =
    ...
    python

[python]
recipe = zc.recipe.egg
eggs =
    ${instance:eggs}
    plone.recipe.zope2instance
interpreter = python-interpreter

This will create a new interpreter ./bin/python-interpreter, which can be configured as a new local Project Python interpreter into PyCharm.

This is enough for all basic features and also for running bin/instance fg as configured task directly from PyCharm.

I struggled a bit to get PyCharm debugger working today. It used to work, but possibly something has changed in the recent versions and now it manages to drop all Plone eggs from its working set. That can be patched in buildout.cfg with:

[instance]
recipe = plone.recipe.zope2instance
...
initialization =
    import pkg_resources
    pkg_resources._initialize_master_working_set()

Obviously, this should only be added to instance parts used during development. It will enforce pkg_resources' working set to include all paths in sys.path (otherwise PyCharm debugger manages to reset it and drop all packages add by ./bin/python).

Now also configured ./bin/instance fg tasks can also be run with PyCharm debugger with its great features.

With this configuration, please, disable omelette from buildout so that you won't accidentally import stuff with omelette paths.

Update: I changed interpreter = python to python-interpreter so that developers having virtualenv in the project directory would not broke their environments. (PyCharm requires Python interpreter binary to start with python.)

1 Like

I don't recall seeing that popup, but I may just have closed it without understanding what I was being asked :slight_smile:

I did not notice slowness when debugging with PyCharm... my brain takes much longer to process what is going on at each breakpoint than the time it takes to get between breakpoints!

I must have stumbled on this... this just worked for me. On other projects there is this weirdness that I can't specify which Python interpreter to use; it presents me with a list of choices that is generated in an obscure way.

Python interpreters are defined and connected to project in project settings. It's surprising limitation that they cannot be configured on that Run/Debug configuration form.

1 Like

You are a master of understatement and dry humour (or is it sarcasm?) :smiley:

Thanks for this idea:

i tried it but cant get it to work:

    options for buildout: 'backupdir' 'base-parts' 'cron-parts' 'logdir' 'nginx-parts' 'prod-environment' 'supervisor-parts' 'testeggs' 'unzip' 'varnish-parts' 'zeo-ha-parts' 'zeo-single-parts'.
Installing _mr.developer.
Generated script '/home/gomez/dev/hs-orga/bin/develop'.
Installing instance.
Getting distribution for 'python'.
error: Couldn't find a setup script in /home/gomez/dev/Python-2.7.15/lib/python2.7/lib-dynload
An error occurred when trying to install /home/gomez/dev/Python-2.7.15/lib/python2.7/lib-dynload. Look above this message for any errors that were output by easy_install.
While:
  Installing instance.
  Getting distribution for 'python'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
  File "/home/gomez/dev/hs-orga/local/lib/python2.7/site-packages/zc/buildout/buildout.py", line 2127, in main
    getattr(buildout, command)(args)
  File "/home/gomez/dev/hs-orga/local/lib/python2.7/site-packages/zc/buildout/buildout.py", line 797, in install
    installed_files = self[part]._call(recipe.install)
  File "/home/gomez/dev/hs-orga/local/lib/python2.7/site-packages/zc/buildout/buildout.py", line 1557, in _call
    return f()
  File "/home/gomez/.cache/buildout/eggs/plone.recipe.zope2instance-4.4.0-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 139, in install
    installed.extend(self.install_scripts())
  File "/home/gomez/.cache/buildout/eggs/plone.recipe.zope2instance-4.4.0-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 638, in install_scripts
    requirements, ws = self.egg.working_set(['plone.recipe.zope2instance'])
  File "/home/gomez/.cache/buildout/eggs/zc.recipe.egg-2.0.5-py2.7.egg/zc/recipe/egg/egg.py", line 84, in working_set
    allow_hosts=self.allow_hosts,
  File "/home/gomez/.cache/buildout/eggs/zc.recipe.egg-2.0.5-py2.7.egg/zc/recipe/egg/egg.py", line 162, in _working_set
    allow_hosts=allow_hosts)
  File "/home/gomez/dev/hs-orga/local/lib/python2.7/site-packages/zc/buildout/easy_install.py", line 925, in install
    return installer.install(specs, working_set)
  File "/home/gomez/dev/hs-orga/local/lib/python2.7/site-packages/zc/buildout/easy_install.py", line 679, in install
    for dist in self._get_dist(requirement, ws):
  File "/home/gomez/dev/hs-orga/local/lib/python2.7/site-packages/zc/buildout/easy_install.py", line 571, in _get_dist
    dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
  File "/home/gomez/dev/hs-orga/local/lib/python2.7/site-packages/zc/buildout/easy_install.py", line 1708, in _move_to_eggs_dir_and_compile
    [tmp_loc] = glob.glob(os.path.join(tmp_dest, '*'))
ValueError: need more than 0 values to unpack

Already updated buildout, setuptools and created a new python. Some1 an idea, whats wrong?

I fixed this by replacing ${instance:eggs} with ${buildout:eggs}. Could be my buildout configs...

Now i get all eggs in the Interpreter:

It still does not find plone.api:

Is there a other setting needed?

I tried Asko's setup and I did get it to run with a recent Plone 4.3.17, zc.buildout 2.11.4 and setuptools 39.2.0, but it doesn't fix the import problems I have in PyCharms code inspections either. I changed PyCharms python interpreter to the generated bin/python-interpreter.

@datakurre In your projects, do you still scan a generated parts/omelette directory?

@gomez Which PyCharm version are you using, the 'stable' 2018.1 or the upcoming 2018.2? Mac or Linux? Interesting that your Project Interpreter lists the correct Python version and all eggs, mine seems 'stuck' on an unknown. Maybe my virtualenv is doing something weird.

Here my versions:

PyCharm 2018.1.4
setuptools == 39.2.0
zc.buildout == 2.11.5
Plone == 4.3.4

I use virtualenvwrapper and removed the omelette directory.

@datakurre Does the import problem is fixed for you?

@gomez @fredvd Unfortunately, I believe that (from plone import api showing api as unknown) is as close as we can get without omelette. And that is understandable as you see that "plone" in that line is mapped to the first "plone" namespace package PyCharm finds.

I'm sorry for overpromising :frowning: When I found out this approach, my project did not have plone.api yet. Personally I'm still happy how little I need to configure in PyCharm for this.

There is a workaround for plone.api like packages, though.

Instead of

from plone import api

you could write

import plone.api as api

And PyCharm finds it.

1 Like

Could be worth telling JetBrains at this point.

We have. It is effectively the problem is that PyCharm supports best VirtualEnv-based installs where installed packages are in a single folder tree (eg. with only a single "plone" folder), but Buildout environment installs each package separately (eg. each plone.x package has its own "plone" folder) https://youtrack.jetbrains.com/issue/PY-20788

(In my opinion the world would be better place if Python would have adopted buildout's approach...)

Has anyone found a workaround for this or is it no longer an issue for some? I still have an issue on 2018.2 professional with Pycharm unable to find zope.schema and plone.api only. I do not know what pycharm considers unique about those packages.

It's still an issue and I'm afraid it's not something the PyCharm team at Jetbrains is ever going to fix.

For 1.5+ years since bringing it to their attention I have received notifications from their issue system where the 'versions' affected are expanded to the next released version .

buildout isn't sexy enough anymore to warrant spending time on it to bring new customers in for PyCharm. I cannot think of any other reason. Even stranger though Jetbrains doesn't remove the buildout support alltogether if they don't intend to support and fix it, but leave it in there, broken and all.

I''m slowly moving all my projects into VS Code and will ditch my professional license for PyCharm next year.

Since it's just these two I suppose I can just do a pip install with --no-dependencies for these two.