PyCharm usage tips for developing Plone? (fixing global module imports)

Only useful for people using PyCharm while developing Plone.....

After winning a pycharm license at last years Plone Conference lightning talk lottery in Boston I finally got so fed up with my current editor that I really want to switch to Pycharm. :slight_smile: I've figured out quite a few things in the meanwhile, PyCharm is very powerful and cool, but for some of its best features to shine it's a hit or miss thing. I want to pay attention to all warnings and inspections it does, but then all the warnings have to make sense be correct, which they aren't.

By setting the interpreter to a virtualenv I'm creating in every buildout and activating the buildout support by selecting /bin/instance, PyCharm figures out and can complete all missing imports. PyCharm even distinguishes between my project code in /src and 'external' default Plone/Zope eggs.

This setup lets PyCharm find most of the modules, classes and functions and I can add a missing import for getToolByName without remembering Product.CMFPlone.utils, but this doesn't work:

from plone import api
from zope import schema

then I get for "Package containing modules 'plone' is not listed in project requirements.". The instance runs fine though, also in Pycharms run environment and debugger. "from plone.api import portal" is not frowned upon by the inspection.

However, in the PyCharms project structure I've excluded the parts and var directories. If I add parts/omelette to the project structure as sources, the imports above work again, but I get the impression that PyCharms indexer is now running much longer and doing double work: it finds imports from bin/instance (buildout support), and finds most of the same imports again in parts/omelette. Also my own custom modules in /src/ are replicated in parts/omelette. What is the correct setup here?

Another question: PyCharm suggests putting the .idea folder in git to share settings with others using PyCharm. Activating buildout support however puts a lot of local developer install paths in the .idea subdir, like the path to a shared buildout cache. On every commit or pull you're battling anther committers paths. Any suggestions for this, maybe explicitly excluding .idea/libraries/Buildout_Eggs.xml in your VCS ignores?

2 Likes

I've excluded .idea altogether because those path related issues.

I do have the same issue with top level imports, and don't have a solution :frowning:

Calvin did a lot to get a seamless Plone+PyCharm+buildout going. The first thing he had to do was get PyCharm's project preferences for buildout to point at the right buildout-generated bin script. One that had all the paths. I think he pointed at bin/instance.

Fred and I do already use that, and as mentioned, it finds the most imports correctly, but eg. from zope import schema highlights as broken. (But from zope.schema import something works.)

bin/test should also be a strong candidate for that in case your instance egg set is a subset of the test egg set.

I am pointing the buildout support script to bin/instance, I already found this hint in a blog post made in 2013 by Franklin from Four Digits. (https://www.fourdigits.nl/blog/pycharm3-and-plone/)

The project navigator also shows the 2 eggs (zope.schema and plone.api) as being found under the External Libraries / Buildout Eggs

Coïncedence: this morning the release candidate for PyCharm 2017.2.1 was announced, one of the solved issues is "Namespace package resolution bug on remote interpreters", the issue is documented here. https://youtrack.jetbrains.com/issue/PY-2813

On the surface it looks similar to what I'm seeing, the orginal submitter also reports this while using a virtualenv, but the original issue is from 2011. I just installed 2017.2.1, and reindexed my project, but it isn't working unfortunately, maybe it is related, or another issue altogether.

No, that has to be another issue, it was working perfectly until some months ago (I think when the switch to 2017, but I can not remember correctly).

I've been using, and enjoying, PyCharm for some years already :slight_smile:

I even made an external script to get bin/code-analysis output right into PyCharm console :slight_smile:

Here is the write up I just published on my experience getting buildout and Plone working with PyCharm. It is pretty life changing :heart_eyes: once you have the debugger going. I wish I would have had this years ago.

3 Likes

Hi Calvin, thank you very much for the thorough blog post on your pycharm setup. Two questions related to my 'import' problems:

  • do you still have omelette in your buildout, generating the symlinked module namespace under parts/omelette ?

  • did you change anything in the Project Structure preferences for certain folders in your buildout? For me it was common sense to exclude parts en var.

@gforcada So some of the imports have been breaking for you since some months now and before it was working fine? Then we'll have to file a bug report.

  • I do still have the Omlette linked in, but that is just because I have my egg cache outside of my workspace.

  • I hadn't been ignoring those directories, but I think I will ignore anything that isn't checked into source control management. This biggest speedup to indexing on a project recently was to not index blobstorage but I already have PyCharm configured to ignore any directory with that name.

I do not notice that I have the same issue with zope.interface imports. They are showing as an Unresolved reference in the editor. I just downloaded and tried with the latest 2017.2 RC so it doesn't appear to be fixed.

For zope.interface imports I've found it is not picking up imports that are done in __init__.py so I will have to change "from zope.interface import Interface" to "from zope.interface.interface import Interface". I have the same problems you mentioned with plone.api and zope.schema too.

These aren't deal breakers - PyCharm has been a huge huge huge improvement over my previous development experience - but it would be nice to clean up.

You could help to improve support of zc.buildout in PyCharm by voting for this issue in the "PyCharm Bug Tracker" - https://youtrack.jetbrains.com/issue/PY-20788

1 Like

@Cykooz Thanks for pointing out the issue. I've voted, if other who use PyCharm could vote as well this could put the issue(s) on the PyCharm development team's radar. It would be great to have a reliable missing imports indication from PyCharm when using buildout.

Hi

I was using PyCharm 5 and top level import were working great.
I've just updated to 2016.1 and those are not working anymore...

Have you found a solution ?

Hello @jihaisse,

No unfortunately there's no solution or fix yet from Jetbrains/PyCharm. 2017.3 was released yesterday, but this issue is still open. As Kirill suggests you can log in to Jetbrains issue system and vote for this issue at (https://youtrack.jetbrains.com/issue/PY-20788).

An alternative is to allow PyCharm to scan a created parts/omelette folder with symlinks to the whole module hierarchy. Big downsides (for me) are that scanning takes a lot longer and that jumping to files or Python classes will now show double entries for the modules that could be resolved already and are now also found under parts/omelette (https://pypi.python.org/pypi/collective.recipe.omelette/)

I'm afraid though that good buildout support is no longer a priority for PyCharm at the moment to bring in more/new customers. python backend and javascript frontend and scientific are the hot topics.

Cheers,

Fred

Ok, thanks.

I've up voted the issue, but with only 14 votes... Anyway...

I will try the omelette.

1 Like

Btw, I tried out that if Plone is installed into virtualenv normally with pip, PyCharm finds global module imports properly (and buildout support is not needed).

2 Likes

I would love to find out how to get debugging working with PyCharm (2017.3)... anyone want to volunteer posting some screenshots of their setup?

1 Like

I voted and tweeted it. https://twitter.com/djay75/status/996956287098236928

If we can get to everyone in plone land using pycharm it should be more than 16…

2 Likes

Plone Foundation Code of Conduct