Summer of Code 2016

Hi all,

As you may know, it's time for Plone to apply for Summer of Code again. I'd very much appreciate any suggestions of things we can propose as projects, as well as volunteers to mentor.

The ideas list isn't the be-all and end-all, but it should provide good pointers towards projects. Students can propose their own things, but everything on the list should be a viable project.

You can look at previous topics at: https://plone.org/events/summer-of-code/2015

Things that work well are work on popular add-ons or more involved core development tasks. Things that don't are new APIs, developer tools, or things the community needs. Nice to haves with clear outcomes (such as a add-on release or a PLIP) are the best bet.

Please do chime in with any suggestions, but let's not debate the merits of each just yet, it'll derail quickly!

Matt

Some remarks on the topics:

  • collective.cover JS improvements Plone

plone.app.mosaic is the way to go

  • Dexterity schema specification through HTML

Really not needed and without any need in reality

  • Better Microsoft/Open Office integration

Alex Pilz asked lately about reviving EE (perhaps he has some funding)

  • WebODF support

WebODF is buggy dead horse. Integration of Prosemirror would give a much higher benefit.
Perhaps a integration of Prosemirror with plone.app.mosaic would be the primary way to go.

  • PyPy support for Zope

Completely out scope for a GSCO projects...Priority #1 regarding the Python version must be support
for the latest Python 3.X version

  • Own idea #1:

Integration of Plone with external systems.

The most advanced solution is my xmldirector.plonecore package which provides a unique API
to external systems.

See

The integration is build on top of pyfilesystem https://pyfilesystem.readthedocs.org/en/latest/

Some integrations require an external SaaS implementing the specific protocols.

However popular systems like Dropbox could be integrated into Plone easily. The only need is to implement a specific driver e.g. making the Dropbox API available to pyfilesystem. At least for Dropbox there are some older driver implementations that could be ported to the newest Dropbox API.

Similar integrations could be done against for example Sharepoint with comes in its newest version with a REST API.

  • Own idea #2

Better integration of Elasticsearch with Plone.

Some people of the Plone devs are in favor for SOLR over Elasticsearch however Elasticsearch is likely the better solution in the long run. There are some integration efforts of ES with Plone (including a half-finished fork of collective.elasticindex which currently lacks integration with Elasticsearch 2.0 due to an incompatibility of the underlaying PyES version that does not support ES 2.0).

I am able to provide mentoring support for both projects.

-aj

  • get a few themes for Plone 5
  • porting existing add-ons to Plone 5
1 Like
  • I really would put p.a.mosaic on the list since we strategically decided to push it and go this direction and remove c.cover here - which is a great addon, but i see it as one step in the direction of the old Deco idea, where mosaic may be the better approach.
  • I'd really love to see an bullet point like: "Port as much as possible to Python 3" (together with the Zope4 efforts).
  • If theres a student good in AST/compilers: the topic "RestrictedPython on Python 3" would be really valuable. @loechel might want to add something here?
  • Also new idea: Splitting up plone.app.registry (i.e. using xinclude) into several files to avoid this unmanageable large beast we currently having, together with a per-controlpanel export (or by one interface). I sketched latter already, base is quiet simple. A good UI is more difficult thing. We may want such an import too. This would improve the TTW story and helps getting TTW things back in FS.
  • ad integrate WebOb: IMO we FIRST should add a WSGI publisher (maybe revive repoze.zope2) and THEN go to WebOb.
  • i dont see any value in Dexterity schema specification through HTML - this more or less scares me away. We already have 2 different ways to extend dx-types. Please first help fixing the existing instead of adding another way of doing the same.
  • I dont think PyPy support is really number one priority at the moment. But as long as code is not bloated by many special if HAS_PYPY: it does not hurt.
  • Is WebODF stable? I played around a bit some months ago and removed it from my lists of things to consider for TTW text-editing.

since I was originally proposing WebODF, I would also remove that. Or better re-phrase it as what it was supposed to bring:

  • collaborative editing (however rudimentary)
  • 'track changes' function that works for non-techie people.

it seems prosemirror can do both, indeed. Thanks, @zopyx

+1 - also in mind with collective.indexing merge into Plone core (Plip got accepted).

2 Likes

Leave out the collaboration aspect in the context of Plone. Plone is and remains a web CMS for most of its users. Collaboration is more than being able to edit a text in parallel (which only a few number of people really need and do in reality). Tracking chance, making annotations, discussing and accepting changes is what is really needed..and only a few commercial web-tools got this right (xeditor.com, Smashdocs). Too many efforts have been wasted over years with half-baked and especially badly designed prototypes...focus on the things that count and that are important to the majority.

-aj

I don't even remember having pushed that proposal at all (AFAIK, the original request is from @zopyx and dates from 2013).

BTW, collective.cover is a content type to solve specific issues for landing pages; that has nothing to do neither with Deco, nor with plone.app.mosaic; anybody looking at collective.cover as a layout system has lost the whole idea.

1 Like

Sorry for confusing this, I just saw cover being used as a layout system out there in the wild :confused:

1 Like

+1 for these ideas, which were already stated:

  • WebOb instead of HTTPRequest, HTTPResponse objects
  • Refactoring the Zope publisher for WSGI support, using zope.publisher and repoze.zope2
  • Porting Zope and plone.* to Python 3 and PyPy (We have a lot of packages which can individually be ported)
  • Making Products.RestrictedPython Python 3 compatible
  • Creating some high quality themes
  • Rephrasing an idea from last years list: Improving collective.easyform - that instead of PFG. It reuses many code parts of PFG, e.g. validation and such, but builds upon the Dexterity TTW form editor. Which is a really nice idea...

More ideas:

  • Working on plone.login
  • Improving plone.app.event recurrence support with individual start/end times per recurrence, individual information for occurrences and immutable occurrences, once they have passed.
  • Timezone support for all date/time fields (e.g. publication and expiration dates, creation and modification dates)
  • Replacing DateTime with Python datetime

Thanks all, keep them coming!

If anyone has any they'd particularly like to happen, feel free to ping me an email with a write up in the same style as on the current list (otherwise I'll have to write them all!)

Matt

Likely good idea long-term, but this is likely to break lots of add-ons and their testing assumptions. Might be good GSoC project, even if the integration story is a longer-curve.

If someone wants to work on a user-created forms story, I will provide assistance, possibly mentor. I have little experience with PFG, but lots of recent experience doing complex user-created form work using the same tools underlying Dexterity (plone.alterego, plone.supermodel, plone.schemaeditor, and lookup/registry systems for dynamic schema).

+1 on all of these, especially the last, which might be pre-requisite to sanely dealing with time-zones anyway.

Maybe ExternalEditor and/or DAV story more generally is something we should strive to work on. It was once a theoretical unique advantage to our platform, that has not recently been given enough love and attention. Other CMS systems that do DAV, don't do it nearly as well (e.g. Alfresco doesn't do PROPSET, etc). OTOH, DAV often gets neglected by folks pushing agenda-driven protocols inventing the wheel (e.g. CMIS). It may well be the case that DAV integration story needs something like EE (and/or Windows shell extensions) to provide a whole product that anyone actually cares about?

Not to hijack your example, having some kind of (simple?) round-trip full-cycle from DOCX to semi-structured (document) content in Plone would be nice? Even if just some base OOTB setup that leveraged ExternalEditor to get DOCX from a document, edit, transform back to HTML fragments. I have colleagues who would love to have something like this for a structured document CMS used for web-publishing of information authored by subject-matter-experts. Maybe this is doable by a student, but not sure.

Sean

As someone who has looked at this codebase in some detail, this feels like a high-risk, high-effort, low-returns project. If anyone wants to try this, I think it might make more sense to simply make HTTPRequest/HTTPResponse implement the WebOb interface.

In any case, a much more fruitful project that would require about the same amount of effort, but would involve a lot less risk, is to take a fine scalpel into the ZServer code base and split it into two parts:

  • An independent ZServer package that is a WSGI server which is also capable of talking FTP, mapping FTP commands into WebDAV HTTP commands. This code should have no dependency on HTTPRequest/Response or ZPublisher.
  • Some request marshaling and "module publishing" code that should be folded into ZPublisher, HTTPRequest and/or App.

This same project would turn ZPublisher into an WSGI app that handles the "module publishing" in the old Zope2 fashion.

It should also clear out the cruft from all previous attempts at WSGI from Zope2, which mostly broke the "module publishing" machinery, but only created a barely functional Zope2 WSGI app (broken error handling, among other issues). This is because it attempted to create the WSGI app at the wrong level of the Zope2 code base.

When you look at the ZServer code base in detail, you come out with the impression that the WSGI spec was mostly taken from its implementation, so the project as I described above would have very well defined boundaries and bring about two large benefits:

  • a WSGI server package that would bring FTP capabilities to any WSGI app that handles WebDAV commands.
  • Turning Zope (hence Plone) into a true and native WSGI app.

All this with zero impact on code bases outside of the lower levels of the Zope2 codebase.

Unfortunately I'm not sure I can afford the time to mentor such a project, but if someone else wants to drive this, I can give detailed pointers to the necessary places in the source code and be on point to answer any questions and provide technical guidance.

1 Like

Some ideas

  1. built in import/export of content. As in a user with permission to a subtree can get a zip of that content, download it, and upload it to a new site.

  2. Improvements to the plomino UI to make it a replacement for PFG. There are some mockups of how this could work in https://github.com/plomino/Plomino/issues/705.

  3. Remove the need for default pages. ie solving the UX problem "The idea of default pages is not obvious" - https://github.com/plone/Products.CMFPlone/issues/519. This requires more work than just installing mosaic, removing folders and making a page folderish.

  1. solving the problem that navigating to add new content is sometimes tedious and not obvious
  1. Improve the understandability of sharing/roles/workflow by updating their UI.
  1. builtin anonymous sharing but integrated into the UI
  1. Dashboard in the toolbar so search, favourites and other plugins can be available to an editor quickly.
  1. Side by side live preview of rendered page while editing
1 Like

Porting Zope2 to Python 3 seems a good idea to me. AccessControl and RestrictedPython could be challenging topics. Most other packages can be done more less automatically with the 2to3 script:

I'd remove DocumentTemplate which would mean to either rewrite or drop the ZMI.

About WSGI support. Did I miss something or is there WSGI support already:
http://zope.readthedocs.org/en/latest/WSGI.html

I probably needs more testing and some examples:

Remove and merge metadata columns and indexes: https://github.com/plone/Products.CMFPlone/issues/1279

I would volunteer to mentor that one if needed.

  • Make a UI for adding indexes, metadata, collection criteria, and collection fields to be displayed in tabular view.

Since I've recently gone through the pain of doing this myself (with code), I'd be happy to mentor this one :slight_smile:

I think its important to consider with priorities that while the zope4, rip out old code stuff is important, we need to ensure that our future releases have improvements mixed with cleanup. As we've discovered with plone 2.5 and plone 5, big changes that result in the same or less functionality are hard to sell to clients and developers. For example if we ripped out the ZMI in plone 5.1 or Plone 6, how long would it take to replace everything we'd lose and what else would be in that release that would make the lose worth it?
Thats partly why I'm suggesting all UX related changes for summer of code despite some not having certain outcomes. We need to experiment to get to an improved experience for end users.

PLIP: Zope 4 / Update Dependencies is not so much about ripping out old stuff at the first place, in fact everything is in place except really minor things like ZMI Icons or the low-level bobo modification date (different from our DC Modified). Its more about getting all packages to the latest releases and having more and more packages Python 3 compatible.

Further thought it will result in ripping stuff out, but IMO its more about reducing the several stacked layers we have than ZMI.

Goal is also to enhance the TTW experience, and ZMI is a bad experience. We wont get rid of it and it has its value for power-users, but we want to add more control-panels for the daily TTW integrators. The PLIP: Portal actions control panel is one of those steps and it would be good to have more.

This idea:

heads exact in that direction and I'd love to see an implementation here, if its SoC or otherwise implemented.

1 Like

I'm sure the goal is to enhance the experience I'm just saying that historically when we change the underlying layers we break plugins in one way or other and/or often replace one way of doing things with another. It's often a better way and for a good reason and perhaps this zope4 change (which I'm not tracking very closely) won't result in any of these downsides. If so then great.
All I intended to say is we should put as much effort into improving the UX as we do the underlying layers because otherwise it makes it hard to sell upgrades to clients.
Not sure about others experience but Improved UX I can sell. Improved security I can sell. Even an easier theming experience I can sell as some of my clients theme their own sites. But something that costs a lot to upgrade but looks and works exactly the same to the end user in the end I can't sell.

1 Like