GSOC 2017 Improve the Customization of Plone Listings

@djay @datakurre I have been following up and trying to figure out the conversation between you guys, and I have a question related to this project idea.

Are we using any js library to do this idea?

jQuery and TinyMCE.

Mosaic Editor is built with just jQuery, because it was designed around 2009 and jQuery really was the best option at that time. In addition to jQuery we have implemented rich text areas with TinyMCE (because that's the current default rich text editor in Plone).

Thank you for your answer. I have read through the tasks description and this is how I understand it (in general):

  • We will have to work on creating a new dynamic template for the listing (by customizing the mosaic template)
  • Customize the rich text editor some how so that people can simply drag and drop the content of the template that they are creating.
  • create options in the template-creating process so that people can choose what to include on their pages

Am I correct on these general points that this idea requires? I am sorry for the late questions but it took me a while to get used to Plone.

Have you already tried Plone Mosaic? Here's a demo with the current content listing tile implementation: http://plonemosaic200rc4.herokuapp.com/Plone/content-listing (username admin, password admin)

The original idea by @djay is to:

  1. be able to drag any Mosaic tile into and out from rich text tiles (tecnically into TinyMCE); this would enable free form layout in addition to the current restricted grid (these tiles inside rich text tile are sometimes called "inline tiles", technically they could be just normal tiles)

  2. use that functionality to allow users to define (repeating) item layout for content listing tile (e.g. by somehow making content listing tile behave like editable rich text tile in the editor and use the resulting layout as repeated template when rendering content listing)

  3. use that functionality to allow users to define search form and (repeating) item layout for search results

Currently Mosaic allows dragging of tiles only into a restricted grid. The current content listing tile provides only a limited set static set of fixed templates for rendering the listed items.

@djay mentioned elsewhere that "advanced_ide" branch of Plomino (https://github.com/plomino/Plomino/tree/advanced_ide) has somewhat similar TinyMCE integration for defining Plomino forms.

I've agreed that the original idea might work from UX point of view. I've a bit concerned about 2), because normally Mosaic editor is WYSIWYG, but this time it would be used to edit a template (and some new method would be required to preview the content listing with that template).

But I do have many technical concerns, which I've mentioned a few time in related GitHub issues.

I add here a discussion, which I had with @djay earlier today. I believe, his answers to my critique gives more depth for the idea.

Ohh it is so much clearer right now. Thank you so much for the information. I really appreciate it.

As far as I can understand (correct me if I am wrong), your idea is not to extend the Mosaic Editor but to expand the text editor with the customized buildout tool that we can actually make (which somehow similar to the SiteOrigin page builder plugin for Wordpress), but Dylan's idea is to extending the Mosaic editor so that it can generate dynamic content instead of limiting within fixed amount of tiles and positions (which is somehow similar to the functionalities of the Live Composer add on for Wordpress), where they both render the content into html code after users are done with customizing the page they want.

Am I right?

Yes. That sounds correct.

My idea was to just implement "minimal viable product" of "customizable listings" by e.g. implementing a new "listing item layout" field with TinyMCE based editor for Collection content type and Content Listing Mosaic tile. That would have been independent from the Mosaic editor (a different UI), and would have only allowed minimal set of building blocks for the item layout (those seen in the current listings).

Dylan's idea is more generic. It would require significant feature for Mosaic Editor to make directly editable listing layout possible (tiles within rich text tiles), but might allow using all the existing tiles in content listings and consistent editing experience with the rest of the layout.

It's not really a MVP. A MVP would be the smallest part of the same UX without all the additional features in order to test the idea. Yours is more an alternative UX and implementation which would require a lot of throwing away to later switch to the idea of using tiles. It's more or less a different idea.

If it was just about any UX to achieve custom listings then we could just upgrade c.listingviews to plone 5.

To me the goal here is to make something really powerful, and useful such that people talking about how flexible plone is, how they can create these really complex sites without having to code or pay someone to customise it. So they then tell others to build their sites using plone. Views in drupal has always been one of the features that helps sell it. Lets try to do better. Being able to use any tile anyone invents in the future, automatically in a listing, is a cool feature. It's worth testing this idea I think, and working how to make it work.

I think I agree with @djay since Plone layout builder since it makes everything much easier for the user to build their website without even have to code. In addition to that, I feel like its even better than the Wordpress plugins since those plugins can slow down the website significantly. It basically a better version of the page layout builders that exist. Moreover, it reuses the Mosaic editor, which is what I like :))

I understand that there might be a lot of technical problems associate with this tasks, but that might be why this idea is fun. The goal is still making Plone greatly improved and I am sure we can sort things out when the project begins :slight_smile:

In the mean time, do you guys have any materials that associate with this (for example: the technology used, or any kind of documentation) so I can get used to it first?

As far as I know, there are 3 steps:

  • Customize the rich text field (which is the TinyMCE editor) so that we can drag and drop items into it
  • Make the Mosaic Layout builder accept the customized rich text field, also remove the grid layout limitation
  • Extend that to custom search page

Since I am very interested in this project (opportunity to learn more, front end and a little bit of back end, I guess), I would like to somehow start on it :slight_smile: Can you guys help me out with this

The backend composition is pretty well documented, but the documentation is spread into *.rst files in multiple packages (plone.tiles, plone.app.blocks, plone.app.deco [predecessor of Mosaic]). The original idea for builder is described in https://github.com/plone/plone.app.mosaic/blob/master/docs/Blocks.pdf, but the actual syntax has been changed since then. I overviewed the current composition at http://datakurre.pandala.org/2017/02/tile-based-layouts-and-esi-on-plone.html

For the Mosaic Editor implementation there's no documentation.

Part 1. is probably the hardest part. There's a lot of technical corner cases related to TinyMCE, drag'n drop and currently focused tile (and the editor code base in plone.app.mosaic.

Part 2. may not be required at all. There is no grid limitation in rich text tiles. Once tiles work within TinyMCE and the rich text is saved as it is. Some backend changes (into plone.app.blocks) are required to actually render tiles within rich text tile, but I may be able to handle that (required improvement has been on my roadmap for some time).

Part for using that rich text tile with inline tiles for rendering the content listing is missing (see Dylan's example in the quote far above).

Part 3. requires implementing form elements as tiles; those available tiles should probably be generated from the currently available search options (https://plone.org/qsOptions ; the qsOptions API is used also by current Collections' query builder). Backend support for the search is a different issue. Applying the layout for a Collection page would give configurable backend search for free..

I just saw your comments on my draft proposal. I am very glad to have you support me on this process. I will let you know when I have questions related to the proposal when I work on editing it :slight_smile: If it is okay
I will also work on researching more about the project based on the resources you shared with me :slight_smile:

Hey @datakurre, I am working on deploying my plone site (which is using the Mosaic package) to heroku but it failed to push thousand times. Can you help me out with this?

remote: An internal error occurred due to a bug in either zc.buildout or in a
remote: recipe being used:
remote: Traceback (most recent call last):
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/zc.buildout-2.9.3-py2.7.egg/zc/buildout/buildout.py", line 2123, in main
remote:     getattr(buildout, command)(args)
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/zc.buildout-2.9.3-py2.7.egg/zc/buildout/buildout.py", line 796, in install
remote:     installed_files = self[part]._call(recipe.install)
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/zc.buildout-2.9.3-py2.7.egg/zc/buildout/buildout.py", line 1553, in _call
remote:     return f()
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/plone.recipe.zope2instance-4.2.21-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 113, in install
remote:     installed.extend(self.install_scripts())
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/plone.recipe.zope2instance-4.2.21-py2.7.egg/plone/recipe/zope2instance/__init__.py", line 617, in install_scripts
remote:     requirements, ws = self.egg.working_set(['plone.recipe.zope2instance'])
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/zc.recipe.egg-2.0.3-py2.7.egg/zc/recipe/egg/egg.py", line 84, in working_set
remote:     allow_hosts=self.allow_hosts)
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/zc.buildout-2.9.3-py2.7.egg/zc/buildout/easy_install.py", line 913, in install
remote:     return installer.install(specs, working_set)
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/zc.buildout-2.9.3-py2.7.egg/zc/buildout/easy_install.py", line 665, in install
remote:     for dist in self._get_dist(requirement, ws):
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/zc.buildout-2.9.3-py2.7.egg/zc/buildout/easy_install.py", line 563, in _get_dist
remote:     dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
remote:   File "/tmp/build_b2b61d136cd6a06db2e02cf2d22ed626/eggs/zc.buildout-2.9.3-py2.7.egg/zc/buildout/easy_install.py", line 1699, in _move_to_eggs_dir_and_compile
remote:     [tmp_loc] = glob.glob(os.path.join(tmp_dest, '*'))
remote: ValueError: need more than 0 values to unpack
remote:  !     Push rejected, failed to compile Plone app.
remote:
remote:  !     Push failed
remote: Verifying deploy....
remote:
remote: !	Push rejected to serene-savannah-18616.
remote:

I used the heroku.cfg that is already defined in the plone.app.mosaic package. Am I missing something?
I just do this because I want to get more experience with the Mosaic package :slight_smile:

@nctl144 Yes, it was broken. Please, try again. The actual error was before that stacktrace (new release of RelStorage-named Python package, which was no longer compatible with the current Plone).

Thank you for your help. The error does not show anymore. However, heroku said that this add-on plan is only available to select users. Does it mean that I have to pay to deploy my mosaic site?

remote: -----> Compressing...
remote:        Done: 206.5M
remote: -----> Launching...
remote:  !     That add-on plan is only available to select users.
remote:
remote:  !     Push failed
remote: Verifying deploy....
remote:
remote: !	Push rejected to banana-shortcake-79340.
remote:

Just to try it out should not required paid plan. I removed email sending support. Could you try one more time?

I am sorry @datakurre but it does not work. I think its partially because of me for missing something. Basically I just clone the mosaic package, add necessary add-ons to the eggs and then run the buildout. Other than that, I did not modify anything in the mosaic package before I run the deployment.

remote: Traceback (most recent call last):
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/parts/instance/bin/interpreter", line 294, in <module>
remote:     exec(_val)
remote:   File "<string>", line 1, in <module>
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/Zope2-2.13.24-py2.7.egg/Zope2/__init__.py", line 51, in app
remote:     startup()
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/Zope2-2.13.24-py2.7.egg/Zope2/__init__.py", line 47, in startup
remote:     _startup()
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/Zope2-2.13.24-py2.7.egg/Zope2/App/startup.py", line 81, in startup
remote:     DB = dbtab.getDatabase('/', is_root=1)
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/Zope2-2.13.24-py2.7.egg/Zope2/Startup/datatypes.py", line 287, in getDatabase
remote:     db = factory.open(name, self.databases)
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/Zope2-2.13.24-py2.7.egg/Zope2/Startup/datatypes.py", line 185, in open
remote:     DB = self.createDB(database_name, databases)
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/Zope2-2.13.24-py2.7.egg/Zope2/Startup/datatypes.py", line 182, in createDB
remote:     return ZODBDatabase.open(self, databases)
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/ZODB3-3.10.5-py2.7-linux-x86_64.egg/ZODB/config.py", line 101, in open
remote:     storage = section.storage.open()
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/RelStorage-1.6.3-py2.7.egg/relstorage/config.py", line 33, in open
remote:     return RelStorage(adapter, name=config.name, options=options)
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/RelStorage-1.6.3-py2.7.egg/relstorage/storage.py", line 180, in __init__
remote:     self._adapter.schema.prepare()
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/RelStorage-1.6.3-py2.7.egg/relstorage/adapters/schema.py", line 969, in prepare
remote:     self.connmanager.open_and_call(callback)
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/RelStorage-1.6.3-py2.7.egg/relstorage/adapters/connmanager.py", line 82, in open_and_call
remote:     conn, cursor = self.open()
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/perfmetrics-2.0-py2.7.egg/perfmetrics/__init__.py", line 133, in call_with_metric
remote:     return f(*args, **kw)
remote:   File "/tmp/build_8519b6fe5b07a1fb93566820c9b50321/eggs/RelStorage-1.6.3-py2.7.egg/relstorage/adapters/postgresql.py", line 195, in open
remote:     conn = Psycopg2Connection(dsn)
remote: psycopg2.OperationalError: could not translate host name "PG_HOST" to address: Name or service not known
remote:
remote: -----> Fix paths in zope.conf
remote: -----> Copy results to cache
remote: -----> Copy results to slug
remote: -----> Copy configure_zopeconf.py script to slug
remote:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
remote:                                  Dload  Upload   Total   Spent    Left  Speed
remote:   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--remote: 100  1100  100  1100    0     0  34029      0 --:--:-- --:--:-- --:--:-- 34375
remote:        Done
remote: -----> Discovering process types
remote:        Procfile declares types     -> work
remote:        Default types for buildpack -> web
remote:
remote: -----> Compressing...
remote:        Done: 206.5M
remote: -----> Launching...
remote:  !     That add-on plan is only available to select users.
remote:
remote:  !     Push failed
remote: Verifying deploy...
remote:
remote: !	Push rejected to banana-shortcake-79340.
remote:
To https://git.heroku.com/banana-shortcake-79340.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/banana-shortcake-79340.git'

Other than messing around with the mosaic package on my local, I also work on researching about the tinyMCE editor and how to modify it as a WYSIWYG. The problems in this project idea seems clearer. I will get there soon :slight_smile:

I also just made a simple example at https://nctl144.github.io to demonstrate dragging and dropping html content into the editor. let me know what you think about it :smile:

It just helps me to understand more about how the first step works. The goal is still making the mosaic tiles be able to be dragged and dropped into the editor :slight_smile:

Does this work for you? https://heroku.com/deploy?template=https://github.com/plone/plone.app.mosaic