Mosaic, Persistent and Transient Tiles and saving layouts as templates

1

I've been testing, stumbling and probing my way to mosaic the last few weeks and there's something I don't really get.

There are different types of tiles programmatically, transient and persistent tiles. Persistent tiles store their data on the context in an annotation, transient tiles need to get their data passed in from the request, or they pull variable data from the context fields, the plone.app.standardtiles viewlets etc.

There are quite some extra tiles now available in the community (see List of tested and production-ready plone.app.mosaic tiles for Plone 5.1?), but these are all persistent tiles.

But when I customise a layout in the Mosaic editor on a context and want to save it for re-use as a content layout on other conten items, persistent tiles totally break. If I look in the ZMI in plone.resources in what actually gets saved, its the 'html contents' of the whole tile urlencoeded and added as <div data-tile="path-to-tile/uuid?content=urlencoded-tile-div />

the uuid is not present on the different context where I want to activate that persistent tile and field information etc is not encoded either.

I have found references to an 'X-Tile-Persistent=Yes" setting I can pass in the query for a transient tile which suggests that a transient tiles' data is then persisted. Is this the way how I could store a reference to a persistent tile in a content-layout so that end users can apply the content layout and then fill int/change the data of the persistent tile?

Or is it simply not possible to create any contentlayout with persistent tiles already present?

Seems like bit pitty because the interesting stuff is only possible in Persistent Tiles. I wouldn't even want to pass in custom data, if it already would use the Defaults from the Schema that would be a start. It would also be fine if a developer/themer had to write the contentlayout by hand, but I haven't found any examples.

There's a technical discusion on persistence and the history in a ticket in the plone.app.mosaic issue tracker at https://github.com/plone/plone.app.mosaic/issues/334 , but I don't really get it. I just want to use/integratie plone.app.mosaic as is at the moment and know what is and what isn't possible in the default distribution available to the community.

2

Mosaic only use transient tiles and that was a decision made at some point of the development (one that I opposed vehemently and uselessly at the time, BTW).

UPDATE: I removed the reference since you were already pointing to it.

3

Ok, I've created another tile and not made it inherit from PersitentTile but just Tile. But when I save this tile to a layout it's still created as a persistent tile with a random ID and the literal html content is added to the querystring, there's nothing encoded by fields

The plone.tiles documentation writes about transient tile data: plone.tiles · PyPI

When saved into a layout, the tile link would now look like:

  href="./@@sample.tile/tile1?title=My+title&count%3Along=5&cssClass=foo" />

And there's test documentation on plone.tiles.data.encode and IAbsoluteURL adapters etc.etc. but none of this seems to happen in plone.app.mosaic 2.0 .

Also I cannot find any examples of TransientTiles.

@datakurre as you seem to be one of the most knowledgable people around here using p.a.mosaic : are there any examples of non persistentTiles that use the querystring mechanism to save Schema's back into a layout or on the context customLayout itself without reverting to Annotations. Should I add something extra to my custom tiles?

4

may I ask what's your use case?

5

TL; DR; PersistentTile was meant for tiles, whose data cannot be serialized into querystring. That makes PersistentTile incompatible concept with "re-usable content layout". Mosaic evolved to save transient tile data persistently to allow centrally managed content layout with customizable tile configurations (save their configuration into query string in layout, but persistent data on content), not the other way around. Technically PersistentTiles (even with serializable data) cannot be serialized into re-usable layout, because they don't provide the header Mosaic uses to read the parametrized url from.

Your use case sounds like the exact issue why Mosaic floated away from the separation of Persistent and Transient tiles we inherited from Deco.

Early Mosaic still worked exactly like Deco. Transient tiles were persisted only as query parameterized tile URLs in customized layouts. Persistent tiles were persisted as annotations of the object (with uuid-based tile id as annotation key) and had no query parameters on customized layouts.

Then Nathan started polishing Mosaic to work as a base for Castle CMS. Before that Deco or Mosaic had no concept of shared layouts with customizable tiles. Shared layouts could only have static tile configuration. Customizing tile configurations always required fully customizing also the layout – making centrally updating layouts later practically impossible.

Nathan's solution was to keep using "transient tiles" (tiles inherited just from Tile, not PersistentTile), but still detach their configuration from layout by saving them similarly to PersistentTiles. When customized layout is saved to be "re-usable", those tiles (because they are still transient tiles) are serialized into layout. When layout is applied to a content, tile configurations are saved into annotations (or existing annotations with matching tile ids are used).

So, AFAIK, in CastleCMS there are only "transient tiles", whose data is saved as PersistentTile annotations.

In Mosaic 2 "transient tiles" are no longer persisted as PersistentTile annotations, but into an another layout attribute as "faux layout". So, in Mosaic 2.0 another attribute contains the tile layout (or reference to centrally managed layout) and another attribute contains configuration data for those tiles. Mosaic 2 approach has better possibilities for versioning, multilingual and import/export than Castle CMS, but the current editor is slower with it, because every tile saves into same attribute (and saves must be done in serial to avoid conflicts).

My intention was to write optimization transform for Mosaic 2 that "flattens" those two attributes (layout and tile configuration) before rendering to make it behave similarly to Deco (and allow "transient" tiles to render as intended without need for data storage lookup). That's still not done.

2 Likes
6

@hvelarde The use case is copying/applying mosaic layouts between content items between multiple sites: crating a mosaic layout, saving it using the save feature as a user or global content layout and then applying the saved layout to another page.

I know a workaround here is (as with collective.cover) to copy/paste the whole content item, but that only works in the same Plone site. Any tile on your layout that has a schema for local configuration data will not work correctly.

7

@datakurre Thank you very much for the detailed history and workings. I think I'm sort of understanding now where all the overlapping and partly conflicting information is coming from, mostly historically. I'll have to reread and try to dive into the code to understand your last explanation about transient tiles being saved as 'faux' layout on another attribute, but I'll get to understanding it. :slight_smile:

Once I get things cleared up I think we should really update the plone.app.mosaic readme with some "current state, do's and dont's"

With the details in README's on transient and persistent tiles this was what I expected to be happening, that as long as you don't use 'complex' types in your schema a transient tile would, upon saving a local or shared layout, serialize the schema in the layout's tile html structure instead of saving it as an annotation.

I've been 'integrator' experimenting more with Tiles and combined with your feedback:

  • It makes not much sense at the moment if you need local tile configuration data to not use PersistentTile as the tiles data is persisted in an annotation anyway.

  • If you don't use PersistentTile, complex schema fields like relationChoice for a related items widget will throw an error by the transient tile as it still tries to serialise the data and throw a jsonencode error. If you switch such a tile to PersistentTile, all is fine.

I've created a simple Link Tile (Persistent Tile) with a title, description and link field using plone.ap.relationfield RelationChoice. Say that I want to create a shared content layout so that edtiors can apply it to a new content item in the site (my use case), most of it seems to be already working:

         <div data-tile="./@@mysite.tiles.link/linktile1?link_external=https%3A%2F%2Fgoogle.com&description=%3Cp%3EMy+description%3C%2Fp%3E&title=%3Ch5%3Emytittle%3C%2Fh5%3E&X-Tile-Persistent=1"></div>

empty_first_rendering_link_tile
empty_first_rendering_link_tile771×146 11.9 KB

  • If I put this in the content layout template, and apply it to a content item as layout a tile is present, but the first file rendering is completely empty. But when edit it, my querystring fields are there, filled in the form! And upon saving they are visible in the rerendered tile and persisted on save.

  • If I leave out the urlparams alltogether but keep the (non exiting) tile id in the data-tile attibute, the schema defaults are used.

  • The linktile1 ID in the data-tile attribute is necessary. If I leave it out the link tile is also empty upon first rendering, but editing throws a javascript error that the tile data cannot be loaded.

When I drop a fresh link tile from the Insert menu it contains a greyed out already filled in 'default' tile and after dropping this is already rendered without any config done.

insert_new_link_tile
insert_new_link_tile622×582 54.1 KB

If that 'first rendering' could happen with 'new' schema tiles showing up after selecting a new shared content layout in the mosaic editor.... there's no reason not to use persistent tiles in mosaic 2.0. Only caveat is that you have to generate a querystring by hand at the moment if you want to override Tile schema defaults and you'll have to provide a 'fixed' ID in the contentlayout. But that fixed ID isn't too bad because it will be unique per context. And if you reapply the same shared layout the previous 'config' data will reeapper.

8

Try @@tile_layout_view with Modify portal content permission. It is an introspection view to render only the 'faux layout' (all saved transient tile configurations). Because changing layout does not remove tile configurations (that's by purpose to support layouts with matching tile ids to just reorder known tiles without removing their configurations), that may contain uncollected garbage.

It has difference on ZODB connection client cache. Tile data is persisted as part of the context object. PersistentTile counts as one extra object per tile.

Agree on that. JSON-serialization part was a mistake (it looks a bit better that querystring serialization, but does not really add any value). It was supposed to ease transition to newer editor that was developed during the same sprint, but that didn't happen. Yet, if it was not JSON-serialization error, it would have been querystring-serialization error. That said, it will help to implement REST API expansion for Mosaic tile configuration...

I have only used (serializable) UUID values for relations in tiles. Relation values, in theory, could allow backlinks knowledge and prevent link breakages, but I doubt it currently happens, because acquisition chain is broken for them (I don't really know what is indexed into relation catalog for relations in tiles).

That sounds wrong. It should always use query parameters unless matching keys are found from annotation. That said, X-Tile-Persistent should always be unnecessary for PersistentTiles. As told previously, it was made for Castle CMS to make transient tiles use persistent tile data storage.

Yes. The part after tile type name is tile id, which is used as part of the key when saving the tile data. If you had multiple content layouts with matching tile ids, you could switch between layouts similarly to switching between views (all those layouts would use the same tile configuration for matching tile ids).

Possibly so :slight_smile: Maybe persistent tile experience could be enhanced.

Btw, I wonder, how I recall @hvelarde warning us about embracing the use of persistent tiles, not the opposite. Maybe that was when versioning of blobs in persistent tiles caused issues... Anyway, I've been mostly using TTW regular tiles provided by collective.themefragments.

9

so we ended persisting transient tiles; let me tell you that was a bad idea because we ended with 2 persistent tiles that use different methods to persist.

I know that @vangheem, was at the beginning, pushing to use persistent tiles for Castle CMS, but at the end it was decided to go with transient tiles; I don't know why, but that, IMO, was a conceptual mistake.

I was no warning about using persistent tiles, but about an ugly issue that happened when using versioning with persistent tiles: Versioning creates zillions of empty blob files.

IMO, you must use persistent tiles if you have configuration information that may change, and transient for simpler tiles. but now is probably too late for that.

better to simplify and remove the concept of transient tiles.

10

Why not the other way around? Managing persistence tiles cause a quite a lot overhead everywhere. If we only had transient tiles, everything could be saved at once in the editor. Now every tile must be saved separatel. And we need plone.app.drafts (or working copy) to prevent editor from updating production tiles too early.

Persistent tiles were invented for persisting images and files, and even that failed, because we don’t have resources to maintain all image and file management related code for both field blobs and tile annotation blobs (not to mention missing versioning).

11

it's conceptual: transient means transitory, something that doesn't last long. both tiles are now persistent; what I'm proposing is removing the concept, what you're proposing is deprecating one way of persist.

plone.app.drafts is not needed; we don't use it, we use versioning with plone.app.iterate.

I don't agree neither that persistent tiles have failed, we have been using them successfully for over 5 years.

I agree that versioning is a problem, but I think pushing versioning and all other features (like multi-language support) into tiles is also conceptually wrong. and is not because it can't be done, but because is complex to implement and maintain.

in other words, it add a lot of entropy and we don't have enough resources to keep it working.

12

Yes, actually, I would like to turn clock back to time when tiles were just views that received query parameters and did not have state of their own, but configuration was persisted within the HTML content (layout). But that only makes sense, if I could refactor the editor to take benefit of that. And obviously the decisions are up to those, who can invest into Mosaic now :slight_smile: (Actually, I recall that when tiles are not given id in, they still behave like the original transient tiles.)

1 Like
13

To me it persistent was always a bad idea. Easier to keep things closer to richtextfields. markup, layout and things that are combined with that get inserting into a single field. Currently mosaic with transient fields is close that (transient is a bad name. they are embedded. sort-codes) except that you can't have multiple mosaic fields on a single content item. Keeping images and files as seperate objects is ok. This isn't google docs.

This is except for the plan to replace portlets with sitelayouts. That makes things more complicated. But the solution @datakurre outlined where two fields are used to store the data seems ok (but the virtual layout/local overrides could be replace with json I guess). But one thing I'm not sure of is how to replace the acquired portlets concept which I don't believe sitelayouts do. I haven't used it yet but I think sitelayouts are acquired so it means either you make a local change to a tile thats not acquired or you have a whole new sitelayout which is then acquired right? Also there isn't a concept or columns right? You can't add a new tile at a certain level, again you have to have a new sitelayout?

14

do you mind to elaborate?

15

Persistent tiles via annotations are a bad idea because its effectively like creating hidden fields that are hard to get to per instance. If its configuration then store it as json or urls or xml. Allow it to be cut and pasted if needed between pages. Allow sitelayouts to be edited by hand. If it's images or files then have them as seperate objects that can be workflowed, imported and exported. Annotations just make things more complicated.

16

Versus requirements demands like:

Is the future of Plone headless-only?

17

no.

18

I have to agree with you on this: worst bugs we have found in collective.cover are related with annotations and versioning.

how can we solve that?

19

We store files and images as seperate objects same as we do now for regular pages and store references in tiles. To make it nicer we make everything folderish so images can be contained in pages if desired and change how "add new" works so images can be contained while adding the page (not sure if that works now or not). if sites want to use a single media folder they can using restrictions.
Its a model that work well. We don't need to fix what isn't broken.

20

I understand, but that seems more like a workaround than a solution.

we still need to think about this and that probably involves a CMFEditions modifier.