Notes from a drupal meetup

1

Hi,

Went to a local drupal meetup last night.

Here's some brief notes

BigPipe - thry said a drupal version soon will support bigpipe. https://m.facebook.com/notes/facebook-engineering/bigpipe-pipelining-web-pages-for-high-performance/389414033919/ . This looks like an interesting idea that could perhaps become part of mosaic? @datakurre? Faster page loading might be a good reason accept less flexibility with theming that a grid system will likely impose.

A talk was about their migrator. Its really not that much different from transmogrifier. Except their plugins can use other plugins as argumetns which is nice. uses yaml. Drupal are building a ui for theirs.
And they can connect the old and new db at the same time and do the conversion in one process
But then they apparently have no other way to do an upgrade. They don't do inplace upgrades. The idea of not having to code to transform your old custom content types into your new ones is not a bad idea.

They mentioned how much they said drupal was point and click. Less coding. I didn't realise that. I heard it was more like plone. More complex to integrate with than wordpress. They mentioned things like views which let you build up sql queries interactively. They mentioned the lots of clicking thing as bad thing but thats to be expected by professional programmers. Fact is it could be one thing getting people to start with drupal.

They did mention that drupal 8 has a lot more to understand to integrate with it. Seems like they have gone for a similar route of wanting to make their stack more robust under the hood so its easier to maintain, at the expense of complexity for integrators. One guy mentioned he never had to use an IDE before. Just text editors. Now PHP storm is a must to help him.

Another talk was about integrating with 3rd party services. One was imgIX which is a CDN doing scaling and image transformations. Sounds like a nice thing have transparently work with plones image scaling.

There was also a lot of talk of specialised drupal hosters esp the ones providing a heroku code only deploy type service. Another thing thats nice about a big ecosystem that can support this.

2

@djay Is the bigpipe significantly different from composing blocks+tiles on browser using javascript? (Or with ESI to avoid JS requirement.)

Transmogrifier can be used to migrate contenttypes, but as usual with Plone, it's not trivial to use OOTB.

3

@datakurre I'm no expert, only what I read about in that article. It seems the idea is different than ESI. Instead of caching blocks in a CDN/proxy, it all comes from the app server but you stream the html. You can send the easy to render blocks first (static), have them appear earlier on the screen, along with dummy place holder content and then fill in the more processer intensive blocks as they reach the browser. It's supposed to work without JS too (ie progressive enhancement).
The general idea is to speed up page loads by not waiting until you have all queries, all backend rendering, transformations etc done before you start rendering the page to the user.

4

@djay Thanks. Streaming all blocks in the same request is definitely different from ESI / JavaScript -composition. So, it would require some kind of absolute position CSS to position the late blocks correctly over the early placeholder (and, of course, JavaScript could simply replace the placeholder without suppport form CSS)?

If we'd really like to, we could do a such streaming with medusa (it's possible to stream multiple ZPublisher responses into single medusa response stream).

Yet, it would probably be easier introduce configurable per-tile rendering strategies for Mosaic: A control panel, where admin could decide, which tiles should be rendred into main response, and which should be composed using ESI or (still missing) JS composer. Mosaic already wraps tiles into grid layout, so estimating grid cell sizes to avoid flickering should be possible (e.g. editor could annotate the cell with the last known contents size).

5

@datakurre It doesn't use absolute positioned css. Have a read of the post. They stream a series of script tags that contain json that contain html that gets inserted into the placeholder that came in the first part of the request.
I think it would good to keep in the back of our minds, so we don't do anything to preclude it.
Not having thought about it much though, some things standout as hard to solve

  1. With no parallelism on the backend it would be hard to determine which is the cheapest tile to render first.
  2. Diazo and browser layers kind of rely on the idea you have the whole DOM to work on and you grab something from one side of the page and put it on the other. It might be possible to be many transforms, one for each tile, but how does the framework know when your diazo is waiting on html that hasn't been generated yet?
  3. looking closer at the description I can't see how it can be progressive enhancement. Without js load the "pagelets" the page would be mainly empty. Maybe the page knows ahead of the bigpipe js has already been cached in the browser before it starts its response?
    For our purposes (gov) progressive enhancement is still a requirement.
6

Hi!
I can talk a little about Drupal.
5 years ago, my company have 14 clients that use Plone, in intranet and/or websites.
Now we only have 3, 2 on Plone 4.16, 1 still on Plone 3.3.
Not one of our Plone costumers wants to upgrade to Plone 5!!
Most of our old Plone clients we moved them to Drupal, and they are happy with that move.

BigPipe will be in D8.1 that will have RC1 next week.

Migrator is mostly to help people to go from Drupal 7 to 8, but it will be a generic tool. Drupal upgrades is still one of their weeknesses.

In my opinion, it is true, Drupal is too much "point and click".
MUCH more complex to integrate than Wordpress and Joomla. Depending on the customization, almost as complex as Plone.
Views are great, but complex to master properly.
So, very easy to start, but if you need to go deep, bah!
There really is a need to be able to do the same via file system coding.

Regarding core code, Drupal as gone the opposite direction from Plone.
Drupal core is very optimised, polished and nice to look, Plone's core is a mess, lots of bugs and worse, bad code.

So, right now, overall, in my opinion, Drupal is the best open source CMS solution out there.
Plone still does some things better, but the true is that Plone 5 is full of bugs and regressions.
The diference, I think, is in the core developers...
In Plone, they tend to implement what pleases them, not what Plone really needs.
With Drupal, the core devs listen to what the community say and only implements new features when they really are usefull for the majority.
Rest of the time, they just polish the code.
If Plone core devs could do the same...
(but it seems that Plone's core has become unmaintenable... at least this is what people I heard tend to think!)

7

Considering that's April 1st, it is really hard to tell if this message is serious or not :slight_smile:

1 Like
8

@djay Sorry. I just guessed the absolute positioning part, because from the docs, I couldn't figure out any other way it to work without JS. Of course, that approach could just fallback to legacy rendering, if JS is not present (and that's detected somehow beforehand).

It's good point that diazo with plone.app.theming would be incompatible with this approach. We could have diazo-transforms for individual tiles, but it would only fix the issue of adapting their markup into theme's markup: you could no longer reorganize page with diazo.

About parallellism. AFAIK, importance of ZODB connection caches and reliance on threadlocals make real parallelism practically impossible for Plone. I've played with an idea of a blocks transform, which would use the whole ZEO cluster for fetching tiles in parallel for tiles transform, but I haven't got far enought to benchmark that.

9

I doubt that it was meant as a joke.

But i think it is unfair to compare us to Drupal.
Their association managed to spend 5 million dollars last year. They have several developers working in the code base full time.
https://assoc.drupal.org/blog/holly.ross.drupal/2014-drupal-association-annual-report-financials

Also, complaining is cheap, I do it a lot too, but most of the time I actually manage to help improve the situation.

Only complaining about the work of volunteers is a great way to not make friends and kill the volunteers willingness to spend their spare time to improve things.

3 Likes
10

Hi Eric!

Unfortunatly it is not a joke, it is the way I see it after testing Plone 5 in the last few weeks.

Reading many of the old posts here at the community forum, it amazes me how some of the core devs try real hard to not see the reality with Plone, and to not listen to others that have different opinions!

I have been pointing some regressions from Plone 4.1, but it just feel like no one really cares about.
Most of the regressions are minor, but others are important, like this

and this

And, even if it is unfair to compare Drupal to Plone in terms of resources (and the people from Drupal worked really hard in the past and deserve what they enjoy now), you can compare the attitude in general, and that means a lot.

11

This is really off-topic for what Dylan was trying to do.

Trust me, core devs and the companies contributing to Plone care A LOT about the product and try to advance Plone in ways that solve problems. Core devs are solving their problems and serving their own clients and don't always have the time to get solutions for every other thing people want in the product. In the end, people who are willing to do the work, will serve their own client's needs before those who aren't.

Just because you didn't get enough help on your 2 problems, doesn't mean no one cares, it just means no one has had the time to help you.

2 Likes
12

You realy don't get the point, don't you?
-> those two problems, in the first place, are not my problems, they are Plone problems...!

The fact you and others can not see the difference probably is the reason why things are in this state.

I don't care about "my 2 problems", my company does not use Plone anymore, I do not use Plone anymore, I just been testing Plone 5 to see what's new, and re-learn a little of Python.
But, if not for the old days, yes, I do care about Plone, and it makes me sad to see the bad quality of the code.

13

I was reading about BigPipe a couple of weeks ago and I took some time analysing it.

if I understood correctly what they do is split the page in fragments (called pagelets) and render in paralel those pagelets using JavaScript in the font end. this seems pretty interesting for them, because almost all users visiting Facebook are logged in; also, that's why Facebook doesn't load if you don't have JavaScript enabled

in our case that only makes sense for huge intranets and is something that would take a lot of time and effort to implement because a complete redesign of the rendering process would be needed (probably not even in Mosaic, but before it).

we care of speed mostly for anonymous users as they represent the vast majority of users in the sites we maintain; we take care of things like PageSpeed, YSlow and, over all of them: caching.

I don't think Plone could benefit in the short term for something like BigPipe.

14

you're wrong: Plone problems are your problems also; Plone is free, open source software; you're not paying anything for it and all that we expect is something back and not just complains.

why do we have to fix those problems for you? are going to pay us for that? or do you want to make a life at the expense of us?

can we have a constructive discussion instead of this kind of rant? I agree with you in many points and I think people can lost the interesting parts of what you're saying if you continue like this.

open a new thread and list all the features that you think Plone needs; don't forget to explain why Plone needs them.

3 Likes
15

@hvelarde: I think its useful for things other than logged in users if there is something dynamic on the page. If you are caching the whole page then perhaps its of less use. i was curious what drupals strategy is in including it. It did seem like drupal was getting used for more "apps' than plone is, because things like django and pyramid exist for that.

16

@jotaMG. Yes drupals community is bigger, they have huge VC backing and they have been smart with marketting so they now enjoy a lot of resources because of it. Plone does a huge amount with very few resources. Yes its regrettable that not everything is perfect in plone 5. It will get fixed.

What would be more interesting and constructive is to know what your clients prefer about drupal. What ideas can we take to improve the next iteration of the plone UI.

2 Likes
17

@JotaMG - as I already said in a different reply: Contributions welcome, and even bug reports are a valuable contribution.
Plone is Free and OpenSource Software - free as free-in-speech - not free as free-beer. So you are not a consumer of a product, but a participant of a community. You may also pay for Plone too, by hiring some developer/company to solve your problems and so you get into the consumer role - if you prefer this.
To be contributor you just need a GitHub account (which you rejected to create) and an open mind to start. Here at community.plone.org is the wrong place for bug reports. They got lost and wont get much attention.
For more on contributing to core we have a whole section at http://docs.plone.org/develop/index.html

1 Like
18

@djay So, the main problem for us in BigPipe is dependency on JS?

In the near future we need to do something for Blocks+Tiles default composition. Unfortunately, anything else but a big fat request would remove many of the benefits from Diazo in plone.app.theming. On the other hand, all Blocks' would introduce some alternatives in terms of site layouts and panels.

The current default composition is done using series of series of blocking subrequests just before plone.app.theming transform. It cannot be the final solution, because for default Plone content it's slower than rendering everything in a single TAL template (the current default).

@vangheem has already tried composition with direct ZCA calls, which is faster than subrequests, but not as generic.

I'd like to try simplify and optimize ESI-support and make it TTW/registry-configurable, and maybe add a built-in JS composition option (we did have "deferred portlets" already in early KSS times).

Another alternative could be BigPipe-style streaming, but it would be Medusa-specific solution, require JS and probably require some cooperation from the frontend-cache in use (to support streaming instead of serving complete page at once).

19

@datakurre in theory with middleware or some smart load balancer we could do all those subrequests as real requests. Ie first request rrturns sturcture to middleware and then the rest of the tiles are requested. Then you would have parallelism. Again u lose the ability to diazo between tiles (perhaps only within them) but for certain sites it would speed up page requests because the slow tiles (like those requiring db loads) would come out last. Esp if your load balancer was smart about which instances handled which tile for which user so the caches were efficient.
It would be interesting to try. I do agree with @hvelarde that for most contentish type sites there won't be much advantage.

20

Yes you are right, and I apologise for messing with this community site.
(I may have the excuse that the problems I've reported, for me they are not bugs, they are something worse, much worse)

Anyway my testing with Plone ends this weekend, maybe will give another try when Plone 5.1 is released, to see if some of the problems had been solved or not.