Plone 5 frontend development plugin for webpack

It might be a good time to start a new thread dedicated to using Webpack for theming Plone 5.

After I managed to make all Plone 5 frontend resources to flow through Webpack as in my reference implementation https://github.com/datakurre/plonetheme.webpack, we've deployed about a dozen Plone 5 sites with Webpack built themes. By theming with Webpack, I mean that:

  • All JS and CSS/LESS files otherwise managed with resource registry are compiled into Webpack bundles, usually one for anonymous and other for authenticated users.
  • Site and theme-related UX-features are developed within theme and merged with Plone resources using Webpack.
  • CSS and JS bundles are injected into theme's HTML-mockup files and manifest.cfg (for TinyMCE), automated with webpack-html-plugin.
  • Diazo rules are used to drop the default resource registry -bundles and toggle between anonymous and authenticated bundles.

Pros:

  • modern JS development flow with ES6 and other Webpack-integrated goodies
  • webpack-dev-server's almost instant code-reload.

Cons:

  • generic Webpack learning curve
  • configuring Webpack to properly shim all existing "legacy" JS may not be trivial
  • JS outside patterns cannot be bundled with Webpack and keeping that functional may need extra care.

My first approach with plonetheme.webpack required a confusing mess of npm-packages, git-checkouts and Webpack resolve aliases to find all the JS and CSS resources required by Plone, and that added some extra cost for maintaining those themes. At the Plone Barcelona Sprint in May, Eric Brehault gave me an idea, how it would be possible to fetch all the resources for Webpack directly from Plone. Last week I finally got that to work, resulting in a new Webpack plugin dedicated for building Plone themes.

Welcome plonetheme-webpack-plugin that

  • reads require.js configuration from a live Plone site and makes it available as Webpack resolve alias mapping
  • reads LESS-variables from a live Plone site and makes those available for Webpack LESS-loader
  • plugins Webpack's resolve hooks for fetching all available resources from a live Plone site
  • provides an increasing amount of Webpack loaders and instantiated Webpack plugins to fix all the seen issues when bundling Plone frontend resources with Webpack
  • provides a customizable Webpack configuration with all the above enabled by default.

That's probably more than recommended for a single plugin, but as I'm going to need to maintain a lot of Webpack built Plone 5 themes, I want them to share as much configuration as possible by maintaining that plugin.

I have the following public examples for trying out the plugin:

The provided Webpack configuration is tightly integrated my theme and project structure, but learning it from the plugin code should help in trying with plugin also with other kind of setups.

The plugin package does not yet include all work from plonetheme.webpack (e.g. some non-default bundles may not build yet), but will, once I migrate my existing themes to use it.

10 Likes

That's awesome!

I have a question: in your example, there is a custom folder https://github.com/datakurre/plonetheme.webpackexample/tree/master/resources/src/custom.
Could you explain what it does?

It provides us TTW editable "ploneCustom".

There was some concern in our team that moving to webpack would require too much technical skills for small fixes and we don't have automated deployment for webpack based themes yet.

So, we ship our sites with two themes. The main webpack built theme" and that "custom folder". Rules file in the main theme includes the rules file from that "custom folder". Whenever we need to make quick theme fixes (while adding technical debt), we copy that "custom folder" in theme editor into a ZODB overridden theme and we have TTW editable ploneCustom for Diazo, JS and CSS/LESS. (Actually, we still need some custom adapters to do cache purging when editing theme TTW on a production site.)

It's quite easy to remove, so I decided to leave it into those examples. Agreed that it might be confusing until one actually runs the example.

That's nice.

@datakurre I have turned your example in a very basic mr.bob template https://github.com/ebrehault/plonetheme.webpacktemplate

4 Likes

Thanks. Would it make sense for me to copy that into plonetheme-webpack-plugin and release it also as a Python egg for mr.bob-templates?

I should also make a template for vagrant setup, because that's been useful for designers with NodeJS but not Python experienc.

Would it make sense for me to copy that into plonetheme-webpack-plugin and release it also as a Python egg for mr.bob-templates?

I think it is better to have 2 separated packages, so plonetheme-webpack-plugin is a pure NPM module. It will cleaner and less confusing for non-Plone people who might try to understand how it works.

It might be good to know, that I've now been using that template myself for a some time now, and I try to make it up-to-date when things change/develop.

The latest addition (already merged) was option for SVG content type icons (generated from Fontello).

1 Like

TL; DR; webpack built themes work just fine with Plone 5.1

FYI. The currently available version of plonetheme.webpacktemplate does not fully work on Plone 5.1, but I'll push a working version at least when 5.1 is released.

For a some time ago there was major refactoring of structure pattern in plone/mockup. The new version introduced "dynamic require calls", which need some extra care with webpack. So far, I have only been able to make it work with a filesystem checkout of mockup, so that even webpack gets all other JS, CSS and LESS are bundle directly from Plone, structure pattern must be built from file system.

Luckily, the required mockup checkout can be defined transparently in package.json and override definations required in webpack.config.js only make a nice example, how to override the default Plone-shipped pattern versions with customized file system ones (and even the current plonetheme.webpacktempate does it to fixed version of a single pattern).

So, after all, not being able to webpack-bundle structure pattern directly from Plone 5.1 will but more of a theoretical problem than a real issue.

PS. A nice bonus we are getting from webpack: We can deliver the bundles (CSS, JS, fonts) outside Plone from a separate httpd with proper gzip and http2 support.

@datakurre @ebrehault We are working on a new UI for Plomino built with angular 2. It will be activated when clicking on a plominodatabase object within the site when you are an editor.
Is this method you are using of duplicating and packaging explicitly all the existing plone js/css the best way of being able to use patterns and style server-side forms within an angular 2 SPA?

The current branch we are working on that requires patterns and some plone styles is - https://github.com/plomino/Plomino/tree/advanced_ide/ide

@djay

Of course, re-using Plone default bundles without re-bundling would be nice.

If you can make a such working webpack bundle, which excludes patterns and other JS already loaded in Plone default and logged-in (meta) bundles, please, share. Webpack has some support for "externals", but I don't know, how to make work with requirejs.

Otherwise, my first approach might help. I made required files available for Webpack using git submodules (buildout or npm should work as well) and defined a lot of aliases and shims for webpack https://github.com/datakurre/plonetheme.webpack/blob/master/webpack.globals.js

Nowadays I use the plugin described in this thread (to get all aliases and resources from live Plone instance), but it requires running Plone instance when running webpack.

@datakurre I am still learning about webpack. Do you think any of the techniques outlined below might work?

Just curious, do you think there's value in ditching RequireJS in favour of a webpack driven approach?

@pigeonflight @datakurre if we did we'd need to do it soon before too many start upgrading plugins etc and we'd better be darn sure webpack is not just another stepping stone to something else.

1 Like

But they are not completely comparable.

RequireJS is both a module / dependency definition syntax and a bundler. And RequireJS is (AFAIK) the only bundler, which can do the bundling in browser (and work TTW without NodeJS backend server).

Webpack is only a bundler. Webpack works just fine with RequireJS-style definitions. Currently I even rely on our ResourceRegistry definitions when bundling with Webpack.

And webpack may not work at all for pre-building additive bundles in add-ons. @djay asked about, if and how Webpack could be used to build add-on bundle for Plomino. Yes, webpack could be used to build a bundle, which includes all the required Plone JS and Plomino JS.

But I don't know, if it's even possible to build a bundle, which expect Plone logged-in-bundle to be loaded before it (and not duplicate anything in it). And it probably breaks if one tries to include two independently built webpack bundles on the same page.

I prefer Webpack, because it helps me to get work done, but does add an another complex buildout-like tool to learn.

1 Like

well said! that' s exactly what we have to start discussing.

@datakurre IMO, we're trying to solve the wrong problem: resource bundling is a thing of the past as HTTP/2 is multiplexed and makes it less desirable, so TTW bundling makes absolutely no sense neither.

I returned from Boston with the idea that JS is finally maturing and things are not going to change so much in the future.

if you were to ask me, I'll vote to remove all the complexity on our current implementation of bundling as it has been proven buggy and not future proof; I'll love to read what others think.

so, what we need? I'm not pretty sure of the whole picture but, IMO, in Plone we should be using RequireJS only and a module bundler like webpack for core development,(defined by the framework team after some discussion.

add-on developers could use the module bundler of their choice on their projects, and best practices must define what Plone is expecting from the resources of an add-on.

1 Like

I still image, even with http2 spec, we'll want bundling. Considering modern framework can build hundreds of files together--even if the client is able to maintain one connection, it'll be many more hits on the server and I also wonder if the client will be less performant.

Also, FWIW, I did an attempt to package all of plone's resources with webpack automatically by reading the registry dynamically: https://github.com/castlecms/castle.cms/blob/master/castle/cms/_scripts/generate-webpack.py

I wasn't impressed with the results. I could never get the LESS/CSS to work completely correctly and it was difficult to control how you manage the bundled files--wanted to split into chunks that it seemed difficult to control. In the end, I didn't think it was worth it. I still use webpack for other development(angular/react) though.

My webpack plugin does read RequireJS configuration and LESS configuration from Plone (config.js and less-variables.js), and I do have LESS/CSS working completely correctly (as far as I know). But I do define bundles manually. I use webpack-level bundles (common chunk, default, logged-in and add-on-specific bundles), include them all as separate link/script-tags in theme/site-layout, and enable/disable them with Diazo.

The most difficult part has been shimming. I decided to not trust shim definitions in Plone registry, but manage shims for core patterns in the plugin using webpack loaders and plugins. For example, making dynamic requires in structure pattern to work with webpack was not trivial. For add-ons, I mostly just use webpack's inline loader syntax (in require/import lines of JS files defining bundles).

Some reasons why I chose and keep using webpack with Plone 5:

  • Managing and optimizing final bundles with Webpack
  • Webpack devevelopment server with auto-reload and hot module reload
  • Easy pattern overrides (by overriding RequireJS alias path)
  • ES6 transpilation with Babel
  • Using both LESS and SASS (Plone is LESS, all the rest SASS)
  • Inlining icons as data-uris in CSS
  • Distributing final bundles outside Plone (we have several sites sharing the same theme)

I'm still on Webpack 1.x. I will only start porting my plugin to Webpack 2.x once it has a final release and all plugins, which I'm depending on, have been updated.

@datakurre I suspect I'm not going to be able to take your webpacktemplate and plugin and integrate them into this angular 2 plugin (plomino). I suspect your plugin is using react, and making assumptions about this being for a theme rather than a plugin. But given I don't know angular, react or even really plones bundling system I think I'm really out of my depth.

Here is my attempt to merge it so far https://github.com/plomino/Plomino/commit/b33516eb350b28bf5a097b5135237daf52192966.

Any help @ebrehault @datakurre or anyone who knows angular 2 or webpack, it would be greatly appreciated.

Lines about React are just examples for overriding requirejs aliases.

Plone Foundation Code of Conduct