Efficient serving of frontend assets

Hi all,

we, at der Freitag, have some ideas on how to improve the frontend performance of our assets (tl;dr; hash every file and put far future headers on them) but we want to reach the wider Plone community to double check if the approach we want to take seems good.

What we have right now
we do some basic bundling (just logged out and logged in) with webpack for all our assets, and we inject on the base template the bundles coming from webpack-plone-plugin (thanks a lot for it @datakurre !!).

When we deploy our theme, we deploy it as an egg, and thanks to yarn.build (sorry for the sameless plug :sweat_smile: ) we generate all the assets and ship them with the egg where the diazo theme (rules and so on) are.

As we have nginx/varnish/haproxy on front, they are generally cached good enough.

What we are aiming for

  • split bundles: separate vendor assets in few bundles, so they can be cached forever
  • smaller bundles: so we can deliver them faster and only the needed chunks
  • better caching: use content hashes on the filenames so the filename does not change if the contents do not change

Those above are easy to do with webpack alone, in fact, we have that ready. The problems are on the following ones:

  • free Plone to serve those files and rather serve them on nginx/something else
  • set far future headers on all those assets (doable with some nginx foo)
  • one single asset: due to Virtual Host Monster, on webpack we have to put the publicPath as ++theme++freitag.theme/blala so we can not put a / in front and thus resources will be downloaded on every different path (am I right on that? :thinking:)
  • keep old assets for browsers that have stale responses (as we server our assets from our released egg, that one only contains the freshly new build assets), it is somewhat mitigated by the content hashes and smaller and vendor split bundles, but still, as soon as a new release is made, all requests to the older assets will end up in a 404.

How we would approach it right now
We are thinking of creating a storage on the delivery server (i.e. the server that has nginx/varnish/haproxy) so that we can make a tool that at release time of our theme egg, it builds the assets and rather than keeping the files and shipping them on the theme egg, it uploads them there on that storage.

  • as the assets will only be pointed to through the diazo theme, as long as the assets upload goes before the theme deployment, no broken links would be generated.
  • as the storage is version independent (the problem with releasing the assets on the egg theme) old files would still be reachable
  • as the storage is located on nginx-reachable filesystem and with some foo magic URLs can be rerouted there, we free our Plone instances from serving static assets
  • as nginx is serving them, we can set far future headers to maximize delivery and second view performance

Sounds that sane? How are you approaching it? On another django project we have, we have been mixing nginx+webpack+whitenoise and the results are really good, check it out yourself (sorry for the second shameless plug :sweat_smile: ): https://digital.freitag.de

Thanks for your time!

2 Likes

I'm all in for smaller bundles, all those Plone bundles/resources are way to big right now.
This would also allow to load them only where they are needed.

Also found out, while doing performance tests on a client site, that adding the async attribute to the script tags in the header makes a huge improvement on mobile devices (Google Pagespeed Insights: before 4/100 after adding async 35/100). Johannes and me will be working on adding this option to the registry very soon.

In our opinion the only thing/resource that should not be loaded async is JQuery. Idea is to put JQuery in it's own bundle and let the Plone bundle depend on it.

One more thing I came across is moment.js some while ago (https://github.com/plone/Products.CMFPlone/issues/1779) is that we still load all available locales instead of those configured for the site. moment.js itself is about 35KB but all the locales add another 135KB even minified. Afaik it should be possible now to lazy load only those locales.

Regarding loading resources, I'd like to see more resources loaded on request. Why should one load TinyMCE for example only because being authenticated. Not sure about all of the implications, but having a bundle for form widgets only loaded when the user is actually editing, or even better if every widget would be able to load those resources that are needed.

1 Like

Remembers me of the good old days were we used fanstatic in some Zope 3 projects were a view/widget/resource could simply call resource.need() and it will be included in the correct order with all it's dependencies. And fanstatic would take care of the rest (using a WSGI config). So creating the bundles with webpack and then including them whenever necessary by a component :wink:

from js.jqueryui import jqueryui

class MyView(BrowserView):

    def update(self):
        jqueryui.need()

As we have the option to load resources and bundles per request, like fanstatic does, we only need to reorganize the resourses to use it better.

https://docs.plone.org/adapt-and-extend/theming/resourceregistry.html#controlling-resource-and-bundle-rendering

For example every pattern in patternslib/mockup should load it's resources when it's used. Or at least the big ones like TinyMCE should be refactured to do so. At the widget leval this should be easy by using add_resource_on_request or add_bundle_on_request.

It's true paterns can be used outside widgets too, but than we could either load the neded resources in the view which is using it or implement the loading via javascript.

.

1 Like

thanks for all the replies (although they went into other directions rather than what I was asking for :sweat_smile: )

For the ones wondering about the publicPath question I was having on the original post: Zope/Plone is so awesome that even locally is able to traverse to the right folder even if you have a / at the beginning. So setting absolute URLs on webpack is fine :slight_smile:

If I understood correctly, these should all be doable with webpack as well already.

publicPath can be absolute URL to anywhere, and it can be different for every theme version.

The trick is to deploy the same theme package both for nginx and to Plone. By generating different path (e.g. based on a version control revision) for every version, versions never conflict and old versions can be served forever. As soon as Plone gets the new version, it starts serving HTML with new publicPath.

We have been doing this for a couple of years for now.

Webpack supports "dynamic imports", where additional chunks can be loaded lazily based on some javascript execution on browser – with easy and well documented way. I made this default in the current plonetheme.webpacktemplate to familiarize it better. Of course, it is not always optimal, because e.g. loading CSS lazily could cause flickering.

if (jQuery('.pat-tinymce')) {
  import('mockup-patterns-tinymce');
}
2 Likes

plone.staticresources = 1.1.0 (included in Plone 5.2rc4) has now an experimental profile for async resource loading.

@davilima6 did some benchmarking on a logged-in site: First Contentful Paint from 16.8s down to 8.4s

Feedback wanted! Test it out!
So we can figure out any issues with addons and further improve loading performance.

1 Like