Add-on deployment TTW

Not entirely sure I understand @gyst. Are you suggesting something like distributions could be another way to solve this? ie someone selects, tests and curates all the possible plugins someone could need and the standard installer has them all included already? Isn't the problem there that someone has to choose, and test and curate? and its good to have competition in an ecosystem so there is innovation.

I think other two options you have somehow combined but they are different suggestions

  1. restricted python addons (from @vangheem)
  2. mode to allow fs package install via sitesetup
  3. distribution(s)

Whats interesting is none solve all the use cases.

  • The only one that makes security patching easier is sitesetup fs packages.
  • The only one that allows easily trying out new stuff securely is restricted python addons.
  • Distributions is the only really secure way to give the full use of tools you need to build many kinds of sites (esp given most restrictedpython regularly needs whitelisting of many python libraries which can only be done with a fs package)

So I'm not sure we can dismiss any of these options yet.

+1

I really don't get the point of this whole discussion.

Who is the target audience of this feature or which target audience do you want to win for Plone?

With the shrinking importance of Plone (and other CMS) you will not win a new target audience with such a feature.

Perhaps 10% of our Plone customers of the last 15 years was running Plone itself. In the majority of projects the hosting and Plone administration is done by Plone administators and integrators. I can not remember a case where a Plone customer had the need or the wish to install some add-on by itself. It would be nice having this functionality but technical details are too complex for moving a complex add-on infrastructure to a TTW installation. This is never going to work properly. In the end you are trying to create a two-class ecosystem of add-ons. The TTW installation may work for a small class of add-ons with a limited functionality...the overall set of add-ons providing a broader functionality and a deep integration into Plone will never ever be installable this work...so please focus on the things that matter instead of wasting your limited and shrinking resources into a feature giving us a major advantage...as said: very high risks and hard to solve if not planned carefully in advance (otherwise you need up with the same nightmare as we see it right now with the resource registries).
Happy to discuss this in person @PyCon...

-aj

1 Like

@zopyx while plone is losing market share yes CMS is not. Only if you see a CMS as a way to build an app in which case its losing to frameworks like django. However 50% is cms sites and growing.
One of the reasons plone is heading to less relevancy is because of exactly what you describe. Plone only works if you outsource it to a paid for integrator and their own hosting. In the world of CMS that makes plone expensive and inassesible despite it being a very good CMS which is easy to use and easy to theme.
Given how cloud is now, we can have one button plone sites very easily. Except for two things

  • how do they write custom code given all the training requires them to learn buildout and provides bo guidance on how to get their code from their machine to their host.
  • how do they install plugins.

Ttw plugin install is probably one of the most importabt things we can cocentrate on to help plone and save all our techincal and business investment in plone IMO. It is exactly this old way of thinking about CMS @zopyx that is causing plone to concentrate on the wrong things and lose relevance.

Like @zopyx says: what't the point?

If you want site admins to be able to one-click install anything, just ship a universe.cfg with a gazillion add-ons. What capabilities does a high-risk project of re-engineering existing add-ons as TTW RestrictedPython ZODB installables deliver, that a simple extends += universe.cfg cannot do?

1 Like

Maybe 'most used add-ons' should be in the buildout.cfg (they can be commented out for that matter)

#uncomment the line below to make it possible to add Disqus commenting platform API into Plone ( https://github.com/collective/collective.disqus )
#collective.disqus 

Or maybe even:
#uncomment the add on below to check out all all available add-ons
#collective.everythingthatisavalableforplone5

Re-running buildout after months or years scares anyone and can lead to a non working environment, plus need a restart. Buildout is ok to build a project, not to add add-ons on the fly, I think.

It is not easy to solve this problem with Python in general, is there a cms written in Python with ttw addons based on eggs?

This is an old Plone3 gripe. Plone itself has been properly version pinned since ages, and anybody who runs a custom buildout in production without proper version pinning is a fool.

If restart is really that big deal for you, you can run ZEO on RelStorage which gives you zero-downtime rolling restarts.

@yuri +1 Buildout scares me whenever I run it and I do this everyday for a living. It's not for the faint of heart. I just spent 2 hours today trying to build plone because something changed in homebrew that made pillow and lxml not compile that lead to cryptic buildout errors. Last few weeks we've had many requests on the forums that amount to issues when installing addons.
Having a buildout with all addons, or even a docker image, is just not realistic.

@gyst whats with the stop energy? The usecase is crystal clear. If we want to grow plone we need an easier addon story and an easier hotfix story. Yes it doesn't help my business or your business directly, or any core developer on this list. But CMS's that are relevant are the ones that are widely used. Widely used is because you are able to get up and going yourself and not require integration services or special hosting. No one expects a CMS to be this hard these days. If we don't help people be able to do plone themselves, no one will ask us to help them when they need more help.

Even if I'am more the Plone-as-a-framework user I would appreciate a story, where simple integrations and customization including TTW-templates and code are installable as zip-files by upload.

I know that there are more people out there using Vanilla Plone TTW than it seems. This are not the loud people speaking up or even reading in here. I think @tkimnguyen was it who told us a good story about such people at PLOG last year.

For sure this would be kind of limited to whats possible TTW and and I dont see Python eggs here as the way to go. I'd also not make a difference between theme package and functionality package, because thats only confusing.

Keep in mind to not run unrestricted code or even do filesystem modifications done TTW, then all is possible.

I think all in control-panel including types and theme, simple template customizations and possibly add ons like rapido are perfect candidates for such a feature.

Since Plone is a community driven project with different needs I understand that there are different points of view on this, but this means also: Go on, PLIP it, discuss it as a PLIP!

1 Like

A said there is a huge technical risk here. If you can not provide a technical solution that is working for all add-ons then you will end up in two-class society for add-ons - good add-ons vs. bad add-ons, old add-ons vs. new add-ons. Since I doubt that a TTW installer will work for all and everything you will introduce a new installation method while leaving the old buildout method in place for the "bad" and "old" add-ons....what do you win?

"Old thinking"....perhaps better "realistic thinking"...True, the world is moving to the cloud. Since we all (including to CMS) lost the "CMS war" against Wordpress we will not see a growing market share for Plone in the future...everything else is wishful thinking. There will be a niche for Plone to exist in the future but this is not the niche where "going into the cloud" is a huge argument. Perhaps the prospective in US is different from the one in Europe or Germany in particular. But the large and important consumers of Plone at this time are organizations where moving their data into the cloud in not an option - for the simple reason of privacy.

And my fear is again that such a feature with lots of risks ends up in some major release - with a half-baked or broken implementation as we have seen it in Plone 5.0 (speaking of the toolbar and resource registries).

Again: please focus on the things that matter and that are well-understood..otherwise you are continuing to dig Plone's grave.

Andreas

1 Like

Some brainstorming
To help the fools: could it be possible that when you (first) run buildout, whatever add-on that is not pinned is 'automagically' added to some version.cfg file.
Then, it could maybe be possible to rerun buildout without much risk of messing up too much ?
maybe even 'rerun buildout from Plone itself' with a 'dry run option' ?

I agree with @gyst and @zopyx on this; there is a lot of wishful thinking on some of the arguments presented above by the supporters of this new feature; seems sometimes we don't learn from our past mistakes.

you are presenting reason 530 on why Plone is losing market share and you think this can be magically fixed by installing add-ons TTW; the reality will be different as has been demonstrated by the TTW content type story: is not that easy.

now I'm going to give you some facts about the internet:

yes, the CMS market is probably going to grow in the next years and the reason is obvious: there are around 900.000.000 web sites worldwide and only around 25.000.000 are running under a CMS, that's way far from the 50% mentioned above.

Plone represents around than 0.1% of that market share on a global basis (a debatable figure, indeed) but you know what? that means nothing, because the websites we run are orders of magnitude more important than our grandma's blog.

CMS market worth an estimated $3.5 billion USD in 2015, but ECM worth $24.6 billion USD; so, where do you want to be? can we have an updated roadmap with a clear path based on solutions and not only on technology? can we deliver?

we have to focus our efforts and limited resources first, on fixing the features that are still broken on Plone 5 (toolbar, resource registries…); second, on helping integrators in porting their solutions to Plone 5 (add-ons, content migration…); only then we can start experimenting with new features on a larger scale (in the form of add-ons before integrating them into the core, as we have done with Mosaic) and learn from mistakes of early adopters.

if core developers think that delivering working functionalities is not fun, we, integrators, think that fixing half baked functionalities is neither. all of the above is causing waste of time, resources and money and is delaying adoption of Plone 5.

and no, don't blame Buildout as reason 531 for our own failures: Buildout works perfectly if you know what you are doing; if you don't, ping me on IRC and I will teach you in 10 minutes.

1 Like

You can set your buildout to display unpinned eggs:

[buildout]
show-picked-versions = true

And then run the buildout with the "non-newest" option (keeping current versions):

bin/buildout -N

After that you can easily copy the output of version pins and add them to your versions.cfg and presto you have a fully pinned buildout.

Obviously one of the tricks of not being scared of buildout is running everything on a local dev copy until you've got complete control.

exactly, but better you add the following to your production configuration so you don't need to remember to run in non-newest mode all the time:

[buildout]
newest = false

done! less than 10 minutes indeed.

IMO newest = false and bin/buildout -N are just temporary stopgaps until you've figured out your pins. In production I never allow unpinned eggs.

[buildout]
allow-picked-versions = false
3 Likes

Exactly, I even keep that for development, no bin/buildout should run successfully without all versions pinned. This way, not even during development you will have version jumping around, nor will you get surprised when deploying that a pin is missing.

1 Like

there's always something new to learn; thanks! :slight_smile:

Elsewhere I commented on the nice Odoo add-on "shop"/gallery... I think it would be ok to show and have nice looking front-facing functionality add-ons displayed and installable from the Site Setup Add-ons panel.

One of our keynote speakers in Boston will be Annette Lewis; she's the one who has done some serious wizardry using Plone entirely TTW. She is not alone; that was what we had to do as much as possible at UW Oshkosh simply because there are never enough file system devs to do everything.

https://2016.ploneconf.org/keynotes-and-talks

THE SUPER INTEGRATOR - Annette Lewis, who has created dozens of departmental and group Plone sites at Penn State University, will walk us through the process she uses to go from design comps to finished site in 2 weeks - with everything done through the web.

2 Likes

That seems to be common case for Plone at universities. Also we depend on our power users being able to implement and enhance features TTW. Of course, that often adds technical dept, but usually things work out well enough.

I would have no issues for Plone having two separate add-on stories (TTW add-ons / filesystem extensions) with the infamous Z-shaped learning curve between them. We just need clear brand/terminology separation between them. And I would not need any new TTW features, just a unified way to easily use all the existing features just by uploading a zip file or editing uploaded file TTW similarly to theming.

1 Like