Every plone site I have keeps shutting down for similar reason

Plone Support
1

I have 4 plone sites and for the last several months they've consistently shutdown with a instance log error very similar to below.

Details: Plone 5.0.6rc1
In each case in refers to the Barcelona theme. (I am using the standard Plone theme for each; however, I have modified by deleted the Hero message. I've hidden the footer Portlets. As best I know I've done the same or very similar to each site.

Any help will be appreciated.

here is the message in the log:

2016-12-09T03:24:48 ERROR Zope.SiteErrorLog 1481253888.390.984340481284 http://www.lwhsboosters.org/robots.txt
Traceback (innermost last):
Module ZPublisher.Publish, line 138, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 48, in call_object
Module Products.CMFPlone.browser.robots, line 24, in call
Module plone.memoize.view, line 67, in memogetter
Module plone.app.layout.globals.portal, line 48, in portal_url
AttributeError: 'NoneType' object has no attribute 'absolute_url'

2016-12-09T21:09:55 INFO ZServer HTTP server started at Fri Dec 9 21:09:55 2016
Hostname: 0.0.0.0
Port: 19977

2016-12-09T21:10:02 INFO Plone OpenID system packages not installed, OpenID support not available

2016-12-09T21:10:04 INFO Zope Ready to handle requests

2016-12-09T21:10:05 INFO plone.app.theming.policy local cache invalidated on thread 140380522305280

2016-12-09T21:10:05 INFO plone.app.theming.policy initializing local cache on thread 140380522305280 for http://lwhsboosters.org::barceloneta

2016-12-09T21:11:00 INFO plone.app.theming.policy local cache invalidated on thread 140380513912576

2016-12-09T21:11:00 INFO plone.app.theming.policy initializing local cache on thread 140380513912576 for http://lwhsboosters.org::barceloneta

2016-12-09T21:11:03 INFO plone.protect auto rotating keyring _forms

2016-12-09T21:11:03 INFO plone.protect auto rotating keyring _anon

2016-12-09T21:11:03 INFO plone.protect auto rotating keyring _forms

2016-12-09T21:11:03 INFO plone.protect auto rotating keyring _anon

2

There isn't enough info to go on here...

3

hi, I posted the entire log here: https://1drv.ms/u/s!AkWPxBUibU6ZxnA04L_teM5wy3nD

4

are you using Ubuntu 16.04? if so, read this and upgrade to Ubuntu 16.04.1.

5

First, thanks for the suggestions.

  1. the part for this site shutdown that seems to be the issue is:

2016-12-03T21:32:58 INFO ZServer HTTP server started at Sat Dec 3 21:32:58 2016
Hostname: 0.0.0.0
Port: 20210

2016-12-03T21:33:02 INFO Plone OpenID system packages not installed, OpenID support not available

2016-12-03T21:33:04 INFO Zope Ready to handle requests

2016-12-03T21:33:05 INFO plone.app.theming.policy local cache invalidated on thread 140416490993408

2016-12-03T21:33:05 INFO plone.app.theming.policy initializing local cache on thread 140416490993408 for https://www.docentims.com::barceloneta

2016-12-03T21:33:05 INFO plone.app.theming.policy local cache invalidated on thread 140416482600704

2016-12-03T21:33:05 INFO plone.app.theming.policy initializing local cache on thread 140416482600704 for https://www.docentims.com::barceloneta

2016-12-03T21:33:21 INFO SignalHandler Caught signal SIGTERM

2016-12-03T21:33:21 INFO Z2 Shutting down fast

2016-12-03T21:33:21 INFO ZServer closing HTTP to new connections

2016-12-09T21:50:15 INFO ZServer HTTP server started at Fri Dec 9 21:50:15 2016
Hostname: 0.0.0.0
Port: 20210

after I restart, it quits for the same reason and references the Barcelona theme.

  1. another one of my sites shutdown and appears to be a mosaic issue. I reported one bug, I report this and see if he agrees it is a bug. I removed all the tiles from this test site and uninstalled mosaic. see what happens.

  2. the third site has the same issues as (1) above.

I've never seen this and have no idea what is going on.

I've seen the suggestions about what I am hosting it on; however, that hasn't changed and prior to these shutdowns (been happening for months, just never saw a common cause). Also, it isn't a memory issue.

Any suggestions? I can make the full instance logs available for all three sites.

I wait and see what happens with the one site were the issue seems to be mosaic related and see if I still get the same error messages.

6

something is restarting Zope from the outside… maybe the cronjob you created on the other thread :wink:

check your other logs.

7

I have two cron jobs running, neither of which restarts zope, both are related to a product comingsoon

*/10 * * * * /usr/bin/curl http://admin:boosters4!All@lwhsboosters.webfactional.com/www/@@comingsoon-notify
0 10 * * * /Usr/bin/curl http://admin:267622!Wa@lwhsboosters.org/@@comingsoon-notify

When I see the SIGTERM, this is a signal to terminate. So something is terminating something else (I assume Plone since it stops, but the INFO doesn't explicitly state what is terminated. This is all way above my pay grade.

Either way, I have another thread going for the proper format for a cronjob restart.

thanks for you thoughts

8

@hvelarde OK, you are right that something is restarting, I found the following throughout my logs.

2016-12-07T06:36:08 INFO SignalHandler Caught signal SIGTERM

2016-12-07T06:36:08 INFO Z2 Shutting down fast

2016-12-07T06:36:08 INFO ZServer closing HTTP to new connections

2016-12-07T06:37:16 INFO ZServer HTTP server started at Wed Dec 7 06:37:16 2016
Hostname: 0.0.0.0
Port: 12982

so if it isn't a cronjob restsarting it, what the hell is??? Interesting!

9

@rileydog change your password immediately. You pasted your admin password into your post.

Can you correlate those restarts with anything else in other logs? access logs? anything. Maybe you have something wrong going on.

It also looks like you're hosting at web faction. Maybe they are doing something?

1 Like
10

You're also accessing a public url with those cron jobs that is NOT SSL. So those passwords are getting sent in clear text.

Dangerous.

11

Maybe there's a memmon running that's killing your client when it thinks the system is running out of memory.

http://superlance.readthedocs.io/en/latest/memmon.html

12

Hi, back from vacation and getting back into this

@vangheem thanks for pointing out I posted the pwd.

I checked the memory usage and It appears that it is a memory issue. I've upgraded to a plan with more memory. We know if a few days if this fixes it.

The reasons I didn't immediately know it was a memory issue (if time proves it is): 1) It's been running on the same hosting plan for a long time without issues. I assume they must have changed how the count memory or something; 2) I didn't get the usual email notification when I run out of memory; 3) the issue regarding the "robots.txt" above appears to be a red herring and unrelated to shutdown (I will post this separately to get input on if it is a bug and if so, where to file.

thanks for your ideas. hopefully this will just be an instance where throwing a few bucks at it fixes the issue.