Plone 4 site is up, but it’s not responding

(posted previously at http://stackoverflow.com/questions/40167344/plone-4-site-is-up-but-it-s-not-responding )

I have a Plone 4 site at http://tinakellyrussell.com/ However, when you go to the site, it stalls for a while and then times out. (This even happens when connecting via IP address, so it’s not a DNS issue.) It’s hosted on an Amazon EC2 instance, and if I connect via SSH and use “plonectl status” the result is:

instance: program running; pid=788
Restarting Plone (or the EC2 instance itself) doesn’t seem to help. It’s been running fine for years, I didn’t make any changes or updates when it started failing. I don’t know what the problem is :frowning:

(On StackOverflow I was advised to check the instance log; so I did, but I didn’t find anything there that looked unusual.)

Does it keep repeating the 'restarting' message? Then it may fail to startup, and try again, and fail again. There should be a better error message, but sometimes this is all you get.

The way to debug this is to start Plone in the foreground. Then it should fail and quit and this time print a hopefully understandable error message. See http://docs.plone.org/manage/troubleshooting/basic.html?highlight=foreground#start-plone-as-foreground-debug-mode

You need to stop the instance first. How you do that depends a bit on the names of the various scripts. It looks like in your case you would need to do this:

plonectl stop instance
plonectl fg instance

Or instead of the last one: 'instance fg' or 'zeoclient fg'. If you can't find the scripts, they are in the bin directory, next to the plonectl script.

Thanks for your response! I tried your method (in my version, it’s “bin/instance fg") and, sadly, no luck; the output shows the usual stuff:

2016-11-08 23:38:46 INFO ZServer HTTP server started at Tue Nov  8 23:38:46 2016
	Hostname: 0.0.0.0
	Port: 8080
2016-11-08 23:38:48 WARNING SecurityInfo Conflicting security declarations for "setText"
2016-11-08 23:38:48 WARNING SecurityInfo Class "ATTopic" had conflicting security declarations
2016-11-08 23:39:10 INFO Zope Ready to handle requests

…and then nothing, no matter how many times I try to access the site. (I’m not sure why it says port “8080,” or if that’s relevant, but in any case using “http://tinakellyrussell.com:8080” doesn’t seem to change anything at all.) In fact, even if I use “plonectl stop” to stop the instance entirely, nothing changes: I’ll try to access tinakellyrussell.com without Zope running and instead of getting an immediate error message, the browser will try to load the site and then time out after a minute or two, same as if I had the instance running. Weird!

I’m hoping this might shed some more light on my problem; please let me know if you have any ideas for what to try next!

as you can restart plone, you should try to do a "wget http://localhost:8080" when connected to the server I mean. If that works, then apache/nginx whatever you have in front of Plone is most likely the problem given the "Ready to handle requests" that you see when starting plone in foreground mode.

Consider trying all of the following;

(1) Use ssh -T -L 8080:127.0.0.1:8080 USER@YOURHOSTNAMEHERE to set up an SSH tunnel from your computer to your server; try to connect to http://127.0.0.1:8080 from your browser. This assumes Mac or Linux desktop; I am not sure what the magic incantation for SSH tunnels on Windows is.

(2) Once logged into your server, as the user running Plone/Zope: $ lsof | grep LISTEN -- when Plone is running, you should see IP and port. You can use this to rule out things like funky IPv4 vs. IPv6 settings, that sort of thing. Listening on 0.0.0.0 should mean that you can access Plone from any IP. But you may well have some iptables or other firewall rules on your server that you are not aware of, that may prevent connection from hosts other than localhost. Ideally you only want your Zope instance responding on 127.0.0.1 (either by config, or by firewall rule; likely the latter in a more complex situation).

(3) Try using a console based browser like links, lynx, or even wget to see if you connect ok to http://127.0.0.1:8080. Use actual IP (v4) address, not "localhost" as sometimes this can get mapped to only IPv6 (IIRC) ::1 address, which likely does not work with ZServer.

(4) Look carefully at your front-end server's configuration (Apache/nginx/etc); make sure that any hostnames used in any rewrite rules, backends, etc are mapped to valid ipv4 addresses that you have verified by above steps that Zope/Plone is listening on.

Hope this helps.

Sean

Sean

(1) Confirmed; with the tunnel in place I was able to view my website at "127.0.0.1:8080/site".

(2) When logged into the server, the result of “lsof | grep LISTEN” is as follows:
python 19314 tina 3u IPv4 4451133 0t0 TCP *:http-alt (LISTEN)
…I don’t really know what that means.

(3) With the tunnel established once again, I was able to view “127.0.0.1:8080/site” with Lynx just fine. (In fact, I’m rather impressed that I was able to use my site so easily with Lynx; I’m glad to know Plone supports even the oldest browsers!)

(4) I wouldn’t really know where to start looking at front-end server configuration, though I will say I haven’t changed anything there from the stock settings.

I hope this additional information will help us figure out the problem. Thanks!

If you haven't changed any setting from the frontend servers, maybe they crashed?

Do you know by chance which one are you using, nginx or apache? Which OS is this server running? You mention amazon...

If you grep on /etc for your domain there should be some file(s) matching, those should be the first ones to check.

I’m running Ubuntu Server 11.10 on an Amazon EC2 instance. http://cloud-images-archive.ubuntu.com/releases/oneiric/release-20120222/ (the image I used is listed as “ami-ae05889e”)

Output of “uname -a”:
3.0.0-16-virtual #28-Ubuntu SMP Fri Jan 27 18:22:35 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Runnning “grep -r apache /etc” gets me pages and pages of settings files (while “grep -r nginx /etc” only gets me a smattering of error messages), so I guess Apache is what’s running.

When I use the tunneling method to get access to Plone’s control panel, the version information is as follows:

Plone 4.1.4 (4113)
CMF 2.2.5
Zope 2.13.12
Python 2.6.7 (r267:88850, Aug 11 2011, 12:18:09)
[GCC 4.6.1]
PIL 1.1.6

Hope that helps!

Sounds like maybe your Apache is no longer running or configured correctly. Can you stop or start Apache and see if browsing to your site with the usual host name still works?

Could an IP address have changed?

I think I have this one finally fixed.
“IP address” was a good hint. The DNS settings for the tinakellyrussell.com domain were set to an “Elastic IP” on my Amazon Web Services account. This IP, though, wasn’t associated with any actual server instance (maybe it was, and the setting got changed somehow). The clue was when I noticed that the EC2 instance (in the AWS dashboard) was showing a different “Public IP” than the one I’d been using before. Now that I’ve reassociated the Elastic IP with the server instance, everything seems to run smoothly. Thanks for your help, everyone!

1 Like

Forgetting to point the Elastic IP back to the instance after a restart or rebuild is one thing that has bitten me more often than I'd like to admit...