Moving the Plone database to another partition

Plone Support Deployment & hosting
1

Hi folks,

having got my Plone installation up and running, I realised that the actual database sits in /opt/Plone/zeoclusters/var.
This is not good practice and doesn't really comply with unix standards in that typically databases should live in /var rather than /opt. Also, my root partition on which /opt site is pretty small and not frequently backed up. Conversely, my /var partition is frequently backed up. This link https://docs.plone.org/manage/deploying/copy.html suggests that I can actually move the database to another folder although this specifically refers to "copying a plone site" which is not what I want to do.

I want to keep the site (so the binaries, config files, log files, etc.) in /opt/plone and move the database to /var in the root unix directory. Having just navigated through the base.cfg file, I'm sensing that this is the place to make the changes but I want to reassure myself that I'm making the right changes and that I do not end up messing up my installation.

Would it be correct to say that in order to move my database to /var/plone from the current /opt/plone/zeoclusters/var/ I would have to modify the base.cfg file from the current:

storage locations

var = ${buildout:var-dir}
blob-storage = ${:var}/blobstorage

to the following lines:

storage locations

var = /var/plone
blob-storage = /var/plone/blobstorage

Is there any other line I need to change/add? Shouldn't there be a line for var/filestorage?

Thanks

2

Honestly, if you need to move your installation or your data directory then please move it and don't discuss about what best practice would be here. Ask 10 people, get 20 different opinions.

This should be fine. There is also a file-storage option, see plone.recipe.zope2instance recipe docs. The default is var/filestorage.

-aj

3

Just move the whole installation to a new location an re-run buildout. Easier, I think.

4

Apologies, wasn't my intention to start a flame of any sort. I'll keep to the facts and not the opinions in the future.

Thanks

5

Hi,

I've just tried to move the database folders blobstorage and filestorage (didn't want to delete the old yet). I followed these steps:
a) stop plone
b) rsync var/filestorage and var/bloblstorage to the new partition
c) change owner, group and permissions as per origina folders
d) make changes as per previous post in the base.cfg file
e) started plone (got no error and can connect to the site)
f) added a few new files and it turns out they being copied in the old rather than the new folder.

I'm assuming this is because at some point I need to tell plone that the config file has changed? How is this achieved? Is this achieved through a rerun of buildout but then will this not just delete the whole database and start a new install/site?

Thanks

6

That's what you have to do.

No, it will only update the configuration, but not change or delete any files from the database.

7

It was perfectly ok question, some people just likes to be rude.

1 Like
8

So I tried "sudo -u plone_buildout ./bin/buildout" and got a few errors with permission setting:

user@server:/opt/plone/zeocluster$ sudo -u plone_buildout bin/buildout
Updating zeoserver.
Updating client1.
Updating backup.
Updating zopepy.
Updating unifiedinstaller.
Updating precompiler.
Compiling Python files.
File "/opt/plone/buildout-cache/eggs/zodbpickle-0.7.0-py2.7-linux-i686.egg/zodbpickle/pickle_3.py", line 178
def init(self, file, protocol=None, *, fix_imports=True):
^
SyntaxError: invalid syntax

File "/opt/plone/buildout-cache/eggs/zodbpickle-0.7.0-py2.7-linux-i686.egg/zodbpickle/pickletools_3.py", line 2049
print("%5d:" % pos, end=' ', file=out)
^
SyntaxError: invalid syntax

File "/opt/plone/buildout-cache/eggs/zodbpickle-0.7.0-py2.7-linux-i686.egg/zodbpickle/tests/pickletester_3.py", line 145
class use_metaclass(object, metaclass=metaclass):
^
SyntaxError: invalid syntax

Compiling locale files.
Error while compiling /opt/plone/buildout-cache/eggs/python_gettext-3.0-py2.7.egg/pythongettext/tests/test_escape.po
Error while compiling /opt/plone/buildout-cache/eggs/python_gettext-3.0-py2.7.egg/pythongettext/tests/test5.po
Updating setpermissions.
setpermissions: Running # Dummy references to force this to execute after referenced parts
echo /opt/plone/zeocluster/var/backups yes > /dev/null
chmod 600 .installed.cfg
'# Make sure anything we've created in var is r/w by our group
find /opt/plone/zeocluster/var -type d -exec chmod 770 {} ; 2> /dev/null
find /opt/plone/zeocluster/var -type f -exec chmod 660 {} ; 2> /dev/null
find /opt/plone/zeocluster/var -type d -exec chmod 770 {} ; 2> /dev/null
find /opt/plone/zeocluster/var -type f -exec chmod 660 {} ; 2> /dev/null
chmod 754 /opt/plone/zeocluster/bin/*
user@server:/opt/plone/zeocluster$

Any idea? I've checked the permissions and they look right in that they are the same as they have always been. I didn't get this error when i installed the site.

9

I don't see any related errors expect the Python 3 related warning which are safe to ignore.

10

Hi,

to summarise the steps taken to move my storage database I have changed, under the following sections:

[instance_base]
[zeoserver_base]
[client_base]

all lines that contain:

var = ${buildout:var-dir}
blob-storage = ${:var}/blobstorage

were changed to the following:

var = /var/plone/var
blob-storage = /var/plone/var/blobstorage
file-storage = /var/plone/var/filestorage

Any idea what may be going wrong? It seems that with these changes, the zope server is not able to initiate listening on port 8080 base on what I read from the event.log (see below the log output from the changes).

event.log with base.cfg original

2018-03-23T22:00:02 INFO ZServer HTTP server started at Fri Mar 23 22:00:02 2018
Hostname: 0.0.0.0
Port: 8080

2018-03-23T22:00:02 INFO Zope Set effective user to "plone_daemon"

2018-03-23T22:00:05 INFO ZEO.ClientStorage zeostorage ClientStorage (pid=18608) created RW/normal for storage: '1'

2018-03-23T22:00:05 INFO ZEO.cache created temporary cache file ''

2018-03-23T22:00:05 INFO ZEO.asyncio.base Connected Protocol(('127.0.0.1', 8100), '1', False)

2018-03-23T22:00:05 INFO ZEO.ClientStorage zeostorage Connected to storage: ('localhost', 8100)

2018-03-23T22:00:11 INFO Zope Ready to handle requests

event.log with base.cfg modified:

2018-03-23T21:52:12 INFO ZServer HTTP server started at Fri Mar 23 21:52:12 2018
Hostname: 0.0.0.0
Port: 8080

2018-03-23T21:52:12 INFO Zope Set effective user to "plone_daemon"

2018-03-23T21:52:16 INFO ZEO.ClientStorage zeostorage ClientStorage (pid=18081) created RW/normal for storage: '1'

2018-03-23T21:52:16 INFO ZEO.cache created temporary cache file ''

2018-03-23T21:52:16 INFO ZEO.asyncio.client Connection to ('127.0.0.1', 8100) failed, retrying, [Errno 111] Connect call failed ('127.0.0.1', 8100)

2018-03-23T21:52:17 INFO ZEO.asyncio.client Connection to ('127.0.0.1', 8100) failed, retrying, [Errno 111] Connect call failed ('127.0.0.1', 8100)

2018-03-23T21:52:19 INFO ZEO.asyncio.client Connection to ('127.0.0.1', 8100) failed, retrying, [Errno 111] Connect call failed ('127.0.0.1', 8100)

2018-03-23T21:52:20 INFO ZEO.asyncio.client Connection to ('127.0.0.1', 8100) failed, retrying, [Errno 111] Connect call failed ('127.0.0.1', 8100)

11

I have not followed everything in this thread, however there are a few things of which you should be aware: the base.cfg is really linked to buildout.cfg in the Unified Installer that you seem to use; the buildout.cfg is really a setup file, not an operational file, in the sense that neither buildout.cfg or base.cfg are used in any way by the Plone software to do their work.
The case is that the operational files are generated FROM buildout.cfg and base.cfg every time you run buildout. IOW if you change buildout.cfg and/or base.cfg, nothing will be changed until you run buildout, that will really build your configuration.
SO... the operational configuration files that are being used effectively by Zope/Zeo/Plone are .conf file. You should never have to change theses files obviously because these manual changes may be rewritten by another buildout.
However, it's in these files that you should look to see what is really used by the Plone processes.
You should have something like zeoserver.conf, that's the config file from the process that handles the ZODB database. This is the process that is (or should) listen on port 8100.
The zeo client are doing requests on this port. The zeo clients are the processes that are doing the real http(s) work to manage users. So the zeo clients (or in your case probably 'the' one zeo client) can't find a zeo server.
Sorry I can't help you more as I said I use a tool called Ansible Plone that handles for me a configuration based on zeo (that is with multiple clients calling a zeo database process) but the test config that I have with the Unified Installer is a minimalistic configuration without zeo so I can't find easily where are the config files. But if you have zeo you should have a file to configure it (zeoserver.conf probably) and also log file for the zeo process handling the Zodb database. You have to find this file (with Ansible Plone it's called zeoserver.log I think it may be the same with Zeo configuration with Unified Installer) and look at it to see what went wrong.
Hope this make it a bit clearer.

12

'find /opt/plone -name zeoserver.conf' does not find anything so I'm guessing the Unified Installer doesn't use this file or has a different one?

When I installed plone, it didn't ask if I wanted to save my database in a different folder so I just went with the defaults.

I'm not sure I'm getting much useful info from zeoserver.log or at least nothing I can make much sense of (see below). What I do see is that when I use the original base.cfg and run buildout, I get a handshake. With my changes to base.cfg as above, I don't get handshake so clearly the changes I made are not correct but I can't figure out where I'm getting it wrong.

Surely there must be someone that has installed the database on a different partition compared to the main install in /opt/plone or is it that I am doing something rather uncommon?

zeoserver.log output

2018-03-23T22:34:53 (19586) created PID file '/opt/plone/zeocluster/var/zeoserver/zeoserver.pid'
2018-03-23T22:34:53 (19586) opening storage '1' using FileStorage
2018-03-23T22:34:53 (19586) removed PID file '/opt/plone/zeocluster/var/zeoserver/zeoserver.pid'
2018-03-23T22:34:53 sleep 8 to avoid rapid restarts
2018-03-23T22:34:53 pid 19586: exit status 1
2018-03-23T22:35:01 spawned process pid=19587
2018-03-23T22:35:01 (19587) created PID file '/opt/plone/zeocluster/var/zeoserver/zeoserver.pid'
2018-03-23T22:35:01 (19587) opening storage '1' using FileStorage
2018-03-23T22:35:01 (19587) removed PID file '/opt/plone/zeocluster/var/zeoserver/zeoserver.pid'
2018-03-23T22:35:01 sleep 9 to avoid rapid restarts
2018-03-23T22:35:01 pid 19587: exit status 1
2018-03-23T22:35:10 spawned process pid=19598
2018-03-23T22:35:10 (19598) created PID file '/opt/plone/zeocluster/var/zeoserver/zeoserver.pid'
2018-03-23T22:35:10 (19598) opening storage '1' using FileStorage
2018-03-23T22:35:10 (19598) removed PID file '/opt/plone/zeocluster/var/zeoserver/zeoserver.pid'
2018-03-23T22:35:10 restarting too frequently; quit

then, when I use the right base.cfg (original) I get the following:

2018-03-23T23:27:31 daemonizing the process
2018-03-23T23:27:31 set current directory: '/opt/plone/zeocluster/parts/zeoserver'
2018-03-23T23:27:31 daemon manager started
2018-03-23T23:27:31 spawned process pid=21671
2018-03-23T23:27:31 (21671) created PID file '/opt/plone/zeocluster/var/zeoserver/zeoserver.pid'
2018-03-23T23:27:31 (21671) opening storage '1' using FileStorage
2018-03-23T23:27:31 StorageServer created RW with storages: 1:RW:/opt/plone/zeocluster/var/filestorage/Data.fs
2018-03-23T23:27:32 listening on ('127.0.0.1', 8100)
2018-03-23T23:27:39 Connected server protocol
2018-03-23T23:27:39 received handshake 'Z5'

13

That really don't seem normal output. Here is part of my zeoserver.log:

2018-03-23T07:55:57 (662) opening storage '1' using FileStorage
2018-03-23T07:55:58 StorageServer created RW with storages: 1:RW:/var/local/plone-5.1/ploneprod/filestorage/Data.fs

so yes your zeoserver can't find its storage or maybe it does not have the right to access
it.

In every Plone setup I have seen so far, the configurations for the different components are stored in a subdirectory called 'parts'. Since you have a zeo configuration, you should have at least one subdirectory of parts called zeoserver, and under this directory a subdirectory called etc containing the zeo configuration file. Yes it can sometimes be called zeo.conf. So:

parts/zeoserver/etc/zeo.conf

You have probably also at least one subdirectory such as

parts/instance/etc/instance.conf

for your main Zeo client (=Plone process)

Yes it's a bit complicated (that's why I prefer to have a tool that is managing all of this for me...)

14

Ah, I found it. its zoe.conf. in parts/zeoserver/etc.

so are you saying that if I change the paths in this file and forget about the base.cfg (guess I wouldn't need to run buildout either?), I can control what the zeo server is doing and which storage directory it uses?

15

Err I think I said that you should use these .conf file to check, to understand what is happening (if it's a file location problem or a rights problem).

If it does not fit where the files should be you should change the .cfg files and run buildout. If buildout fails the .conf files will not be modified obviously.

The .conf files should not be modified directly that's not very good practice.