Partial site exports anyone?

gforcada · July 7, 2026, 2:44pm

Hi all!

We have the use case of wishing to have a small data set from production database that our developers can use it locally to not have to manually re-create production content to simulate a front page etc. auto generating it is also not really an option, because we all know that devil is on the details and the object that breaks or does not render properly is never something that you can manually re-create.

Anyhow to the topic: partial exports.

Is anyone having the same needs as described above? How are you solving the problem? We are very interesting in it!

So far we were using some custom export views based on collective.exportimportbut unfortunately almost never work, or every once in a while it had to be patched to make it work again (either the export or import part).

Now that we have plone.exportimport which works more reliable, but unfortunately does a full site export, we came up with the idea of tweaking it to do partial exports.

As outlined in this issue:

github.com/plone/plone.exportimport

Export parts of the site

opened 10:35PM - 23 Dec 25 UTC

gforcada

04 type: enhancement

**Is your feature request related to a problem? Please describe.** If a site is… huge (millions of objects), it is not feasible to use `plone.exportimport`, as the generated folder is on the +10Gb order. That's impractical for either a developer to download it to work locally, or to bundle it on a Docker image to pass it to external partners (designers, integrators...). On top of that, a full export might contain sensible data that one might not want to export. **Describe the solution you'd like** Provide another export option that allows the user to graft parts of a site. Either `MANIFEST.in` like, i.e.: ```ini include /some-important-folder exclude /private-folder ``` Or simply a plain list of paths: ```text /path/to/a/deep/nested/document /path/to/another/document /list/goes/one ``` How each of these files are to be generated would be up to the user, they know their content, so they probably know best how to generate it. Another option would be add a `IExportMe` marker interface, they via some actions/forms you can mark the objects that you want to be exported, and `plone.exportimport` looks for that content via a catalog query. This one is probably not that flexible and appealing (IMHO, but YMMV 😅 ). The second option (a plain list) looks much more feasible to me. **Describe alternatives you've considered** So far we have hacked a few views built on top of `collective.exportimport` functionality, but it is brittle. `plone.exportimport` export folder structure and command line approach looks much more suitable to this task. **Additional context** Our use case is double: - get internal devs with fresh data on a daily basis (given that this export described above is performed on a daily basis on a production database copy), so they can fix bugs and reproduce problems easier 🚀 - get a safe data subset to external partners (real use case): an external provider needs to interact with our Plone site, but we don't want to make that external provider go through the hassle of getting all our code, building a Plone instance, and teaching our (quite complex) workflows that you need to go through to build the front page. **Solution?** Get this partial export, lump all that together with a Docker image of our fresh code straight from some specific in-development branches, so they can test our new code (that has not reached production still) on real data without any configuration hassle ✨

Our idea is now to add an adapter that is expected to return paths that need to be exported. Then a default adapter could list everything (and thus mimic the current export all behavior), but if on a policy package/add-on one defines its own adapter, then it would be up to your logic to define what needs to be exported.

We could go fancy and add a dexterity behavior to manually mark things to be (or not) exported, so you can influence the adapter’s logic via the UI…

Anyway, back to the topic: is anyone interested in the idea and already solving it in a way that can be shared?

Thanks!

yurj · July 7, 2026, 3:14pm

I prefer to clone the entire site database in my dev instance. Fast and I get the exact same data. Sometimes I've used zexp exports.

The big problem of partial exports is the related content that don't get exported or control panel settings that are missing too.

gforcada · July 7, 2026, 3:17pm

Thanks! we were doing that some time ago, but we stopped, as it was impractical.

For us cloning 100 Gb is not feasible to do on a daily basis… and most of the times we are not trying to replicate exactly production to debug an issue, but rather have a website with some content that avoids us having to create fake content that takes time and never really matches what is being done in production…

Esoth · July 7, 2026, 3:55pm

This sounds like a good use case for plone.distribution. I have a dev site that we rebuild nightly from a distro, so we can test functionality that way. It also makes it really easy to start up a new site locally, that is preloaded with dummy content.

erral · July 7, 2026, 7:52pm

We are using collective.exportimport and export blob paths.

Then we add a custom import product removing the blobs from the data dict justjust before importing.

Then, you have all the content without the images.

zopyx · July 8, 2026, 4:38pm

An inc

Use rsync for incremental updates...this should be fast even for a larger blob storage or are you talking about the Data.fs?

gforcada · July 9, 2026, 8:21am

Well, not at Data.fs but a PostgreSQL export, which can not benefit from a rsync incremental download.

But my main original goal and question here is not how to export a full database, that we are doing it since the very beginning, but get a small sensible data set so we can toy around when bugfixing or developing new features.