Seeking Advice on Automating Page Updates in Plone 5 Using REST API

Plone Support Administering Plone sites
,
1

I'm seeking your expertise regarding a challenge I've encountered at the university where I work. We use Plone 5 for managing our website content, with an IT department that maintains our servers. My role involves using the graphical UI to build pages for our institution, but I've been exploring ways to streamline our processes, especially regarding list updates on our pages.

We frequently update various lists on our site, and I'm considering automating this process using data from CSV files. The idea is to use a Python script that interacts with the Plone REST API to automatically update these lists, ensuring our content is always current without manual intervention.

So far, I've successfully logged in using the API and managed to retrieve some JSON objects and HTML from our pages. However, my main concern revolves around my limited rights and capabilities in this context.

My question to you all: Is it feasible for someone with my level of access to build, update, and edit pages via the REST API in Plone 5? If so, I would greatly appreciate any guidance or insights on how to approach this, including any potential limitations I should be aware of.

Thanks in advance.

2

The specific problem is that, like any API, you must read and study the documentation, review the examples, and gain experience. The Plone API is adequately documented and provides enough examples for someone with average programming experience to begin. Experience is not acquired effortlessly; it requires a time investment.

3

That's absolutely true, zopyx. Luckily I was today able to make my script work, and I am now able to change the text field of a news item. Through a strike of luck I was able to guess the payload required to change the "Text" content of a news item:

{"content": {"data":"IT WORKED!"}}

Do you know how / where I can learn the payload /JSON syntax to interact with our university Plone sites? Do I need to contact the Plone administrators which run the servers? The problem is that they have already told us that, thought they do offer support for the GUI, they do not offer it for the API. I believe they have just one or two people who have advanced Plone knowledge, and that they are busy with more important things.

---code---

import requests
import json
import getpass
import base64

PLONE_SITE_URL = 'https://***'

def encode_credentials(username, password):
return base64.b64encode(f"{username}:{password}".encode()).decode()

def get_headers(auth_token):
return {
'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': f'Basic {auth_token}'
}

def process_response(response):
try:
if response.status_code in [200, 201]: # OK or Created
return response.json()
elif response.status_code == 204: # No Content
return "Operation successful, but no content returned."
else:
return f"Error: Status code {response.status_code}, response: {response.text}"
except json.JSONDecodeError:
return "Received non-JSON response or no content."

def update_content(url, headers, payload):
response = requests.patch(url, headers=headers, json=payload)
return process_response(response)

if name == "main":
username = input("Enter your username: ")
password = getpass.getpass("Enter your password: ")
auth_token = encode_credentials(username, password)
headers = get_headers(auth_token)

document_url = PLONE_SITE_URL + 'testraum/test/'

update_payload = {"content": {"data":"IT WORKED!"}}

print(update_content(document_url, headers, update_payload))
4

As said, the documentation has enough examples and how to be used from different backend:

plone.restapi must be installed on the Plone server and your user account must have the related permissions on the content tree are working on.

5

@thenarfer Volto (the Plone 6 React-based frontend) is fully using the restapi to communicate with the Plone backend. You can watch the network traffic on https://demo.plone.org/ to take a peek at that communication protocol.

plone.restapi has its own section in the official Plone docs

6

Thanks for your insights, tiberiuichim. I'm encouraged by our university's upgrade path from Plone 3 to 5 and eagerly anticipate Plone 6.

So far, I've managed to change the HTML content of a news item, as detailed in my previous post. However, this success is limited to simpler site types. My challenge now lies in understanding the syntax for more complex payloads, especially for sites built with multiple building blocks and templates. Unfortunately, I haven't been able to retrieve information about these complex sites' content from the server. I'm not sure if this is a limitation of the REST API or if I'm missing the correct approach to request this information. Ideally, I'd love to be able to target and modify specific HTML elements, like a

with a particular ID, directly through the API.

Have you or anyone else in the community dealt with similar challenges? Are there examples or resources that could guide me on how to structure payloads for such complex interactions? Any further assistance or pointers would be greatly appreciated.

7

Please provide more specific questions or problems. Your questions lack context, making it difficult to provide meaningful assistance.

8

"Divide and conquer". There's no solution to create/mutate multiple bits of information in a single request, so why not do multiple requests? As zopyx was saying, you have to be specific about a problem. For example, I'm not sure how to understand the "multiple building blocks and templates".

9

Off topic, but just using plone.api could be an option (?)
Then you dont have to deal with JSON at all.

To update a field if very easy, something like this:

    #from zope.lifecycleevent import modified
import plone.api
import transaction

brains = app.mysite.portal_catalog(id="someid")  # or some other search

if brains:
	for brain in  brains:
		obj = brain.getObject()
		obj.somefield = 'something'

	transaction.commit()

Also off topic, but in case you dont want CSV but use Excel directly, that is quite easy with Pandas. Something like this (I think):

#One or two of these
import pandas as pd
#from pandas import *
#import openpyxl

excelfile = 'somefile.xlsx'

df = pd.read_excel(excelfile)
#print(df)

my_dict = df.to_dict(orient='index')

def update_something(brain):
    #then probably search for id in plone vs id in excel
    for key, value in my_dict[i].items():
10

Completely off-topic. The thread is about Plone REST API.

11

Maybe, but sometimes one does not know about the other way of doing it. After all, plone.api is super simple to use. For example, I have a site where the users just selects an Excel file and whatever field has changes since last time is updated. Maybe a bit similar so it is not 100% sure doing this with restapi is a better approach.

12

The question is about remote access over Plone REST API. Nothing else. Please read the subject, please read the thread completely. The person asking has no access to the Plone server for using plone.api.

13

All he needs is ssh access to the server, even I have that (to our local university, and I dont even work there)

14

Thank you for the valuable insights and suggestions so far. I'd like to provide additional context to further clarify my situation and seek more targeted advice.

To illustrate my challenge with complex pages, here's a glimpse of what the GUI looks like:

grafik
grafik1495×811 67.9 KB

In contrast, for a simple site like a news item, I can target the content effectively as shown here (left: the page, right: edit mode):

NewsItem
NewsItem1636×1254 107 KB

When I perform a GET request on the complex page, the JSON I receive from the server looks like this:

GetResponse
GetResponse1354×189 12.9 KB

In this scenario, I can modify several fields, but not the "content" field as I can for the simple site.

I'm hoping these visuals might spark some ideas or insights within our community. If it seems like my current approach won't work, I'm open to feedback and willing to delve into more detail if needed.

Looking forward to your thoughts and suggestions.

15

Something is wrong, maybe with the UnibonnNews or something else.

For the Plone news items, the "content" field is a richtext field called "text". I've tested on https://classic.demo.plone.org, I've created a new news item demo.plone.org and then accessed https://classic.demo.plone.org/++api++/en/dsadas. I see the richtext field published right there:

image
image909×1366 140 KB

16

Thank you for testing this out for me! I highly appreciate it!

I do wonder if you reproduced the example where I am also successful: The news items

The specific page which I would like to automatically add names to is this one: Alumni — Department of Economics / BGSE

I hope the link gives some idea how the page differs from a news item. Do you have an idea of how I could add a new row to this page via the REST API?

Thanks again for your time and effort.

17

Looking at the HTML markup produced by that page, it seems that it's using some sort of Tiles (maybe Plone Mosaic?). I think the tiles are stored in annotations, which is not exposed as a Dexterity field (the other fields). I may be wrong, as I have almost 0 experience with Mosaic. I have experience with plone.restapi, though.

So the answer to your dilemma is: the text field doesn't appear because you're not actually editing "simple" Plone pages, but an content that's prodused by an addon that's not fully integrated with plone.restapi. It is possible to develop a way to make calls to the Plone backend to edit programmatically that content, but you would need a Plone developer for this, or time and willingness to learn this. For an experienced Plone developer, your task is medium complexity.

1 Like
18

"Even you" is an argumen for what? For nothing.
Stay on-topic.

19

I have to say that espenmn's comment was helpful and a discussion point for our team as it gave insights into how liberally ssh access could potentially be shared with us. We will try to talk to our Plone5 administrators, even though they initially have said that they will not help with API issues, just GUI stuff.

1 Like
20

Hi @thenarfer, sorry for being late to the party. I am the main author of plone.restapi and we have a very similar use case with the institute websites at www.dlr.de right now, where subsite site administrators want to manipulate Plone content via the REST API.

@tiberiuichim's previous comment is correct. The customization or layout system that you have at Bonn University is not fully integrated with the REST API. Since it is most likely Mosaic-based or a custom-built solution, it needs to be exposed properly. Since the system is built in a non-standard way, you would need to create custom serializers/deserializers. This is a task for a Plone programmer, not a user of the Plone REST API.

I'd therefore recommend reaching out to your Plone administrators and asking them to properly expose the existing layout system via REST API or, in the long run, move to a Plone standard layout system (e.g. Volto with Plone 6) that is supported by plone.restapi.

I wouldn't recommend going with the other options like SSH access and manipulating the content via plone.api. You will run into the same problems if the layout system is built in a non-standard way.

We had to train experienced Python programmers lately with Plone 6 and Volto (the standard layout system). The REST API is indeed well-documented. However, there is still lot of domain-specific knowledge necessary, even if you stick with the standards.

If you are dealing with a system that is built in a non-standard way by an integrator, this task becomes almost impossible without proper support from the Plone integrator company that built the system in the first place.

If your layout system is "tiles-based" (just guessing here), your solution provider might want to look into the tiles endpoint of plone.restapi:

https://plonerestapi.readthedocs.io/en/querysources/tiles.html

However, we removed support for tiles some time ago from plone.restapi, since the lack of interest in the Plone community and because tiles never became part of Plone core in the first place.