Souper.plone records, groups and local permissions

I'm planning a project which manages access control to information based on folder structure and local permissions. For performance reasons I'm considering souper.plone. Just putting my main "research" considerations out there, in case someone already knows the answers off their head.

If I use souper.plone to create lightweight records instead of using dexterity:

  1. Are souper.plone records aware of local roles and permissions? (my guess is that soups are aware NOT individual soup records)
  2. Can I organise souper records hierarchically? (my guess is that I'll have to organise the hierarchy at the soup level NOT the individual soup record)

Use case - replace dexterity content with souper.plone records

Let's say this is a school and the information to be stored in souper.plone records is student information.

There are three Groups types:

  • School Level (I can see all information) e.g Main School Group
  • Grade Level ( I can see all information for a grade) - eg. Grade 2 Group
  • Class Level (I can see all information for a class) - eg. Class 2-1 of Grade 2

Using a pure dexterity approach, I would create "student" content types and then place them in their various classes. Permissions would be managed based on a folder hierarchy and local sharing permissions. The folder structure would end up looking like this (Grade 2, Class 2-1 expanded for illustration):

   ----- Grade 1
   ----- Grade 2
  |          |
  |          ----- Class 2-1
  |          |            |
  |          |            |--- student 1
  |          |            |--- student 2
  |          |            |--- student 3
  |          |
  |          ----- Class 2-2
   ----- Grade 3

Other things I'm considering

  1. I need to confirm that I'll need to create multiple ISoupRoots to simulate hierarchy
  2. What out of the box dexerity capabilities will I sacrifice?

no souper records aren't heirachical or have per record security. They also aren't searchable in the main plone catalog. The whole idea is they are independently indexed table of data.
Sounds like you want dexterity rather than souper.

A souper model you likely have one soup for students and one for class and then store ids into them linking them.
A lot also depends on how you want to query this data.

Thanks @djay
I like folder hierarchy and local sharing permissions and already have my head wrapped around that. For my use case performance issues may force my hand.

  1. It's thousands of records not dozens of students
  2. Importing dexterity content is super slow compared to rows in a relational db (or souper.plone I assume)
    If I can't improve the speed of importing dexterity (my major source of pain at the moment) I'll have to sacrifice dexterity. No dexterity also seems to mean reworking the security approach and architecture. At some point I wonder if I should just look at a traditional relational db :thinking: (thinking aloud)....

Take a look at (Integrating SQL Data into your Plone Site by Eric Steele) and see if that might allow you to integrate a simple relational DB incl permission checks.


So far it looks like:

Map a dexterity schema to an SQLAlchemy schema

His example is of a product

Create a class based on a trajectory Model and have it implement the mapped schema

Then he says ProductWrapper (the trajectory Model) implements IProduct

This could work.

Using this in production, would love to see this package go into the collective...

Does it deliver on the promise? For me I want to be able to set local permissions on the objects.

Good question, since your data does not exist in Plone (you generate a view based on the SQL query) I am not sure that you would be able to use local permissions. Maybe store your permission attributes in your source data and use a filter before returning it to Plone? Depends on how complicated, extensible etc. your permissions model is.

We use it to generate a view of product information which is stored in an external legacy system. Have dabbled a bit with edit forms and looks promising (planning to work on that feature for the next major update of our system).

I think you can add __ac_local_roles__ function to your School, Grade and Class classes which can return 'Student', 'Authenticated' or some other role name for the current user.

Maybe there's an easier way, but I don't know of any.

So my solution for the moment....
Stick with dexterity and throw more ram at it.
We're working to a deadline. It's good to know about this trajectory stuff though.

Helps if you state your problem not solution up front.
If your problem is import speed then I'd be looking at ways to make indexing more efficient. I think there has been much written about that and how much affect that has on import speed. Obviously not using restapi and having a more direct method to import is going to help too.
Souper import I'm not sure is going to be super fast but I don't know. @ebrehault is the person to ask. He imported wikipedia into souper.
As far as I know souper still writes its indexes at teh same time you import a record. Look at solutions like elastic or solr to see if you can delay indexing and do it external to plone?

I assume you are breaking it down into seperate transaction (not just subtransactions). That will reduce your ram. Using subtransactions you have ot be very careful you aren't keep references to items between subtransactions as then GC won't do anything.

@pigeonflight another idea (I've no idea if someone tried or not before) is to use zopes low level copy functionality if your objects are largely the same then just update any values that have changed. That could reduce the amount of code run to create each dexterity object a lot?

If you also disable zope events for copy then helps since there is a bunch of code hooked into that which does nothing in the end (like do you really need content rules and redirectiontool evaluation stuff during inport). We had a plugin somewhere that disabled zope events during zxp imports and it sped things up a lot.

1 Like

For completeness... yes my problem has to do with import speed. Throwing ram at it has helped but... I'm reviewing other options from this thread.

Yes, but the indexes are souper internal and super small compared to Plones indexes.