Creating Custom AddOn to analyse audio

Plone Support Add-on Development
1

I would like to create an AddOn with the following functionality.

  1. User uploads some audio via a webform
  2. Server analyses the audio
  3. Plone presents the results of the analysis
  4. User can then annotate and share the analysis

Now, the actual functionality is the easy bit for me (step 2 is written), but I'd like some information about how to package this up as a Plone AddOn so that I can add it to my site.

2

I think the Plone training will give you all the required information to do this step by step.

3

For me, this looks like a new "content type" ("audiofile") derived from Plone "file" content type.

4

Just some brainstorming:
Maybe you dont even need an add-on for this.
You could use the webform to send the audio file to a server and then (when it is finished ) add the content to Plone ( restAPI or maybe even webdav)

Maybe the (easyform) webform could add a (folderish probably) content item (add a script) so you get the permissions and then upload the analysis and the audio (if you need that) to it

5

What exactly do you want to analyse/achieve?

For a client I'm building a shop for production music licenses.
I've got a custom content type with two file fields (preview/high-quality)
The preview file is getting analysed with pydub and peaks are calculated with numpi.
Peaks are then store in an additional field.
These peaks are then used to display the waveform of the audio file with GitHub - collective/collective.wavesurfer: Implementation of https://wavesurfer-js.org audio player for Plone there is an extra branch for those prerendered peaks.

Screenshot 2021-03-17 at 08.21.47
Screenshot 2021-03-17 at 08.21.472876×1524 661 KB

6

I've written an AI component that identifies spoken languages and I want to put a demo of it on my website. So first I want a form that captures audio from the microphone. That audio will then be fed to the AI component, which will produce its maximum likelihood estimate of the language being spoken, its confidence, and a chart showing how the estimate evolved over the course of the sound clip. The user will then be given a form showing these results and giving them the opportunity to annotate them with such things as what language they were really speaking (I expect a lot of people to put in languages that the system hasn't been trained on just to see what happens) and then save the results to a gallery, from where they can share them.
At the moment, I think I need to create a content type for the upload form, one for the annotation form, and one for the final results.

Thanks for sharing your AddOn. It looks like I might be able to get some useful ideas from looking at the code.