I doubt that there is something working out of the box for Plone 5.
In general MathJax is the way to go.
It should not be that hard to register a dedicated paragraph style with TinyMCE for MathJax,
adding MathJax JS files to the Plone resource registry and a renderer for MathJax that is triggered by the related paragraph style.