It was too suspicious to be something as broken as it showed, and usually in this cases the solution is easier than expected 
So, for me the problem is that at some point between Products.CMFPlone 4.x and 5.x the dependency on plone.app.upgrade was removed and as I'm not depending on Plone distribution, but rather on CMFPlone directly I was missing plone.app.upgrade.
So for anyone having similar problems, double check that your bin/instance has plone.app.upgrade listed there!