Using seamless mode with a different internal api path

Plone Support Deployment & hosting
1

We are trying to set up seamless mode so we can run multiple sites from the same frontend deployment.

However, when simply passing the public url of the host header this would mean the internal SSR and express middleware requests are routing back through the proxy instead of talking directly to the backend.

This seems problematic as for example in our case it will be going via an external proxy which will require TLS encryption etc, whereas our internal docker comms are already inside of an IPSEC network which provides adequate security. This is added overhead which would reduce performance of SSR rendering.

Is there currently no way of setting a header or any other way to control the internal api path on a per-request basis?

If not, is there any reason that it is undesirable? Would it be a feature we could add as a PR or is there some reason why it is not necessary?

1 Like
2

run multiple sites from the same frontend deployment.

As far as I know, this is unsupported by Volto right now, as it would imply a "thread-local" or shared global configuration (including some of the domain name)

@sneridagh ?

3

It does allow you to run multiple sites as per Seamless mode – Frontend – Deploying — Plone Documentation v6.0-dev

However it does require volto to be running in production mode (no problem) and we also have some custom avatar middleware that is not playing nicely with it (but that is already resolved in a later volto/restapi so we can fix that by upgrading).

It is working nicely aside from this issue with it requiring the same api path for internal and external requests.

Seamless mode is a really useful feature for us as the overhead of having to deploy a frontend per site will quickly build up.

4

Hi Jon,
The scenario that you are describing was one of the goals that we tried to achieve when we designed and implemented seamless mode. We did initial trials on how to do it, and although we achieved to improve some of them, some of others were left and postponed and are still untested/undiscovered. We haven't returned to the idea since some time ago.

Theoretically, you could deploy several sites using the same Volto SSR server without recompiling (just using the Host header in the reverse proxy). Before seamless mode, you would've had to have a Volto build for every site domain.

I have to apologise because I wrote this piece of documentation, now I realise that I might not have used the correct wording, or it could be misunderstood.

As @tiberiuichim pointed out, we've stumbled upon some walls when trying it out (eg. isolate properly the requests in Express server so the header detection per request does not leak to other requests). This doesn't mean that it's not possible, but that seamless mode opened the door for making it possible.

In addition, it seems that you have found an issue, which is that in docker scenarios with the internal_api_path enabled, you'll have to make it vary depending on the header too.

As you could imagine, it would be great if you could finish the work and made it work reliably. It seems to me since you have the need for this, you might have a perfect test bed for the use case.

Please let me know if you need anything or directions.
V

5

Hi Victor, thanks for the detailed response!

It is seemingly straightforward to set the internal api path based on a header.

In https://github.com/plone/volto/blob/master/src/server.jsx#L161 we have

if (!process.env.RAZZLE_API_PATH && req.headers.host) {
    res.locals.detectedHost = `${
      req.headers['x-forwarded-proto'] || req.protocol
    }://${req.headers.host}`;
    config.settings.apiPath = res.locals.detectedHost;
    config.settings.publicURL = res.locals.detectedHost;
  }

So I can just use some similar code to to the same for config.settings.internalApiPath.

I've tested this locally and it works just fine.

I made a draft pr which adds this functionality, but I guess it needs a little discussion/decision on the correct header name and also documentation in the seamless config page.

This is a little more concerning :see_no_evil: do you have any further information on the problems you were seeing? I haven't noticed anything like this in the (admittedly very cursory) testing I have been doing.

Unfortunately my knowledge of express is very limited so I'm not eager to promise much in terms of making it work reliably. I will be at the conference and sprinting, so I will endeavour to catch you and maybe see if we can move it forward successfully.

6

Yeah, but it can only be seen putting the Express server under high load (and changing headers). We pinpointed and fixed an error back in the day, and it didn't show up again.

Unfortunately we are as you, not really experts in Express so we did our best back in the day. Maybe we can bring it up during the conference and test it throughly to assess all is fine.

Yeah, let's follow this up in your PR, discuss with the other Volto Team and community members.

See you in the conference!!

7

I think the solution is to avoid saving the apiPath, always calculate it either from window.location or the x-host header

8

I tried to dive into this a little last night.

One of the main places that the api path is used is making api calls, so I started looking at @plone/volto/helpers/Api.js

There we have a block of code that is using the global config:

function formatUrl(path) {
  const { settings } = config;
  const APISUFIX = settings.legacyTraverse ? '' : '/++api++';

  if (path.startsWith('http://') || path.startsWith('https://')) return path;

  const adjustedPath = path[0] !== '/' ? `/${path}` : path;
  let apiPath = '';
  if (settings.internalApiPath && __SERVER__) {
    apiPath = settings.internalApiPath;
  } else if (settings.apiPath) {
    apiPath = settings.apiPath;
  }

  return `${apiPath}${APISUFIX}${adjustedPath}`;
}

As it is being called from the Api class which has the res object, it can be changed to something like:

function formatUrl(path, req) {
  const { settings } = config;
  const APISUFIX = settings.legacyTraverse ? '' : '/++api++';

  if (path.startsWith('http://') || path.startsWith('https://')) return path;

  const adjustedPath = path[0] !== '/' ? `/${path}` : path;
  let apiPath = '';
  if (__SERVER__) {
    if (!settings.internalApiPath && req.headers['x-internal-api-path']) {
      console.log('A:', req.headers['x-internal-api-path']);
      apiPath = req.headers['x-internal-api-path'];
    } else {
      console.log('B:', settings.internalApiPath);
      apiPath = settings.internalApiPath;
    }
  } else if (!settings.apiPath && req.headers.host) {
    console.log('C:', req.headers.host);
    apiPath = `${req.headers['x-forwarded-proto'] || req.protocol}://${
      req.headers.host
    }`;
  } else {
    console.log('D:', settings.apiPath);
    apiPath = settings.apiPath;
  }
  console.log('Returning api path of:', `${apiPath}${APISUFIX}${adjustedPath}`);
  return `${apiPath}${APISUFIX}${adjustedPath}`;
}

It requires making sure that the settings are undefined for both apiPath and internalApiPath when running volto (which requires index.js to be modified).

The internal api side seems to work fine, but I was having issues on the client side.

I didn't get much further than this, but I think that was down to not changing index.js properly (I was using shadowing in a separate package I am developing and it seems to have caused other issues). So I will try to make these changes in a clean volto and see if I can get it working.

I just wanted to see if this was along the right lines. There's another ~80 places or so that seem to rely on settings.apiPath and I'm not sure how/if they will all have access to the request. But it might be manageable?

9

But it might be manageable?

It has to be, or we change those places to use the dedicated Url helper functions.

10

I would go as far to actually consider a "smell" the presence of settings.apiPath in any "normal" code. Such code should use one of the Url helper functions.

11

@iojon awesome! I miss the api requests, made by the Volto server, being done internally, when Volto and Plone are on the same network. See my question:

This will likely improve performance. Mainly in the case of images and files. To see:

12

@wesleybl I think the issue you are having with images and authentication is that you are proxying @@images to Plone. I believe volto now has express middleware that will fetch the image from the backend with the correct authentication. Perhaps you could try removing the proxy rule to plone for those requests and see if it solves your problem.

I don't think either of those issues are related to seamless mode though :slight_smile:

13

@iojon yes, by redirecting @@images requests to Volto, the authenticated user can see the private image. But until I saw your comments here and your PR code, I didn't know that requests made by the Volto server could have a different url (using the variable RAZZLE_INTERNAL_API_PATH) than requests made by the browser. So, until now, my image requests were on a "long way":

browser -> nginx -> volto -> nginx -> plone -> nginx -> volto -> nginx -> browser

Now with the possibility of making internal requests between the Volto and the plone, this way will be smaller.

I'm starting at Volto now.