Hello,
while debugging LocalFS I noticed that Zope makes two reads for every browser visit to a localfs file.
I didn't research it since BaseRequest.traverse is quite a big function, so I wonder if this is always the case or something specific to LocalFS which I'm overlooking?
A request will be repeated in case of a ZODB conflict error. If your code is executed twice within one request then take the debugger for figuring it where and why.
well, I don't have any ZODB conflict error, not sure it is even involved since I'm accessing a local file.
what I can see is that bobo_traverse is called two times from BaseRequest.traverse for the same request.
As Andreas said, you should be able to see the reason for that from the debugger. Typing where or w in the debugger should print the current call stack. If you wish us to take a look, you could also paste stacks for both of the calls here.
thanks for your feedback, @datakurre. I need to check it again more carefully because I don't think the stack helps. IIRC, they are the same in both calls. Once I tracked back but I got lost somewhere in the middle of waitress code. I'll come back to it asap (in days, I hope).
So, this is the stack I get when traverse calls LocalFS through bobo_traverse:
(Pdb) w
/usr/local/lib/python3.8/threading.py(885)_bootstrap()
-> self._bootstrap_inner()
/usr/local/lib/python3.8/threading.py(923)_bootstrap_inner()
-> self.run()
/usr/local/lib/python3.8/threading.py(865)run()
-> self._target(*self._args, **self._kwargs)
/.../webportal/zope4/eggs/waitress-1.3.0-py3.8.egg/waitress/task.py(89)handler_thread()
-> task.service()
/.../webportal/zope4/eggs/waitress-1.3.0-py3.8.egg/waitress/channel.py(356)service()
-> task.service()
/.../webportal/zope4/eggs/waitress-1.3.0-py3.8.egg/waitress/task.py(176)service()
-> self.execute()
/.../webportal/zope4/eggs/waitress-1.3.0-py3.8.egg/waitress/task.py(447)execute()
-> app_iter = self.channel.server.application(env, start_response)
/.../webportal/zope4/eggs/Zope-4.1-py3.8.egg/ZPublisher/httpexceptions.py(30)__call__()
-> return self.application(environ, start_response)
/.../webportal/zope4/eggs/Paste-3.0.8-py3.8.egg/paste/translogger.py(69)__call__()
-> return self.application(environ, replacement_start_response)
/.../webportal/zope4/eggs/Zope-4.1-py3.8.egg/ZPublisher/WSGIPublisher.py(337)publish_module()
-> response = _publish(request, new_mod_info)
/.../webportal/zope4/eggs/Zope-4.1-py3.8.egg/ZPublisher/WSGIPublisher.py(243)publish()
-> obj = request.traverse(path, validated_hook=validate_user)
/.../webportal/zope4/eggs/Zope-4.1-py3.8.egg/ZPublisher/BaseRequest.py(523)traverse()
-> subobject = self.traverseName(object, entry_name)
/.../webportal/zope4/eggs/Zope-4.1-py3.8.egg/ZPublisher/BaseRequest.py(350)traverseName()
-> ob2 = adapter.publishTraverse(self, name)
/.../webportal/zope4/eggs/Zope-4.1-py3.8.egg/ZPublisher/BaseRequest.py(84)publishTraverse()
-> subobject = object.__bobo_traverse__(request, name)
> /.../webportal/Products.LocalFS/src/Products/LocalFS/LocalFS.py(600)__bobo_traverse__()
-> method = REQUEST.get('REQUEST_METHOD', 'GET').upper()
(Pdb) p subobject
(<WSGIRequest, URL=http://machine:8787/tinymce/LICENSE.TXT>, 'LICENSE.TXT')
The exact same stack is followed on the second call to bobo_traverse and this is returned by LocalFS in the two calls:
--Return--
> /.../webportal/Products.LocalFS/src/Products/LocalFS/LocalFS.py(608)__bobo_traverse__()-><File ObjectW... 7F1B98AF7840>
-> return self._safe_getOb(name)
(Pdb)
> /.../webportal/zope4/eggs/Zope-4.1-py3.8.egg/ZPublisher/BaseRequest.py(85)publishTraverse()
-> if isinstance(subobject, tuple) and len(subobject) > 1:
(Pdb) p subobject
<File ObjectWrapper instance at 7F1B98AF7840>
--Return--
> /.../webportal/Products.LocalFS/src/Products/LocalFS/LocalFS.py(608)__bobo_traverse__()-><File ObjectW... 7F1B94D7CC40>
-> return self._safe_getOb(name)
(Pdb)
> /.../webportal/zope4/eggs/Zope-4.1-py3.8.egg/ZPublisher/BaseRequest.py(85)publishTraverse()
-> if isinstance(subobject, tuple) and len(subobject) > 1:
(Pdb) p subobject
<File ObjectWrapper instance at 7F1B94D7CC40>
I'm not sure how to help from here, following the stack up and trying to figure out why this happens is just unrealistic for my knowledge of Zope. Let me know if I can do help otherwise.
Is the request object same for the both requests? (I believe, if not else, id() should give some identifier for the request.
different ids:
-> subobject = object.__bobo_traverse__(request, name)
> /.../webportal/Products.LocalFS/src/Products/LocalFS/LocalFS.py(600)__bobo_traverse__()
-> method = REQUEST.get('REQUEST_METHOD', 'GET').upper()
(Pdb) id(REQUEST)
139756585784080
(Pdb) c
> /.../webportal/Products.LocalFS/src/Products/LocalFS/LocalFS.py(600)__bobo_traverse__()
-> method = REQUEST.get('REQUEST_METHOD', 'GET').upper()
(Pdb) id(REQUEST)
139756585784032I[quote="zfm, post:8, topic:8985"]
different ids:
[/quote]
So, that sounds like there is only on read per request, but there are really two requests made by the client you are using to test this? Is the method same for both requests? For example, it is pssible the the client is doing HEAD and GET requests.
That's it! I see now that I click in the link .../LICENSE.TXT/manage_workspace which gets redirected (302) to .../LICENSE.TXT/manage_main.
The problem is that this implies two file reads in LocalFS. This means LocalFS.bobo_traverse should do something different?
Read the file only when the content is really needed (there should be no need that __bobo_traverse__ already reads the file content; an existence check should be sufficient).
Sorry, not clear to me how this is defined. What could be chequed in the REQUEST to make this decision?
I assume that you wrap the "local file" via a proxy resembling OFS.Image.File. If this assumption is correct, then you would look which of its methods accesses the file content. The most prominent of those is index_html. Your proxy could provide implementations for those that access the content of the "local file" instead. Alternatively, your proxy could emulate the data attribute of OFS.Image.File (it contains the file content) via an (intelligent) property accessing the "local file" content instead. Other attributes/methods (e.g. content_type, get_size) may need to be properly handled as well.
Sorry, I sort of understand what you say, LocalFS does a mixin with OFS.Image.File on the fly to create the proxy, but I fail to see the connection with the fact that Zope redirects from manage_workspace to manage_main. How should LocalFS know that Zope is doing a redirect and should not provide data on the first request?
Please explain your current usage LocalFS? What is it actually used for? Can't you replace it with a more modern implementation like xmldirector.connector? What functionality of LocalFS is actually needed?
For that, look into REQUEST.response. It should have status code and Location-header in its headers, but at the moment, I don't remember the exact attributes (or possible helper functions).
There is no need to know that: manage_workspace has no need to access the file content (indeed it only looks at the available actions and which of them the current user has permissions for and makes a corresponding redirect). Thus, if you delay access to the file content until there is real need (e.g. in index_html), you do not need to know what manage_workspace does in detail (apart from the fact that is does not need to access the file content).
My recommendation: make (lasy, potentially cached) properties/methods for data (access to file content), get_size (the file size) and content_type (the file's content type). "lazy" here means that you access the file system only when those properties/methods are first accessed for the proxy object.
If you follow this recommendation, then manage_workspace will (automatically) not access the file system (as it does not use any of those attributes/methods).
Use case is simply serving a few files through Zope and I'm using it for legacy reasons and for learning experience. A simpler alternative would be to configure Apache (currently used as reverse proxy for zope). But I wanted to have this flexibility in Zope and after 15 years using it, I thought LocalFS deserved a refresh. It looks like good software to me, with the exception of many "except: pass". When I searched for an alternative to LocalFS, I did not see your add-on and anyway it needs Plone which I don't use.
Thanks, will look into that.
Ok, thanks a lot for your detailed feedback, I understand what you mean, but it's not a solution I can follow now for several reasons, a small one of them being that I don't quite understand this redirect. What is the semantics of manage_workspace, how it differs from manage_main, and why is it redirected to manage_main. Sorry, still learning Zope, never looked into its publication system.