Collabora 24.04.3.1: losing content of file even after saving it on Nexctloud

I have Nextcloud 28.0.1.1 installed on a vm and configured to work with collabora.
I have collabora 24.04.3.1 installed on a separate vm. both of nextcloud and collabora are behind an nginx reverse proxy.

I had collabora 23.05 installed and everything was working fine till i upgraded to 24.04.

the recommended troubleshooting in Troubleshooting — Nextcloud latest Administration Manual latest documentation is ok too.

here are the logs written in collabora while i do the following steps:
1- I create file.odt on nextcloud
2- I write something on it
3- I save it
4- I close it
5- I open it again
6- the file is empty and the content was not saved.

frk-20704-20704 2024-05-25 18:43:44.394977 +0200 [ forkit ] WRN  The systemplate directory [/opt/cool/systemplate] is read-only, and at least [/opt/cool/systemplate//etc/hosts] is out-of-date. Will have to copy sysTemplate to jails. To restore optimal performance, make sure the files in [/opt/cool/systemplate/etc] are up-to-date.| common/JailUtil.cpp:432
frk-20704-20704 2024-05-25 18:43:45.727964 +0200 [ forkit ] WRN  The systemplate directory [/opt/cool/systemplate] is read-only, and at least [/opt/cool/systemplate//etc/hosts] is out-of-date. Will have to copy sysTemplate to jails. To restore optimal performance, make sure the files in [/opt/cool/systemplate/etc] are up-to-date.| common/JailUtil.cpp:432
wsd-20686-20742 2024-05-25 18:43:45.897498 +0200 [ websrv_poll ] ERR  FileServerRequestHandler: File not found: Invalid URI request: [/browser/594b605/images/lc_zoom.svg].| wsd/FileServer.cpp:747
wsd-20686-20742 2024-05-25 18:43:46.320299 +0200 [ websrv_poll ] ERR  FileServerRequestHandler: File not found: Invalid URI request: [/browser/594b605/images/lc_invertbackground.svg].| wsd/FileServer.cpp:747
wsd-20686-20959 2024-05-25 18:44:02.034763 +0200 [ docbroker_010 ] WRN  Upload is already in progress.| wsd/wopi/WopiStorage.cpp:594
wsd-20686-20959 2024-05-25 18:44:02.034997 +0200 [ docbroker_010 ] WRN  Failed to upload [https%3A%2F%2Fnextcloud.instance.com%3A443%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F58833_xxxxx] asynchronously. Activity::None| wsd/DocumentBroker.cpp:1941
kit-20950-20950 2024-05-25 18:44:02.039737 +0200 [ kitbroker_010 ] WRN  #18: Background save process disconnected but not terminated 20963| kit/KitWebSocket.cpp:325
wsd-20686-20959 2024-05-25 18:44:14.828088 +0200 [ docbroker_010 ] ERR  #35: Read failed, have 0 buffered bytes (ECONNRESET: Connection reset by peer)| net/Socket.hpp:1146
frk-20704-20704 2024-05-25 18:44:17.094310 +0200 [ forkit ] WRN  The systemplate directory [/opt/cool/systemplate] is read-only, and at least [/opt/cool/systemplate//etc/hosts] is out-of-date. Will have to copy sysTemplate to jails. To restore optimal performance, make sure the files in [/opt/cool/systemplate/etc] are up-to-date.| common/JailUtil.cpp:432
wsd-20686-20742 2024-05-25 18:44:17.249574 +0200 [ websrv_poll ] ERR  FileServerRequestHandler: File not found: Invalid URI request: [/browser/594b605/images/lc_zoom.svg].| wsd/FileServer.cpp:747
wsd-20686-20742 2024-05-25 18:44:17.653661 +0200 [ websrv_poll ] ERR  FileServerRequestHandler: File not found: Invalid URI request: [/browser/594b605/images/lc_invertbackground.svg].| wsd/FileServer.cpp:747
frk-20704-20704 2024-05-25 18:44:30.935093 +0200 [ forkit ] WRN  The systemplate directory [/opt/cool/systemplate] is read-only, and at least [/opt/cool/systemplate//etc/hosts] is out-of-date. Will have to copy sysTemplate to jails. To restore optimal performance, make sure the files in [/opt/cool/systemplate/etc] are up-to-date.| common/JailUtil.cpp:432
frk-20704-20704 2024-05-25 18:44:38.513485 +0200 [ forkit ] WRN  The systemplate directory [/opt/cool/systemplate] is read-only, and at least [/opt/cool/systemplate//etc/hosts] is out-of-date. Will have to copy sysTemplate to jails. To restore optimal performance, make sure the files in [/opt/cool/systemplate/etc] are up-to-date.| common/JailUtil.cpp:432
wsd-20686-20742 2024-05-25 18:44:38.704992 +0200 [ websrv_poll ] ERR  FileServerRequestHandler: File not found: Invalid URI request: [/browser/594b605/images/lc_zoom.svg].| wsd/FileServer.cpp:747
wsd-20686-20742 2024-05-25 18:44:39.231258 +0200 [ websrv_poll ] ERR  FileServerRequestHandler: File not found: Invalid URI request: [/browser/594b605/images/lc_invertbackground.svg].| wsd/FileServer.cpp:747

Hey @azz. Thanks for all the info + log info. :slight_smile:

(Next time… if you’re having trouble getting your full log on the forum… you can also try to uploaded it to one of those other sites like pastebin, then share the link here.)

I edited your first post to make the log more readable. :slight_smile:


Also, instead of sticking it inside of a:

> blockquote

it’s better to stick it inside “triple backticks”:

```
giant block of
code goes in
between here
```

(It’s the button to the left of the number ‘1’ on your keyboard!)


Hmmm… I see there’s a Nextcloud 28.0.5. Mind doing an update?

And what’s your version of richdocuments as well?

You should be able to see that by:

  1. In your browser, going to the IP of your Nextcloud.
    • In my case, https://10.0.2.15.
  2. Click on your “Profile” icon in the upper right.
  3. Choose “Apps”.
  4. Along the left-hand side, choose “Your Apps”.
  5. Look for “Nextcloud Office”.
    • Should have a “Version” number next to it.
    • In my case, 8.3.7.

Hmmm… Well, I have a Nextcloud 28.0.3 + 24.04.2.1 Docker running fine as of tonight.

But I’ll update to latest as soon as I can and retest. Maybe something went weird and was recently fixed in the previous weeks.

And the only variable that changed was the Collabora install? Or did you do OS + Nextcloud updates between that too?

Hello @Tex

Thanks for the information.

I have another instance of Nextcloud in preproduction where I’ve just done an update to 29.0.0. In this instance richdocuments is at 8.4.2 and the same problem is reproducible. (Both of Nexcloud production and preproduction are connected to same Collabora instance via <group></group> configuration in Collabora)

Without updating Nextcloud on production I’ve done the update of collabora on the same vm that had collabora 23.05. when I noticed the problems I created a new vm with collabora 24.04 (both of the old and the new vm have the same OS: Debian12)

I could get rid of the warning

frk-20704-20704 2024-05-25 18:43:45.727964 +0200 [ forkit ] WRN  The systemplate directory [/opt/cool/systemplate] is read-only, and at least [/opt/cool/systemplate//etc/hosts] is out-of-date. Will have to copy sysTemplate to jails. To restore optimal performance, make sure the files in [/opt/cool/systemplate/etc] are up-to-date.| common/JailUtil.cpp:432

with the command: coolconfig update-system-template indicated in Updating systemplate — SDK https://sdk.collaboraonline.com/ documentation

but I still see random behaviours while creating a file, saving it and opening it immediately after.

Opening the file immediately after saving it results in two behaviours:

1- The screen displays an error “Socket connection establishment failed or socket connection closed unexpectedly” as shown in the screenshot. After waiting a little bit the file is rendered either empty or with the content I wrote.
Screenshot 2024-05-30 at 10.15.49

or

2- The screen doesn’t display the error and the file is rendered either empty or with the content I wrote.

I still see those logs while saving the file:

wsd-10789-11216 2024-05-28 18:39:11.771791 +0200 [ docbroker_00e ] WRN  Upload is already in progress.| wsd/wopi/WopiStorage.cpp:594
wsd-10789-11216 2024-05-28 18:39:11.771981 +0200 [ docbroker_00e ] WRN  Failed to upload [https%3A%2F%2Fnextcloud_url%3A443%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2Fxxxxx] asynchronously. Activity::None| wsd/DocumentBroker.cpp:1941

Here are the logs I see when the error “Socket connection establishment failed or socket connection closed unexpectedly” appears in the screen:

wsd-10789-10861 2024-05-28 18:39:38.898677 +0200 [ websrv_poll ] ERR  #39: Failed to create DocBroker [https%3A%2F%2Fnextcloud_url%3A443%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2Fxxxxx]: error: cmd=load kind=docunloading| wsd/RequestVettingStation.cpp:333
wsd-10789-10861 2024-05-28 18:39:39.679691 +0200 [ websrv_poll ] WRN  DocBroker [https%3A%2F%2Fnextcloud_url%3A443%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2Fxxxxx] is unloading. Rejecting client request to load session [045]| wsd/ClientRequestDispatcher.cpp:139

It appears that because of the unloading state of the file, this one can’t be rendered to the client and the request keeps trying to get it until it’s opened and rendered.

Apart from these logs I have these errors when I open the Nextcloud office interface. It seems that they are related to the absence of some icons:

wsd-10789-10861 2024-05-28 18:39:20.944231 +0200 [ websrv_poll ] ERR  FileServerRequestHandler: File not found: Invalid URI request: [/browser/594b605/images/lc_zoom.svg].| wsd/FileServer.cpp:747
wsd-10789-10861 2024-05-28 18:39:21.341236 +0200 [ websrv_poll ] ERR  FileServerRequestHandler: File not found: Invalid URI request: [/browser/594b605/images/lc_invertbackground.svg].| wsd/FileServer.cpp:747
2 Likes

Wow this is a really interesting report; thank you so much for the detailed analysis; I have a few questions that would really help going deeper here.

  1. Just to confirm - you are not using richdocumentscode - there is a plugin with an included appimage and other things which is strongly not recommended; it sounds like you are not - which is good - with your separate VM for this.

  2. Failed to create DocBroker - error: cmd=load kind=docunloading - this is a really interesting one - my belief was that we had removed this in 24.04.

  3. This is gold dust: WRN Failed to upload … asynchronously. Activity::None - it is not clear why that happens.

Its really great that you can reproduce this; can I ask that you:

  • Enable trace logging so we can get more debug output - and send us a whole log; to do that set to ‘trace’ - and then we’ll need to find somewhere to up-load the log to - gzipping and attaching to a github ticket might be good.

  • Enable logging on the Nextcloud side - I think there is an audit plugin that we should turn on - but … we really need to see why we failed to up-load to Nextcloud: I imagine this is close to the root of the problem :slight_smile: and of course we’d need those logs too.

Thanks again, so much for helping out

2 Likes

Im pleased to help.

No i’m not using richdocumentscode app on Nextcloud, indeed i’m using an installed Collabora instance on a separate VM. The installation of this instance was done from packages: URIs: https://www.collaboraoffice.com/repos/CollaboraOnline/CODE-deb

Here are the logs attached to github ticket:

Thanks!

1 Like

Fantastic. Thanks @azz.

I just read through the Github report, and it seems like the problem was narrowed down to:

  • Very low # of CPU cores in your VM.
    • At least 4 cores is recommended. (You had 1).
  • Antivirus running in your Nextcloud interfered (slowing things down dramatically).
    • 100% CPU during upload of files.
    • Dropping upload speeds down to ~0.5–1 KBps.

This would be helped by:

  • Giving it the proper # of cores. (4+ cores)
  • + Setting up your antivirus to run “every so often” instead.

Thanks for all the helpful debug information. :slight_smile: