High Performance Back-end for Nextcloud Files

eganonoa

@girish it looks like this simple script works:

#!/bin/sh
NEXTCLOUD_BASE="/app/code"
NOTIFY_PUSH_BIN="$NEXTCLOUD_BASE/apps/notify_push/bin/x86_64/notify_push"
if [ -f "$NOTIFY_PUSH_BIN" ]; then
sed -i "/[Trusted_Proxy_IP_0_in_config.php]/a \ \ \ \ 1 => '[LOCAL_IP]'," $NEXTCLOUD_BASE/config/config.php
sudo -u www-data php $NEXTCLOUD_BASE/occ app:disable notify_push
  "$NOTIFY_PUSH_BIN" --allow-self-signed --nextcloud-url http://[LOCAL_IP] --port 7867 "$NEXTCLOUD_BASE/config/config.php" &
sudo -u www-data php $NEXTCLOUD_BASE/occ app:enable notify_push
sudo -u www-data php $NEXTCLOUD_BASE/occ notify_push:setup https://[NEXTCLOUD_URL]/push
fi

Save it into /app/code/config in the nextcloud container, make it executable and run it. Runs the notify_push command in the background, so no need of the tmux hackyness from my earlier post. It makes the set up simpler, requiring just installing the app in the Nextcloud portal, changing the nginx config (as per the prior post) and creating and running this script. The script adds the necessary trusted proxy before running the other commands, getting around the fact that it gets overwritten on restart of the container.

So now it's just about testing the cron concept you have and testing this out on others' systems to see if it works for all. Any feedback from others much appreciated.

girish

@eganonoa Looks great! The cron stuff should be released with the next release in the coming 2-3 days.

necrevistonnezr

@eganonoa said in High Performance Back-end for Nextcloud Files:

Save it into /app/code/config in the nextcloud container, make it executable and run it.

…. but you can’t do that as a Cloudron admin or was that directed at @girish ?

Have you noticed performance improvements - in particular against the improvements coming with NC 24?

eganonoa

@girish Awesome! Thank you. Put it into production today and can see all the team on it with no problems. Looking good.

One thing that would be very helpful would be if the default Nextcloud nginx.conf could be set to include the block from the guide, i.e.:

location ^~ /push/ {
    proxy_pass http://[LOCAL_IP]:7867/;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

Presumably that won't cause an issue for installs that don't want the HPB Files (though please correct me if I have that wrong). But it would massively simplify things for those that do, guaranteeing no overwrite of the nginx config and allowing for a very simple instruction to be added in the manual (i.e. find local IP, modify script and make executable, run executable to make sure everything is working, add to cron on reboot).

eganonoa

@necrevistonnezr

"…. but you can’t do that as a Cloudron admin or was that directed at @girish ?"

Sorry, I don't quite understand what you are meaning here. If you mean, can a Cloudron customer add something into /app/code/config and have that survive a reboot, surprisingly the answer is "yes". That folder isn't read only and whatever is put in there survives a reboot.

"Have you noticed performance improvements - in particular against the improvements coming with NC 24?"

Keep in mind that we are only a small team (10 people). They say that the speed increase will only be marginal for a team my size.

Having said that, yes! We're noticing a big increase in speed on two fronts:

(1) File Sync: Last Tuesday I was giving a new person a walk through Nextcloud. I was showing how sync works by creating a new file on the cloud and showing how that shows up on the desktop. It took forever for it to happen and was a bit embarrassing given all I had said in advance. Now it is as close to instant as you can get. Create new document online and immediately see it reflected on the desktop. Very nice.

(2) Notifications These are nearly immediately coming through. Start a call and the "join call" button shows up on the desktop within 2-3 seconds. Send a direct message to someone and it seems to nearly instantly arrive.

As it happens, I did this before upgrading to NC 24, but also wanted to see how things work with it, so did an upgrade. I don't see NC 24 adding that much additional speed to those two things vs the HPB files. But NC 24 is clearly a positive upgrade, just from a day of use. Getting positive comments from the team on the browser performance (inc. people in places with much worse internet than me). Some nice additions to certain apps (e.g. mail, office, text editor, all noticeably improved, file locks on desktop files being edited online). And if you are on Windows and presumably Mac you can reply to directed messages from the desktop client (sadly we use linux desktops so don't have that).

All-in-all I'd say we had a solid upgrade with Nextcloud between figuring out the HPB Files (that was the biggest) and NC 24 (a smaller but meaningful upgrade) (in addition to recently adding the high-performance back-end for talk). Would love to see what happens if/when Cloudron updates to PhP 8. But irrespective Nextcloud is getting closer and closer to being an all-in-one solution for us, which is pretty nice. Not there yet, but I can imagine it now that things are snappier.

jdaviescoates

@eganonoa that all sounds great, thank you for working this out. I've not closely followed the whole thread, but it sounds like we're getting close to be able to say: it you want to enable Nextcloud's High Performance Back-end for Nextcloud Files do this: ....

I'd really love to be able to use the HPB with my Cloudron Nextcloud!

girish

@eganonoa I have to read more on that this service is , but we have to use the apache config - https://github.com/nextcloud/notify_push#apache .

BTW, why is this just not part of nextcloud itself? Seems complicated having to maintain this as a parallel system outside of nextcloud.

eganonoa

@girish

"I have to read more on that this service is , but we have to use the apache config - https://github.com/nextcloud/notify_push#apache".

The way I did it was to set it up via the nginx config in the main cloudron server and then use the direct IP when running the binary in order to bypass the second proxy. So I didn't touch the apache config in the Nextcloud app (I understand it goes through both the Nginx proxy on cloudron and the apache on the Nextcloud container, right?). As of now everything works perfectly and if you don't add the block into Nginx it will fail. That's why I think it would be useful to automatically have that block in there (it redirects [NEXTCLOUD URL]/push).

"BTW, why is this just not part of nextcloud itself? Seems complicated having to maintain this as a parallel system outside of nextcloud."

Indeed it does. Given that it is just a simple service in the end, it doesn't make a whole lot of sense. Probably because it's ultimately all about desktop sync speed (with notifications a bonus) they keep it out of the main one for those who only use the web-interface or to keep overheads lower for people who either don't care about speed or would rather not add overhead to their nextcloud offerings (I'm thinking of those who might be providing it for free or for very low cost).

eganonoa

@jdaviescoates said in High Performance Back-end for Nextcloud Files:

@eganonoa that all sounds great, thank you for working this out. I've not closely followed the whole thread, but it sounds like we're getting close to be able to say: it you want to enable Nextcloud's High Performance Back-end for Nextcloud Files do this: ....

I'd really love to be able to use the HPB with my Cloudron Nextcloud!

I think now we're close enough to be able to say that indeed, and you can use the HPB with your Nextcloud without too much hassle. Once the change to Cron comes through, I think we're in good shape. Indeed it would be useful if others would try out what I have above so that we can be sure it's not just some fluke on my end!

eganonoa

@eganonoa said in High Performance Back-end for Nextcloud Files:

Noticed that the app-level nginx conf is getting overwritten daily, breaking the backend. Am trying the fix suggested by @girish here.

@girish Just to say, still no half-daily overwriting of the ngnix config since I made that change, as per the link. Everything moving smoothly. So indeed this change appears to have worked. Not sure its something you intend to incorporate into a coming release. But thought I'd give you that feedback in case you do want to.

girish

@eganonoa thanks for the notes! I will test this out once we have get 7.2 out and see how we can fix up nextcloud.

eganonoa

@girish said in High Performance Back-end for Nextcloud Files:

@eganonoa thanks for the notes! I will test this out once we have get 7.2 out and see how we can fix up nextcloud.

Awesome. Thank you. Glad I've been able to help out somewhere. Cloudron is a very important business to us.

eganonoa

@girish I've updated to 7.2.2 just now and cannot seem to get the binary to run with @reboot. I have tried in the crontab:

@reboot /app/code/config/notify_push.sh

and

@reboot sleep 60 /app/code/config/notify_push.sh (just in case a delay is necessary)

and watching the logs I can't seem get the binary to run.

Any thoughts on what I'm doing wrong.

girish

@eganonoa said in High Performance Back-end for Nextcloud Files:

/app/code/config/notify_push.sh

Is this path correct? For a start you can just put an @service echo "Push service running" just to double check that the cron configuration is picked up.

marcusquinn

Is this incorporated now?
We're having issues with speed in Nextcloud Talk (just the text chat, not calls) specifically.
Wondering if this is a solution, or these threads ending in 2020 are still relevant advice:

Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.

Cloudron Forum

High Performance Back-end for Nextcloud Files