@girish

At the end of the day, I found the culpret.
Also upgraded to a bigger server.

While yes N8N will happily crash it's self - I had a few perfmance bottlenecks and for now N8N seems to be happy with the traffic. Here is a breakdown of the solution without setting up queue mode.

1. N8N uses postgress - so bumping up the RAM for N8N is not enough that just grows the list of RAM eating active processes, you also have to go bump the RAM for Postgres
2. I am using Baserow for logging and followup task Queueing. By default baserow only has 3 Auth workers which is not enough to cover heavy load. So I had to install the CLI and run the command cloudron env set --app baserow.draglabs.com BASEROW_AMOUNT_OF_GUNICORN_WORKERS=9

Conclusion - Step 1 Significantly improved the performance of N8N dropping normal RAM consumption by about 30%
Step 2 Caused active processes to complete 10X faster further reducing N8N footprint.

Now that I upgraded to 24 gigs I am only using 8

@girish said in out of space error leading to missing certs:

e-generate the nginx config for the dashboard alone. Once you have access to the dashboard, you can go to Location section of each app and click save. This will regenerate nginx config of the app

FIXED!!!

It is difficult to tell if deleting the conf files from the folder /etc/nginx/application and then restarting unbound Instructions then using systemctl restart nginx and systemctl restart box

I say that because unbound definitely was not working at at one point.
And as I remember nginx did start momentarily.

However the solution came when I deleted the corrupted zero byte private key from the folder /home/yellowtent/platformdata/nginx/cert/

When that file was deleted I was able to log in without ssl using firefox. Once in under the domains and certs section of cloudron I was able to click on Renew All Certs. That fixed SSL, and I was able to go into each program and re assign the dns settings by clicking save.

@msbt
Hi, I would like to take a look at this. I recently built my own N8N instance. And would really like to get cal.com running on cloudron.

Could you share more of your work in progress?

I am running into this problem with N8N under heavy load it will easily take down Cloudron. On the N8N website they talk about "queue mode" where one instance of N8N drives another.

It is important to have n8n using postgres or mysql
And it is important to have redis running to direct the sub instances.

beyond that... I am wondering if it is going to be possible to make this happen while using Cloudron.

I have the current problem where a load spike will bring in 2000 requests in a few minutes which reliably crashes N8N. It seems premature to upgrade the whole server just to handle a load spike.

https://docs.n8n.io/hosting/scaling/queue-mode/

Any ideas?

I am interested in being able to override the source code in running apps for a number of reasons.

Reason 1. I Found a bug in N8N and need to test the code edit before I submit a PR. Maybe it is just a one line change but the ability to slip in a custom hot fix would be super awesome!

Reason 2. Also an N8N example, but hard customizations may be good for me but won't necessarily get approved. You know.. changing the way an API works etc. How awesome would it be to have some granular control over the code running on your server?

Idea... Would it be possible to overwrite the source location to my own github fork? Then I would be manually pulling in new updates as opposed to relying on whatever the community publishes.

Most Awesome... would be to have an override folder where I could re-define a specific page while maintaining the normal update system in the background.

@robi

Related Feature Request. https://forum.cloudron.io/topic/7472/remote-file-access/5

It would be really awesome if rclone was implemented through the CLI enabling the synchronization of target directories and mounts.

In my specific use case I am trying to figure out a reliable way to be able to quickly edit my GRAV site. While GRAV has a GUI it was really build for the text editor.

I am trying right now to figure out how to install Rclone as a system level app to provide synchronization to my desktop computer of several cloudron folders.

If rclone was implemented in such a way that it could have access to other application's data folders that would be super awesome!

@girish

Also this whole issue was caused by running out of space - I took a look at some of the other posts on out of space crashes and can tell it is a difficult problem to solve.

Supposedly there is a running out of space warning but i never got that warning.

I was thinking that a good solution for the running out of space error would involve taking the remaining space cron which calculates remaining space every 'n' minutes and integrating it over 'x' hours to arrive at time to disk full.

This could relatively accurately predict if an out of space crash is pending or imminent - and if so... do things like stop processes prevent backup (if backing up to local filesystem) etc.

Essentially

1. predict the crash with a pinch of calculus.
2. send a warning to the administrator.
3. follow a contingency to protect the sever.

Because I could imagine many ways this could happen, and my example is ONLY one way. A program can crash Cloudron I could have been copying video files, It could have been NextCloud, a spam attack on a mailserver.

@MooCloud_Matt said in trying to scale N8N:

ference is that n8n workers need access to db and redis for the queue.
And they don't work correctly (they add a lot of del

The delay is actually what I was most excited about. N8N is really happy to crash load spikes which may only last a few minutes. I was hoping that queue mode would allow me to handle 5000 requests over say... 5 minutes, and then work them over the course of an hour....

@mehdi

Yah... Maybe it would be enough just to give the administrator a few controls.

1. the cron interval
2. the warning level
3. the kill level (kill all modules)

@msbt
Hi, I would like to take a look at this. I recently built my own N8N instance. And would really like to get cal.com running on cloudron.

Could you share more of your work in progress?

@robi

Related Feature Request. https://forum.cloudron.io/topic/7472/remote-file-access/5

It would be really awesome if rclone was implemented through the CLI enabling the synchronization of target directories and mounts.

In my specific use case I am trying to figure out a reliable way to be able to quickly edit my GRAV site. While GRAV has a GUI it was really build for the text editor.

I am trying right now to figure out how to install Rclone as a system level app to provide synchronization to my desktop computer of several cloudron folders.

If rclone was implemented in such a way that it could have access to other application's data folders that would be super awesome!

@roofboard
Hmmm I am looking at Rclone It appears to be the backend that many file sync apps like google drive use to "do the magic"

This may be a part of the better solution for folder synchronization...

@roofboard
What would be really awesome, would be a program which I could install on the server and on my desktop. Such a program like a CLI would run in the background like google drive and maintain real time synchronization at all times while being much faster than simple FTP.

@nebulon
Alright... Looks like I have a solution here.
I used a software called FreeFileSync
Also. I added my public key found in ~/.shh/id_rsa.pub to the allowedhosts file on my server in the folder ~/.ssh/allowedhosts

You can test this by trying to connect to your server wish ssh, when ssh works without typing in your password then you are in.

For simplicity I connected with root, but maybe someone else can hop on here and talk about creating a special user at the system level for ftp access.

Root has access to everything!

As root, I told FreeFileSync to connect to the folder where grav is installed.
/home/yellowtent/data/<long hash code>

I also created a directory on my computer where I wanted to see Grav mirrored.

Then inside of grav I used the connect with ssh option to connec to the server as root and synchronize the directory where I need to have desktop access.

This process could be repeated with any service on Cloudron, but I would also love to find an even better solution.

Problem

Some programs on cloudron really benefit from having direct file access on your computer. Like Grav for example. The ability to open you website up in a text editor and work on it is critical.

Ask

I would like to remote access and mount a specific cloudron folder to my computer like for example the "data" folder in grav. I have located it's exact path in the /home/yellowtent folder.

Method

I have looked at some methods using sshfs to mount the remote directory, but its not quite what I am looking for. I would really like to create a clean persistent tunnel between the server and my computer.

1. I am thinking about options like installing nextcloud directly on cloudron to run a synchronization service between local files and cloudfiles.

2. Permanently mounting the remote folder using CIFS SMB or SSHFS

3. Finding a commandline tool for continuous 2 way sync.

4. accessing remote folders using webdav or ftp

Has anyone had luck with the above methods? I am really hoping to find a solution which is easy and permanent and hopefully passwordless, making cloudron data folders persistently easy to access directly.

@JOduMonT

This looks really cool, any progress on building a docker container or getting this running?

@roru2k20

This looks cool, will it accomplish something similar to Segment? And progress on the integration…

@fbartels
I see, so...
Say I went with plan B, What are the steps to getting that custom package running on cloudron? Can I change the source location for an already installed package? Or is the way to install it separately as its own thing. If soo... how?

I am interested in being able to override the source code in running apps for a number of reasons.

Reason 1. I Found a bug in N8N and need to test the code edit before I submit a PR. Maybe it is just a one line change but the ability to slip in a custom hot fix would be super awesome!

Reason 2. Also an N8N example, but hard customizations may be good for me but won't necessarily get approved. You know.. changing the way an API works etc. How awesome would it be to have some granular control over the code running on your server?

Idea... Would it be possible to overwrite the source location to my own github fork? Then I would be manually pulling in new updates as opposed to relying on whatever the community publishes.

Most Awesome... would be to have an override folder where I could re-define a specific page while maintaining the normal update system in the background.