Custom Default Password.

Many N8N programs install using default usernames and passwords as opposed to active directory. Which is understandable.... however very insecure. If you forget to change that default password you could easily get crawled and hacked.

Solution

Create a global variable inside of Cloudron where an admin can customize the default username and passowrd for this type of application.

Example

Wordpress installs with the username admin, and password changeme
how awesome would it be if the changeme part was customized to "customdefaultpassword"

@girish said in out of space error leading to missing certs:

e-generate the nginx config for the dashboard alone. Once you have access to the dashboard, you can go to Location section of each app and click save. This will regenerate nginx config of the app

FIXED!!!

It is difficult to tell if deleting the conf files from the folder /etc/nginx/application and then restarting unbound Instructions then using systemctl restart nginx and systemctl restart box

I say that because unbound definitely was not working at at one point.
And as I remember nginx did start momentarily.

However the solution came when I deleted the corrupted zero byte private key from the folder /home/yellowtent/platformdata/nginx/cert/

When that file was deleted I was able to log in without ssl using firefox. Once in under the domains and certs section of cloudron I was able to click on Renew All Certs. That fixed SSL, and I was able to go into each program and re assign the dns settings by clicking save.

@girish

At the end of the day, I found the culpret.
Also upgraded to a bigger server.

While yes N8N will happily crash it's self - I had a few perfmance bottlenecks and for now N8N seems to be happy with the traffic. Here is a breakdown of the solution without setting up queue mode.

1. N8N uses postgress - so bumping up the RAM for N8N is not enough that just grows the list of RAM eating active processes, you also have to go bump the RAM for Postgres
2. I am using Baserow for logging and followup task Queueing. By default baserow only has 3 Auth workers which is not enough to cover heavy load. So I had to install the CLI and run the command cloudron env set --app baserow.draglabs.com BASEROW_AMOUNT_OF_GUNICORN_WORKERS=9

Conclusion - Step 1 Significantly improved the performance of N8N dropping normal RAM consumption by about 30%
Step 2 Caused active processes to complete 10X faster further reducing N8N footprint.

Now that I upgraded to 24 gigs I am only using 8

@msbt
Hi, I would like to take a look at this. I recently built my own N8N instance. And would really like to get cal.com running on cloudron.

Could you share more of your work in progress?

I am running into this problem with N8N under heavy load it will easily take down Cloudron. On the N8N website they talk about "queue mode" where one instance of N8N drives another.

It is important to have n8n using postgres or mysql
And it is important to have redis running to direct the sub instances.

beyond that... I am wondering if it is going to be possible to make this happen while using Cloudron.

I have the current problem where a load spike will bring in 2000 requests in a few minutes which reliably crashes N8N. It seems premature to upgrade the whole server just to handle a load spike.

https://docs.n8n.io/hosting/scaling/queue-mode/

Any ideas?

@mabaker @chetbaker I am working on this happy to join up and hack together on it. If you wanna hack on this with me just set a time on my calendar and we can work on it. https://short.draglabs.com/cal

@girish

Also this whole issue was caused by running out of space - I took a look at some of the other posts on out of space crashes and can tell it is a difficult problem to solve.

Supposedly there is a running out of space warning but i never got that warning.

I was thinking that a good solution for the running out of space error would involve taking the remaining space cron which calculates remaining space every 'n' minutes and integrating it over 'x' hours to arrive at time to disk full.

This could relatively accurately predict if an out of space crash is pending or imminent - and if so... do things like stop processes prevent backup (if backing up to local filesystem) etc.

Essentially

1. predict the crash with a pinch of calculus.
2. send a warning to the administrator.
3. follow a contingency to protect the sever.

Because I could imagine many ways this could happen, and my example is ONLY one way. A program can crash Cloudron I could have been copying video files, It could have been NextCloud, a spam attack on a mailserver.

@MooCloud_Matt said in trying to scale N8N:

ference is that n8n workers need access to db and redis for the queue.
And they don't work correctly (they add a lot of del

The delay is actually what I was most excited about. N8N is really happy to crash load spikes which may only last a few minutes. I was hoping that queue mode would allow me to handle 5000 requests over say... 5 minutes, and then work them over the course of an hour....

@robi

Related Feature Request. https://forum.cloudron.io/topic/7472/remote-file-access/5

It would be really awesome if rclone was implemented through the CLI enabling the synchronization of target directories and mounts.

In my specific use case I am trying to figure out a reliable way to be able to quickly edit my GRAV site. While GRAV has a GUI it was really build for the text editor.

I am trying right now to figure out how to install Rclone as a system level app to provide synchronization to my desktop computer of several cloudron folders.

If rclone was implemented in such a way that it could have access to other application's data folders that would be super awesome!

I am interested in being able to override the source code in running apps for a number of reasons.

Reason 1. I Found a bug in N8N and need to test the code edit before I submit a PR. Maybe it is just a one line change but the ability to slip in a custom hot fix would be super awesome!

Reason 2. Also an N8N example, but hard customizations may be good for me but won't necessarily get approved. You know.. changing the way an API works etc. How awesome would it be to have some granular control over the code running on your server?

Idea... Would it be possible to overwrite the source location to my own github fork? Then I would be manually pulling in new updates as opposed to relying on whatever the community publishes.

Most Awesome... would be to have an override folder where I could re-define a specific page while maintaining the normal update system in the background.

@d19dotca

Thanks for that answer the guide is perfect.

Is it possible to move postgres to an external xfs mount.

I just mounted a new drive to house large things, and postgres is the largest. I would like to move it's storage location onto that XFS mount. Is that possible?

Thanks.

@mabaker @chetbaker I am working on this happy to join up and hack together on it. If you wanna hack on this with me just set a time on my calendar and we can work on it. https://short.draglabs.com/cal

@humptydumpty said in Security Recommendations...:

for that site?

Thanks for all the replies, yes I am using wordfence. The whole story is that I had just spinned up and was working on a new website and (the big admit) Never changed the default password.

So @robi I think I just got picked up by a crawler. I caught the hack in a matter of hours, rolled to a backup and rolled passwords pretty quickly. Then I installed wordfence....

In the mean time it got me thinking... If I was a hacker and was able to get into xxx.aaa.bbb.ccc then I would try again on every port. So while it is easy to install a firewall and get monitoring on wordpress....

How do I get that monitoring for the whole server? It is a rude awakening when your VPS provider wakes you up with an unusual traffic notice....

Just had my first brush up on security... a wordpress site got hacked... what a headache!

Moving on, I am happy that Cloudron is holding up against the subsequent DDOS attack I seem to be getting pinged now over 200mm times a day, but now I need to start thinking security. Any recommendations from the community on how to beef up server security while running Cloudron?

There are many subdomains etc to think about... Maybe this security layer would need to be installed directly on the server side by side of cloudron?

@girish
The problem seems to be that on "Server A" I have to enable mailservices for the domain in question. Then when I try to send mail from Server B the

server B produces an error

{
  "ts": 1663009665363,
  "type": "deferred",
  "direction": "outbound",
  "uuid": "B17FA8D4-5997-4FE6-8636-400140195457.1",
  "messageId": null,
  "mailFrom": "<>",
  "spamStatus": null,
  "rcptTo": [
    "<no-reply@serverB.com>"
  ],
  "message": "Tried all MXs",
  "delay": 8192
}

and server A produces the error

{
  "ts": 1663009909718,
  "type": "bounce",
  "direction": "outbound",
  "uuid": "380CB958-8DD9-45DC-B9C3-13F89C1FD866.1.1",
  "messageId": "<f993a729-8533-909f-f3d0-be386c19df96@draglabs.com>",
  "mailFrom": "<no-reply@draglabs.com>",
  "spamStatus": null,
  "rcptTo": [
    "<user@serverB.com>"
  ],
  "message": "Some recipients failed: <user@serverB.com>",
  "mx": {
    "priority": 0,
    "exchange": "127.0.0.1",
    "port": 2424,
    "using_lmtp": true,
    "family": "A",
    "bind_helo": "my.draglabs.com"
  },
  "bounced_rcpt": [
    {
      "original": "<user@serverB.com>",
      "original_host": "serverB.com",
      "host": "serverB.com",
      "user": "user",
      "reason": "550 5.1.1 <user@serverB.com> User doesn't exist: user@serverB.com",
      "dsn_action": "failed",
      "dsn_smtp_code": "550",
      "dsn_smtp_extc": "5.1.1",
      "dsn_status": "5.1.1",
      "dsn_smtp_response": "<user@serverB.com> User doesn't exist: user@serverB.com",
      "dsn_remote_mta": "127.0.0.1"
    }
  ]
}

What I think is happening is that because I enabled email on Server A, Server A is automatically looking for that user's email inbox to drop the email as opposed to routing it though the internet where is would use the MX records listed in the DNS.

So the question becomes how do you log into an email server without enabling inbound email.

Or how do you convince cloudron to be happy sending email from a different subdomain? like mail.serverb.com without setting the root of server be to my.mail.serverb.com

yes confirmed

I did some tests and they way it is configured now I can email anyone other than <user>@DomainB.com and if I disable "Incoming Email" then the server A user "relay@serverB.com" cannot authenticate for the purpose of sending emails.

@subven @girish - Any Ideas on a solution here?

Custom Default Password.

Many N8N programs install using default usernames and passwords as opposed to active directory. Which is understandable.... however very insecure. If you forget to change that default password you could easily get crawled and hacked.

Solution

Create a global variable inside of Cloudron where an admin can customize the default username and passowrd for this type of application.

Example

Wordpress installs with the username admin, and password changeme
how awesome would it be if the changeme part was customized to "customdefaultpassword"

I have this situation coming up over and over - and it is annoying.
Issue: When setting up Cloudron for a client they 1. Have port 25 blocked, 2 do not currently use the other options for email, and 3. would rather not add something to their SPF records just for the sake of sending out individual password reset notifications.

So my ideal solution is to use my cloudron server to run these notifications. And for my cloudron server to send using email addresses originating from it's own domain.

So... Cloudron server A (my server) has port 25 available and a happy sending domain.

I want to subscribe Cloudron server B to send all emails from a that happy sending domain described above.

When I do this I get all kinds of errors like

1. Email address "admin.app" is not a valid alias to send emails for this user.
   and.
2. Cannot send emails for otherdomain from this domain because bla bla bla
   Or
3. You have to enable server A to receive email in order for server B to connect to it via SMTP and send emails out creating the errors above.

Anyway I would love to find an elegant solution which allows Cloudron-Server-B to send notification emails via a different domain without having to individually enable every single sending address.

@msbt
Hi, I would like to take a look at this. I recently built my own N8N instance. And would really like to get cal.com running on cloudron.

Could you share more of your work in progress?

@robi

Related Feature Request. https://forum.cloudron.io/topic/7472/remote-file-access/5

It would be really awesome if rclone was implemented through the CLI enabling the synchronization of target directories and mounts.

In my specific use case I am trying to figure out a reliable way to be able to quickly edit my GRAV site. While GRAV has a GUI it was really build for the text editor.

I am trying right now to figure out how to install Rclone as a system level app to provide synchronization to my desktop computer of several cloudron folders.

If rclone was implemented in such a way that it could have access to other application's data folders that would be super awesome!