Cloudron Forum

I just thought I'd mention a fun use of Tailscale, which I'm unreasonably pleased with, even though it was minimal effort to do.

To start, I have Cloudron backing up to a local SSD. It's an old 2.5" that has enough space for the host backups (which are... rsync format, I think).

This weekend, I took a an old, small machine (a NUC I accumulated from years ago), installed Ubuntu 22.04 on it, plugged in an old 1TB USB HDD, and took it to an undisclosed remote location. (Read: a family member's house.) I set up my fstab to mount the USB drive by ID, so it should always come up on boot. (I also remembered to set the machine's bios so it would come on after power failures.)

I then installed Tailscale on both my Cloudron host and this aging NUC. Finally, I set up my crontab on the Cloudron host to run rclone every now and then. It clones my backup (from the SSD) to the remote, undisclosed location over the Tailscale network.

This saved me a ton of time in terms of setting up a hole in the remote router (for a secure SSH connection), as well as worrying about whether or not I have secured SSH adequately. Granted, I'm trusting Tailscale to do the right thing here, but I figure it has a better chance of being secure than me quickly hacking things together.

Although it isn't a full "Cloudron using Tailscale" story, it is nice that the default Tailscale configuration is to leave all your public networks alone. As a result, the Cloudron host can be set up to replicate backups elsewhere very quickly and easily.

@andreasdueren I've packaged one or two things for myself on Cloudron, and I took a look at Frappe/ERPnext.

First, for the thread: Cloudron does not run Docker images "as-is." Or, if you prefer, simply because a project runs in Docker does not mean that it will immediately be runnable under Cloudron. A Cloudron app needs to be packaged up so that it will "play nicely" with the control architecture that Cloudron provides. Put simply, to get that friendly Cloudron experience, some work is needed when packaging an app.

In the case of ERPnext, it has a compose file that specifies many software services. Traefik is used for routing and load balancing (I assume); Nginx fronts the service; it seems like Frappe (the API backend) is written in Python (another service). There's worker processes of several flavors, a scheduling service, a Redis cache, MariaDB (which, for porting to Cloudron, we'd want to integrate with it's built-in DB add-on), the site creator service... and a large number of storage volumes.

Cloudron does not, to the best of my understanding, support running docker-compose files. As a result, to package this, we'd have to pull all of these services into a single container image. That would take some thinking, especially since Docker "likes" to have one process per container. Or, if there is another way/it is possible, I don't personally know how to package up a multi-container Cloudron application.

The Cloudron team may have something else to say, but I thought I'd drop a note in the thread that helps explain why this app is a more complex proposition than others (perhaps) when it comes to packaging. Yes, it is open, and yes, it installs easily on a VM when you do a docker-compose up. Unfortunately, that is not the same as packaging things up to run under the Cloudron framework.

2FA with authenticator apps are, by-and-large, all using TOTPs (https://en.wikipedia.org/wiki/Time-based_one-time_password), and therefore are effectively standardized. Whether you use Google's Authenticator, Authy, FreeOTP, Keepass, Vaultwarden, or something else, it doesn't matter. Or, if you find a provider where it does matter, you might want to be concerned.

https://alternativeto.net/software/google-authenticator/?license=opensource

You can also, in many 2FA contexts, use a hardware key.

https://www.yubico.com/

which have some added benefits (and drawbacks, mostly "it's a thing you can lose). Or

https://www.crowdsupply.com/sutajio-kosagi/precursor

if you really want a serious bit of kit from an open-and-secure perspective.

In short, and with kindness: I think you're searching for a boogeyman where there isn't one. I want 2FA on every account that matters to me, and I especially want stronger authentication frameworks in my software supply chain. I want 2FA on my bank accounts, I want 2FA on my email... really, I want something that goes beyond a single, salted/hashed password everywhere.

I'm not saying you shouldn't want to self-host your code on your own stack, and only use the most libre of free software. However, I think worrying about TOTP/2FA is like worrying about the "forced" transition to HTTPS everywhere. It's actually a good thing, and it isn't a "give us all your information" play. 2FA is a smart thing to do.

That said, I'm not keen on biometrics as a second factor.

@girish Hi Girish. You got it.

I have a small IoT service hosted in the home network that adding auth/protection to would be painful. (It doesn't have a notion of auth, I don't think. So, Cloudron-as-LDAP-authority doesn't even save me.) The App Proxy feature made it easy to bounce through to it from outside, but I don't want it unauthenticated.

So, yes. You nailed exactly what I was wondering.

Someone more familiar with packaging Cloudron apps would be able to answer better than me. However, I find that whenever a docker-compose.yml is involved, it is probably hard to move the app to Cloudron.

https://github.com/AppFlowy-IO/AppFlowy-Cloud/blob/main/docker-compose.yml

In this case:

• It wants nginx. That might be avoidable, or it might be serving static assets/code for the app.
• It wants minio. This could probably be accommodated by requiring users to run minio on their Cloudron before installing this.
• It wants postgres, which might be able to be leveraged from the internal stack.
• It wants redis. Again, possibly from the default stack... I can't remember.
• gotrue is an auth component from supabase. This will need its own container, and may (or may not) play nice with the SMTP/OAuth running on Cloudron.
• appflowy_cloud is the hosted app. It wants its own container, and configuration information for all of the services included.
• admin_frontend has its own Dockerfile. I haven't looked. Looks like more things.
• ai. I have no idea. It looks like it wants some kind of OpenAI. This is getting heavy in terms of resources.
• appflowy_history is... another Dockerfile. Looks like a rust application that has been Dockerized.

The problem, I think, is that Cloudron assumes/is structured such that applications run as single containers. The compose is suggesting that this application has a number of independent components. Perhaps those could be bundled up/run separately... but, it might be a real trick to make work.

This isn't to say it isn't possible, but that's what I see that needs to run, and it isn't clear to me that this is an easy app architecture to move over to Cloudron. YMMV, etc.

Flatpack and Appimage formats do not provide the kind of process and filesystem isolation that Docker does. I would not be comfortable running run a service like Cloudron if it allowed running things other than well-defined/managed containers.

Yes, there are ways to break isolation and escalate out of a Docker container, but flatpack apps can, essentially, write anywhere on my system. Updating the ecosystem is also much more difficult; by comparison, it is possible to auto-build Docker containers from source and track recent updates/manage timely security vulns in a way that flatpack/appimage packages do not/rarely do.

OK.

This was very exciting.

I read some documentation. Specifically, https://docs.opnsense.org/manual/how-tos/nat_reflection.html.

Once I slowed down, undid all the weird thrashing I did with various DNS shortcuts for route domains internally/directly (e.g. changing my unbound config, or creating aliases for my domain), and instead read the documentation for both reflection and hairpin NAT in OpnSense, I was good to go.

Thank you for joining me on this journey where I create noise on the forum and discover that, by reading the manual, I can solve my own problems.

@humptydumpty I run OPNsense in front of Cloudron. I'm not doing anything fancy with it, but it does live between the world and my self-hosted Cloudron instance.

I have no idea what would happen if the machine was DDoS'd. I'm pretty sure it would fall over. At this point, I'm just excited that I have cron'd backups locally and to offsite.

Hi all,

This thread is intended to document my reading of other support tickets, and what I've done in an attempt to bring my cloudron back.

BLUF: I succeeded, but I don't like how I did it. Ultimately, I had to edit resolv.conf. This ticket can likely be closed.

Initial problem

My experience so far is similar to this thread: https://forum.cloudron.io/topic/12294/autoupdate-cloudron-aborts/9

I pushed the button for the update to 8.0.3. Unfortunately, I did this shortly before leaving the house for two days. I thought "everything will be fine."

Return from trip

I came back to find apps and services down. (Unbound was down, for example.) I did a reboot to see what would happen. ("Turn it off and turn it back on again.") The services came back, but the apps did not.

Looking at one of the apps, I see this error:

An error occurred during the configure operation: Network Error: Network error downloading icon : getaddrinfo EAI_AGAIN api.cloudron.io

Explore the forum

I tried some possibly obvious fixes from the forum.

Update unbound settings

Following this page:

https://docs.cloudron.io/networking/#dns

I added a file to /etc/unbound/unbound.conf.d called override.conf, and it contains the following:

server:
    val-permissive-mode: yes
forward-zone:
    name: "."
    forward-addr: 10.0.0.1

My cloudron lives behind an OpnSense instance which serves as my router, and it is at 10.0.0.1. If you are following this, your DNS server is probably not 10.0.0.1. So, if you're also trying to fix something similar... you cannot just "plug and play" the values I use, but you might be able to follow the process.

After that change and an unbound restart, I could do:

host www.cloudron.io 127.0.0.150

and it reported back

Using domain server:
Name: 127.0.0.150
Address: 127.0.0.150#53
Aliases: 

www.cloudron.io has address 165.227.67.76
www.cloudron.io has IPv6 address 2604:a880:800:10::b66:f001

This is good. I then tried to configure a failing app, and it still would not talk to api.cloudron.io. So, the problem did not go away.

Update netplan

I still had a DNS problem. Based on this thread:

https://forum.cloudron.io/topic/12266/auto-update-to-8-0-3-systemd-resolved-empty-response/6

I considered the possibility that the issue might be deeper down. I went into my netplan, and modified my configuration. It was set for straight DHCP. My netplan now reads as:

network:
  version: 2
  renderer: networkd
  ethernets:
    enp0s31f6:
      dhcp4: true
      dhcp6: false
      dhcp4-overrides:
        use-dns: false
      nameservers:
        addresses:
          - 10.0.0.1
          - 1.1.1.1
          - 8.8.8.8

This should, I think, set my router as the first DNS option, with Cloudflare and Google as fallbacks.

I did a netplan apply and then I rebooted the machine (shutdown -r now), and everything came back. I also enabled DNS query forwarding in OpnSense at this point, just in case. I don't know that it mattered.

host www.cloudron.io 127.0.0.150

works.

curl https://releases.cloudron.io/versions.json

does not work. So, curl is not picking up the DNS; it seems to be looking to the default :53, which I suspect is not unbound, but instead... something else. dig has the same problem.

 dig google.com
;; communications error to ::1#53: connection refused
;; communications error to ::1#53: connection refused
;; communications error to ::1#53: connection refused
;; communications error to 127.0.0.1#53: connection refused

; <<>> DiG 9.18.28-0ubuntu0.22.04.1-Ubuntu <<>> google.com
;; global options: +cmd
;; no servers could be reached

Edit /etc/resolv.conf

I don't like the solution I ended up with.

Based on this thread:

https://forum.cloudron.io/topic/12286/dns-failure/8

I decided that my empty resolv.conf was a problem. (I don't know why it was empty, and I don't like editing a file that claims it will be overwritten.)

I entered

nameserver 10.0.0.1

(again... for me is a good value.)

Reboot

After that change, I was able to manually configure/restart two apps. From there, I decided the fastest way to bring everything back (and confirm things might be good in general) would be to do a reboot from the admin panel.

This is not fix all of the apps. I had to retry the configure task on each, and Restart App for each.

Conclusion

At this point, I now have all my apps back. Cloudron claims v8.0.3 (Ubuntu 22.04.1 LTS), which is the most recent version.

Platform Version: v8.0.3 (Ubuntu 22.04.1 LTS)
Vendor: Dell Inc.
Product: OptiPlex 7040
CPU: 4 Core "Intel(R) Core(TM) i5-6500T CPU @ 2.50GHz"
Memory: 33.51 GB RAM & 8.59 GB Swap
Uptime: 9 minutes

I think this ticket can be closed, but given that a number of updates to 8.0.3 seem to have had DNS issues, I am going to submit this here. It may also be that I fixed the problem incorrectly.

Hi all,

I once, long ago, ran an exim mailserver with a colleague. At some point, we got zero-day'd, and I decided that running a mailserver was less fun than I thought. I've never looked back.

I maintain a domain on an external provider almost entirely for the email forwarding. That is, I have a domain and addresses that I only forward on to other email hosts (e.g. Gmail).

I could move that domain to my Cloudron. At that point, I would be putting all of my personal infrastructure on a box in my basement, and be relying on it for my most important piece of comms infrastructure. And, I know I'd need to actually test my backup and restore process at that point, because I really couldn't afford to have an outage take out my family's email for (say) days.

Do people use Cloudron for production mail? (I mean, I assume they must.) Are there any concerns? Gotchas? Are there other paths people have walked? I've tried experimenting with Cloudflare's email forwarding solution, but was unable to get it to work reliably (a number of months ago).

Many thanks,
Matt

I would like to politely ask for a bit more info around cert updates, please.

I received an email yesterday from LetsEncrypt saying I had one day left on my cert renewal for the my domain in my Cloudron instance. It is not spam or a scam; I believe this email was legitimately from certbot.

I pushed the "renew certs" button in the control panel, but... did it work? How would I know? I read the logs, and played with openssl; of course, the latter is an external measure, and may be confused by my Cloudflare proxy. I think the update worked... but, visual log parsing is the hard way to figure out if a complex process succeeded. Was that null result buried in a JSON log an indicator of success? It was late, I was tired, and perhaps tomorrow my certs will fail.

1. It would be nice if the dashboard/control panel told me, after pressing the button to update certs, if the system thought it succeeded.
2. It would be nice if that same panel area are simply told me "Certs last renewed on <date>, expire on <date>, and the next cert update will run on <date>."

Cheers,
Matt

Understood. Thank you.

I'll signal explicitly via @ if I have questions. Otherwise, consider this a documentation thread that helps us understand whether this is "user error" (as in "I have a poorly configured LAN"), or something that can improve the product. And, I'll try and capture something a bit more linear that makes clear exactly what I'm trying/doing, which would help me regardless.

This is not a source of stress for me; I do want to figure it out, but that will take the time it takes (given work, etc.).

@girish No, I don't think I have any particular DDoS protection configured in OPNsense. However, this conversation makes me curious to investigate it.

I run the DNS through Cloudflare, which... may or may not provide me with some protection. (I'm not being cagey, so much as haven't really dug in to understand how/if Cloudflare protects my Cloudron domain/subdomains.)

I'm happy to experiment with OPNsense configuration, if it's a space of question. However, it serves more for internal protection than external (at the moment). I mostly use it to partition the Cloudron machine off from the rest of the house via VLAN as a "just-in-case" measure. It keeps Cloudron separate from dumb internet lightbulbs, which I consider a kind of mutual protection.

@nebulon Many thanks, @nebulon and @girish . The concern wasn't so much that I could not figure out what the status of my certs were external to Cloudron, but more that it would be nice if the area of the dashboard regarding certs would, as a matter of course, just say "You have 47 days remaining, and Cloudron should automatically update your certs in 17 days."

And, if I do mash the button to manually run a cert update, it would be nice to get a response in the dash that says "Success! New certs will expire in 90 days!" (Or, whatever it would say.)

I was mostly surprised that I got a certbot email saying I only had one day left, making me wonder what was up. (I did do a domain registration move at some point, and possibly other things that could have somehow upset the automatic update process. So, this isn't a bug report.) Not having a simple UI response to the act of hitting "update certs" (and instead being dumped into the log) is all I'm poking at.

I don't know how long my personal instance has been running (a month or two now), but it has been a joy. Thank you.

Assuming this is just for you, and not that you want to build out a hosting infrastructure for many people...

Not quite what you're looking for, but you could

1. Render the 11ty site locally.
2. Push it to a LAMP stack or similar on Cloudron.

https://forum.cloudron.io/topic/4042/beginner-s-guide-hugo-gitlab-ci-surfer

suggests a path where you're hosting both Gitlab and runners, which feels heavy. I think Nebulon wrote about this:

https://forum.cloudron.io/topic/1118/hugo-cms/3?_=1717342765687

with Hugo. He used Surfer

https://docs.cloudron.io/apps/surfer/

which would, potentially, do everything you need. You could do your build, and then push the resulting build outputs (_site or similar; I forget what 11ty spits out) to your server using the Surfer CLI.

I've done this for many sites over the years; typically, it's been a recursive scp to sync the build output to the remote host. I've wondered about a setup like you describe, but ultimately, I'm just as happy:

1. Having a git(tea/lab) instance that hosts my code
2. Doing the test and build locally.
3. Taking the resulting build product and pushing it to the web host.

The number of moving pieces to connect up Gitlab/Netlify (now Decap) is more than I would want to do on any given evening, but your mileage may vary. I've been wondering about moving a domain to cloudron, and all it has is one static website. This might be the path I go (that is, use Surfer on Cloudron to be my webhost).

This is all good.

@luckow , @Kubernetes , thanks for the context. The... difference between a data center is mostly immaterial, I think. I mean, unless you consider that there is no power infrastructure in my basement, the ethernet cables are tacked to the ceiling, and there's no redundancy... OK. So, it's a little different. That's a separate issue, though, from the question of "Cloudron as my mail solution." So, thank you for the +1s.

@fbartels , @Dave-Swift , and @BrutalBirdie , I think you've all raised good questions.

• How will I send mail out? Do I relay? If so, what service?
• Will my ISP allow port 25 all the way down? I don't know. I know they're letting :80 and :443 in, because I'm hosting Cloudron at the end of my fiber connection. But, that doesn't mean they're not blocking :25.

Many thanks all. Good considerations.

@jdaviescoates Yep. That was it.

I will look at the source for App Proxy, and look at adding it/making a pull request. (Unless someone beats me to it.)

A huge +1 to what @girish said: you and your friends need to think about what it means to secure yourself against a hostile nation-state.

(You might just strike the word "hostile," and leave "nation-state" unadorned.)

https://revealnews.org/article/how-to-stay-safe-online-a-cybersecurity-guide-for-political-activists/

https://antigravitymagazine.com/feature/digital-security-for-activists/

and other similar articles (look for authoritative sources in this space where possible -- those seem "OK" but are only a start) are going to help you game out just how paranoid you need to be. And in this case, "paranoid" means "what do we have to do to keep people out of jail and, possibly, alive?" It is a level of paranoia I do not typically ever operate at, and I suspect, neither do you. (That's not a slam... just that most of us don't operate there on a day-to-day basis.)

For a lighter (but still not really wrong) take, James Mickens is always a potential inspiration:

https://www.usenix.org/system/files/1401_08-12_mickens.pdf

Although it is a light-hearted take, Mickens is getting at just how complex cybersecurity is. You're exploring a space where the consequences of failure/poor choices are going to cost your friends dearly. This is more than just a "simple nginx config change" that you need to make in order to create a secure communications space for people who might be targeted by their home country's internal police/intelligence forces. You need to think about your threat vectors front-to-back, as this is about more than just the tech. It's operational security from the start to end.

Or: XKCD's comic "Security" now applies:

https://xkcd.com/538/

If your solution is susceptible to someone being beaten with a wrench to yield enough secrets and access to unlock all of the comms and, in doing so, expose everyone else in the comms ring, you're doing it wrong.

@tomw Apologies; I wasn't trying to suggest you shouldn't do this. I was only trying to emphasize that there is an entire system/chain that leads to your server.

You might have:

• The nation-state working in tandem with local (or, are they state-owned?) ISPs to implement man-in-the-middle cert attacks, so that attempts to securely connect to your server are actually plain-text.
• The nation state, working with ISPs to compromise/log all traffic through DNS servers.
• ...

https://www.cisa.gov/news-events/alerts/2015/04/30/securing-end-end-communications

is an article that speaks to some of the kinds of things that you might have to do to begin securing end-to-end communications.

Ultimately, I really don't know. I'm just suggesting---YMMV, etc.---that this sounds like something with high stakes.

I wish you and your colleagues all the best of luck.

PS. https://www.cjr.org/tow_center_reports/guide_to_securedrop.php looks interesting as well. Again, it doesn't apply directly to your case, but speaks to the broad spectrum of design considerations that go into architecting and delivering secure systems, where "systems" means "a combination of technology and people."

Hi all,

Reading through some forum threads and in particular, #11 on a thread re: OIDC from @nebulon

https://forum.cloudron.io/topic/10293/cannot-login-after-switch-to-oidc/11?_=1699652776020

I just tried to install Outline. It only does login via Cloudron OIDC. This is the first app I have tried that uses that login mechanism. It would not let me log in; it times out.

My concern is that I have a DNS issue in my home hosting setup. (I have suspected this before.) The dynamic DNS is handled by Cloudflare, I have an OpnSense firewall/router in front of Cloudron, and I wouldn't doubt that I have a configuration issue that breaks lookups for Cloudron's base hostname (e.g. my.home.net and friends) when they're handled inside my network.

Or: external requests are fine, but I have run into issues before where requests from Cloudron to Cloudron seem to fail or timeout, and I am guessing it is because of DNS resolution behind the OpnSense box.

Without asking someone to help me debug my particular configuration, I'm wondering if anyone might be able to throw me a bone as to where I should go digging... (definitely mixing the dog-chasing-bone metaphors there)... before more of Cloudron switches to OIDC, and I can't log in to anything.

Many thanks for any pointers that might spring to mind.