Aha, you were right actually.. Noticed some other quirks around DNS (most noticeably, it was repeatedly failing when I was ssh'd in). Turns out that unbound isn't very happy when something else above it hijacks DNS traffic (which I'm doing to filter our junk).
Quick fix for now was to add a forward-zone to unbound, but I wonder what the right way to handle - and detect - this is...
Yeah, it's green. I was fiddling with switch config a few days ago which I think was the cause in this case.
I noticed at some point an app I don't use very often failed to update. I noticed this because it wasn't running, and was in error state. In the repair section of settings, I saw this:
The update operation failed with the following error:
Network Error: Network error downloading icon : getaddrinfo EAI_AGAIN api.cloudron.io
My guess is that I had a network outage at some point at just the wrong time while the app was trying to update, and this then caused it to hang in that error state permanently.
Perhaps this sort of transient failure should (automatically) schedule a retry later on?
@girish yeah, well, since when has following the rules ever been fun? 
I would guess that they say that because if your actions misbehave, or want to, they can interfere with the running gitea instance, so yeah, a separate host would be nice. but I don't know if cloudron has the ability to provision multiple containers for a single app?
tried to play with it a bit. enabled actions in the ini, as documented here:
https://blog.gitea.io/2022/12/feature-preview-gitea-actions/
installed a runner binary from the link here:
https://gitea.com/gitea/act_runner (just to try get up and running quicker..)
placed it into /tmp/ on the gitea container, chmod +x, ran it. could successfully register the runner.
actually executing actions seems to fail, because by default, it wants a docker connection. looks like this can be hacked out if one builds a runner by hand:
... obviously, at the price of having no docker available to the actions, which mean some won't work.
but the docker hurdle aside, this is so easy to set up it that it might be worth exploring as part of the app's package..
looks really exciting, if it works well 
@nebulon said in OpenHAB not starting after installation:
gosu cloudron:cloudron /app/code/runtime/bin/karaf daemon
Weird.. When I ran /app/pkg/start.sh in recovery mode, I got this:
==> Ensure directories
==> Changing ownership
==> Starting OpenHAB
Killed
When I started the last command by hand, it actually seemed to run okay (at least, no output, and it didn't get killed...
I had a hunch that maybe it was getting OOM killed, and sure enough, I raised the memory allocation to 512m, and it now seems to start successfully, so maybe the default allocation (256m I think it was?) is just a little too low to start, sometimes?
I've installed it, but in the dashboard, it stays in state "Starting..." and the web UI never comes up. Running Cloudron v7.3.4 on 20.04.1 LTS.
Logs says this (in a loop), unfortunately, without more details:
Dec 12 22:20:28 ==> Ensure directories
Dec 12 22:20:28 ==> Changing ownership
Dec 12 22:20:28 ==> Starting OpenHAB
Dec 12 22:20:34 ==> Ensure directories
Dec 12 22:20:34 ==> Changing ownership
Dec 12 22:20:35 ==> Starting OpenHAB
Dec 12 22:20:42 ==> Ensure directories
Dec 12 22:20:42 ==> Changing ownership
Dec 12 22:20:42 ==> Starting OpenHAB
Dec 12 22:20:51 ==> Ensure directories
Dec 12 22:20:51 ==> Changing ownership
Dec 12 22:20:51 ==> Starting OpenHAB
@girish said in Incorrect error when removing an obsolete alias for a service:
Did you happen to set this up manually?
Hmm, I don't actually know. It's not impossible I guess...
Good to know though, I'll work on cleaning DNS up
For the record, it did actually remove the alias, so I think the only confusion here is the unexpected extra presentation of an error message.
I had an app hosted at, let's say, "foo.bar.com". It had an alias for "barfoo.net", which was an old domain that subsequently has expired. Today, I tried to remove the "barfoo.net" alias by just trashing it and clicking save, but got an error:
An error occurred during the location change operation: Already Exists: DNS CNAME record already exists
The CNAME that already exists is for "foo.bar.com". It doesn't seem correct that this is an error, since the CNAME it is complaining about was already set up by Cloudron. All I wanted to do was to remove the alias, not also change the location of the service.
(Running Cloudron v7.3.2, using Hetzner DNS if it matters)
Nothing that mentioned the _ (wildcard) cert at least, which is part of why I'm stumped. I don't know what is creating it, or where to look.
So, I still got more cert warnings despite moving the app locations around, and removing the dashboard config, so I decided to try be a bit more brutal about it. I removed the wildcard cert:
rm /home/yellowtent/platformdata/nginx/cert/_.xxx.*
Restarted box, and everything seemed OK. Then I triggered Renew All Certs again, and ... ended up with a new wildcard cert! That doesn't make sense to me... Is there something I can look at that would explain why it wanted to create that? Or some proper way I can nuke the DNS configuration from orbit so it ends up sensible again?
@girish Hmm, one remaining problem... What do I do about my dashboard, which is also affected by the same problem? Would changing its location temporarily (and then switching back) fix it?
@girish Hmm, that didn't match the behaviour I saw. I ended up with two separate SPF records in Hetzner. One that I had already set up from my own DNS setup, and one added from Cloudron - which didn't contain the content of my own record. Thankfully it wasn't difficult to fix, once I read up on it.
@girish I switched from wildcard over to programmatic (via Hetzner). I didn't update individual apps after that change, but I guess I can try that and see what happens...
So as I discussed a while ago, I did a DNS migration (https://forum.cloudron.io/topic/7429/how-to-do-a-smooth-dns-migration/2) from a manually updated wildcard, to Hetzner. For the most part, this has gone smoothly, but I seem to be running into a corner case with Lets Encrypt certificates. I've started getting upcoming expiry warnings for a bunch of domains now. I tried to force renewal, which gave me the following logs (for each service on subdomain.example.com)...
Aug 15 12:01:04 box:tasks update 6255: {"percent":76,"message":"Ensuring certs of xxx.subdomain.example.com"}
Aug 15 12:01:04 box:reverseproxy ensureCertificate: xxx.subdomain.example.com certificate already exists at /home/yellowtent/platformdata/nginx/cert/_.subdomain.example.com.key
Aug 15 12:01:04 box:reverseproxy expiryDate: /home/yellowtent/platformdata/nginx/cert/_.subdomain.example.com.cert notAfter=Oct 26 11:00:56 2022 GMT daysLeft=72.04156950231481
Aug 15 12:01:04 box:reverseproxy providerMatchesSync: /home/yellowtent/platformdata/nginx/cert/_.subdomain.example.com.cert subject=CN = *.subdomain.example.com domain=*.subdomain.example.com issuer=C = US, O = Let's Encrypt, CN = R3 wildcard=true/true prod=true/true issuerMismatch=false wildcardMismatch=false match=true
This looks okay, in theory, but then at the end I see the following:
Aug 15 12:01:04 box:reverseproxy expiryDate: /home/yellowtent/platformdata/nginx/cert/xxx.subdomain.example.com.cert notAfter=Sep 3 11:00:56 2022 GMT daysLeft=19.041567395833333
And the daysLeft here seems to match up with the mail warnings I'm getting...
So they don't seem to be renewing properly... Is there something I can do to force a renewal? And is this some kind of a bug/unhandled edge case in Cloudron, perhaps caused by the DNS provider switch?
@nebulon Hmm, okay. Been way too long since I've looked at running an email setup, so I had no idea! Good to know.
There is still a bit of a problem, though, in that it seems like the record creation doesn't play very well with other existing SPF records. I had an existing SPF record for my main domain, so this meant I ended up with two of them when Cloudron pushed its one. I've now merged them together, so I guess I'm fine, so long as Cloudron doesn't try install it again...
Not sure if this can be a problem or not...
I ended up doing pretty much exactly that. Configured DNS at Hetzner as it should have been, swapped NS records, and prayed. Then I let Cloudron loose on it too. Guess this can be considered solved.
I recently switched to using managed DNS for a domain. The domain in question is configured to use outbound-only mail via an external SMTP server, as I have mail already hosted elsewhere. Isn't it wrong that I got an SPF record automatically created for this domain, then?