DNS resolving problem

[nichu42]

Hi there,

I have massive problems with images not loading in Mastodon ("not available").
Reading through the logs and looking into Sidekiq, I found out that numerous host names do not resolve (anymore).

I went to the Mastodon app terminal window and tried 'dig {hostname}' with some of the host names (many S3 endpoints that host images for Mastodon instances, e.g. Wasabi) that I found in the dead jobs section of Sidekiq, with the following results:

Possible result #1:

; <<>> DiG 9.18.1-1ubuntu1.2-Ubuntu <<>> {hostname}
;; global options: +cmd
;; connection timed out; no servers could be reached

Possible result #2:

; <<>> DiG 9.18.1-1ubuntu1.2-Ubuntu <<>> {hostname}
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 61160
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;{hostname}. IN      A

;; Query time: 4 msec
;; SERVER: 127.0.0.11#53(127.0.0.11) (UDP)
;; WHEN: Wed Mar 22 09:42:21 UTC 2023
;; MSG SIZE  rcvd: 58

However, if I do 'dig {hostname} @9.9.9.9", it resolves just fine!
I can edit resolv.conf, but after re-booting, it will revert to 127.0.0.1 - is this Cloudron's unbound server?
What can I do?

[nebulon]

Yes the dns resolving will be through unbound on the system. So first make sure unbound is running fine. If not, try https://docs.cloudron.io/troubleshooting/#unbound

[nichu42]

@nebulon said in DNS resolving problem:

Yes the dns resolving will be through unbound on the system. So first make sure unbound is running fine. If not, try https://docs.cloudron.io/troubleshooting/#unbound

unbound was running.
I have added the forward-all as suggested in the doc, and now everything appears to be fine.
I retried the whole dead job list and there are no resolving issues anymore.

Can I just leave the forward-all setting, or do you suggest further troubleshooting?

[girish]

@nichu42 Is yours an internal/home server? Maybe direct DNS requests are not allowed from your network?

[nichu42]

@girish said in DNS resolving problem:

@nichu42 Is yours an internal/home server? Maybe direct DNS requests are not allowed from your network?

No, I'm on a VPS hosted by Windcloud.

[girish]

@nichu42 OK, thanks. I am not sure why the forwarding is needed. But if it works...

[nichu42]

@girish said in DNS resolving problem:

@nichu42 OK, thanks. I am not sure why the forwarding is needed. But if it works...

Alright, then I will keep it that way.

But I have a bonus question: The favorite DNS server I'd like to use (UncensoredDNS, very close to my VPS location and thus extremely quick) is only reachable through DoT or DoH. Do you know if it's possible to make this work?

I've tried to modify the forward-all.conf like this

forward-tls-upstream: yes
forward-addr: 91.239.100.100@853#anycast.uncensoreddns.org
forward-addr: 1.1.1.1@853#cloudflare-dns.com

but it didn't work (SERVFAIL).

[nichu42]

@nichu42 said in DNS resolving problem:

I've tried to modify the forward-all.conf like this

forward-tls-upstream: yes
forward-addr: 91.239.100.100@853#anycast.uncensoreddns.org
forward-addr: 1.1.1.1@853#cloudflare-dns.com

but it didn't work (SERVFAIL).

Answer to myself (in case someone else ever needs this):

It is necessary to also add the following to the conf file:

server:
  tls-cert-bundle: /etc/ssl/certs/ca-certificates.crt

so the validity of the certificates can be verified.
After that, you can add TLS upstream servers as mentioned in my previous post. I've decided for anycast.uncensoreddns.org and dns.quad9.net now.

[DualOSWinWiz]

Today, I encountered the same error again. Although this thread is closed, I wanted to share the issue I discovered and how I resolved it.

I’ve been running Cloudron on my local cloud, and until yesterday, I was using a Cisco Meraki MX. However, due to outdated equipment, I recently upgraded to a UNIFI cloud setup. Unfortunately, my IP address started appearing on the Spamhaus blacklist. When I checked the Spamhaus website, it didn’t list my IP, so I thought the issue was elsewhere.

After some troubleshooting, I found that Unbound wasn’t providing any useful information. I decided to take a closer look at the logs, where I found the following entries:

2025-02-23T05:53:25.125Z box:mail checkRblStatus: myinboxZZZlabs.com (flippedIp: 123.456.111.111) is in the blocklist of {"name":"SpamHaus Zen","dns":"zen.spamhaus.org","site":"http://spamhaus.org"}
2025-02-23T05:53:25.125Z box:mail checkRblStatus: myinboxZZZlabs.com (error: null) (txtRecords: [["Error: open resolver; https://check.spamhaus.org/returnc/pub/123.456.111.111/"]])
2025-02-23T05:53:30.178Z box:mail checkRblStatus: myinboxZZZlabs.com (ip: 111.222.333.444) blockedServers: [{"name":"SpamHaus Zen","dns":"zen.spamhaus.org","site":"http://spamhaus.org","txtRecords":["Error: open resolver; https://check.spamhaus.org/returnc/pub/123.456.111.111/"]}])
2025-02-23T05:53:33.085Z box:apphealthmonitor app health: 17 running / 14 stopped / 7 unresponsive

At this point, I shifted my attention to the new security gateway. While configuring the firewall policies, I realized that I had enabled the "Encrypt DNS via Google/Cloudron" option. After disabling this setting, the email system immediately came back online.

Solution Summary:
The root cause was the "Encrypt DNS via Google/Cloudron" option causing issues with DNS resolution and email functionality.
Solution: Disable "Encrypt DNS via Google/Cloudron" in the security gateway settings, which immediately restored the email system.
I wanted to share this here so others experiencing the same issue can benefit from the solution. I hope it helps!

[joseph]

@DualOSWinWiz do you think "Encrypt DNS" in your router mean DNSSEC ? If that is the case, an idea is to just disable DNSSEC in unbound instead of for your domain . https://docs.cloudron.io/networking/#unbound .