DNS domain expiration monitoring

[mehdi]

@potemkin_ai said in DNS domain expiration monitoring:

@mehdi even if, it’s better than nothing

I honestly don't believe so : a check that has more false-positives than true positives is not very useful... It would worry people about nothing all the time.

[potemkin_ai]

@mehdi we definitely have a different perspective here.

I believe, that giving me a chance to prevent service shutdown for 72 hours for the outside world is a very valuable thing.

I also like the idea of fetching fresh data from whois on expiration data and letting me to correct whois server or the expiration data itself.

There is little I can do to turn you into my faith, I'm just saying what I see and experience.

[d19dotca]

@potemkin_ai I'd be in favour of Cloudron adding in some nice-to-have like a domain expiry notification if it has that access but also only if it's feasible to include in the first place. I wouldn't want that taking away hours from the Cloudron team.

Ultimately though as I see it, the trigger for this feature request seems like more of an issue with notifications from the registrar which is well outside of Cloudron's scope.

Additionally the result of losing a domain can easily be avoided by enabling auto-renew. I recognize Auto-Renew isn't for everyone (some people don't like having credit cards on file for instance) but it's used by the vast majority and is quite literally designed to avoid this very issue. If the risk of losing a domain is too high, then auto-renew should be used.

Also, a simple Calendar entry can help. I have my Calendar with entries with 2-week reminders for all my domains just in case so I can double-check that it has auto-renew enabled and if not then I can set it correctly or renew it manually right then and there.

We have to take responsibility with ownership of domains and ensure these things don't happen and there many ways to do that ourselves (two most common mentioned above). I don't believe Cloudron should be the vessel for that task personally, but hey if it's an easy add then totally cool to see.

[ruihildt]

@d19dotca Depending on countries, credit card can actually be pretty hard to obtain for a lot of people.

But I agree this is out of scope, unless there's an easy adn reliable way to figure it out.

[potemkin_ai]

@d19dotca I can work this around, trust me. But I used to let tools to assist me, especially, since there is a very easy way to do that - described above.

[colonelpanic]

If this is such a big issue and you can't pay attention to the emails from your registrar to renew your domain then transfer it to a registrar that will auto-renew the domain for you.

[potemkin_ai]

@colonelpanic thank you for the advice; it's just that I didn't look for one, sorry.

But, to recap: as I mentioned earlier, when you start managing your client's domains and earn money with Cloudron, things get complicated.

[d19dotca]

@potemkin_ai said in DNS domain expiration monitoring:

when you start managing your client's domains and earn money with Cloudron, things get complicated.

That is exactly what I do though (and I suspect many others in here), and have never had a problem with domain expirations in about 12 years of managing them for a couple dozen customers. It makes me wonder if the issue for you comes down to the registrar you're using or your own administration techniques that may need to change or be tweaked to improve managing domains for customers. I don't find it complicated at all, personally, but maybe that's just down to what I do.

What I do that may help you:

• Generally aim to use one domain registrar if possible, or at least as few as possible to allow support for them to be easier to use and some registrars even give small discounts after 10 domains are registered with them for example
• Setup all domains for auto-renewal enabled
• I bill customers annually for hosting which includes their domain name and it's always for the next calendar year. So I'll bill in late November 2022 for the service calendar year of 2023 for example, meaning that if they choose not to renew services then I simply disable the auto-renew feature on their domain so it doesn't accidentally charge my card when it's due in 2023 (and if I forget, I'm only out about $20 CAD which is really just a cost of doing business as it would be my own fault if I forgot to disable auto-renew)
• I add domain expiry reminders to my calendar with an alert 2 weeks in advance for me to double-check everything ahead of time (once it's all set though it's basically set-and-forget, but nice to have the time to double-check if needed)
• I remind my customers periodically (about once or twice a year) to ensure they want the same domain name going forward and if they want a different one (or even just a different TLD for example) so that I can plan accordingly if they desire any changes to be made for the next year ahead

I'd still enjoy seeing Cloudron assist if possible (after all, technology is meant to help us not harm us), but I also recognize that's really outside of their scope of work at least IMO when domains are all registered and renewed and generally managed outside of Cloudron, and would hate to see hours wasted on it when so many other things are very highly requested and more inside the scope of what Cloudron is meant to be responsible for. If it's an easy thing to add though, then cool. Hopefully the advice above helps you out in the meantime though.

[potemkin_ai]

@d19dotca thank you - I can't change my customers, so I was looking for the tool that could help to avoid quite an outage I had.

[rmdes]

@potemkin_ai Why at cloudron?

my setup :
I get my domain renewal notices from my registrar but for this to work flawlessly,
you can't use a cloudron email to get the alerts, it needs to be an out of cloudron context email (gmail etc..)
in my case it's dedicated to this & billing of domain/vps's, so I know every time there is an email there it's important and need to be taken care of.

When everything is smooth, I usually auto-forward these type of emails to my associates, so I'm not the only one getting it or not and since the mailbox getting them is outside cloudron it's very rare to miss something.

On top of that, at my registrar, I set auto-renewal for all my domains, why set manual if you know in advance you need that domain Up for the foreseeable future ?

When the domains is owned by the client, I request a technical access to their registrar and configure auto-renewal and their payment with them so that it's out of the way, if the client insist on doing it on it's own, it's their responsibility to renew and face downtime because of domain expiration.

[potemkin_ai]

@rmdes because when app hosted on cloudron stops working, cloudron is the first thing I check.

I don't believe e-mail notifications is good, but a kind of red line at the top of the cloudron if domain is about to expire seems like a good thing to do.

Especially, if domain is handled by a customer (like it should) and cloudron and linux stuff is outsourced.

[rmdes]

@potemkin_ai I understand your point, maybe a temporary fix would be to use something like this https://github.com/nixcraft/domain-check-2 (but you don't want email notif)
I wish I could have something similar but able to send a webhook where I want (N8)

[potemkin_ai]

@rmdes thank you; my ideal case would be for cloudron to monitor this, giving a fix for malformed whois queries - we even made a simple algorithm for that somewhere at the top of the topic.

webhooks (to n8n / integromat / etc) would be just fantastic, indeed!

[rmdes]

@potemkin_ai in my use case, webhook only would be even better, since I have space tuned in my Matrix server where other alerts end up

might be a cool fork to do on this
https://github.com/nixcraft/domain-check-2 and since it's bash, adding a webhook here would be trivial, maybe a fun exercise for next weekend for me

[potemkin_ai]

@rmdes that's exactly my thought as well - webhooks are really nice, as they let every user to set up things and notifications the way they want and I do use work chat as well (just self-hosted Mattermost), so it would be really convenient.

For the script - I don't like (and don't have now) bare Linux servers running, so script could be Ok, but I would rather use an n8n node for that - easier to configure, update and get notifications from, if that can't be done out of the box on Cloudron - which is always the best thing, as it helps me to keep things on radars without thinking about them.

[d19dotca]

@potemkin_ai How about this free tool (free for 35 domains): https://domain-monitor.io/pricing/ - will let you know when domains are about to expire.

[robi]

It's not a far reach to request one of the OSS uptime monitoring apps we have to add domain monitoring as well.

[potemkin_ai]

@d19dotca thank you, I'm sure there are tons of tools, but my idea was for 1 tool I use to take this monitoring task on itself, which means I don't have to have one more task in my head, which is the goal of every automation, in my understanding.

[d19dotca]

In case Cloudron team decides to implement this, I just noticed that MainWP is creating an extension for this, and they seem to document all their sources for the various WHOIS data for each TLD they support here: https://kb.mainwp.com/docs/mainwp-domain-monitor-extension/

Anyone who has MainWP... feel free to use that extension too in the meantime.

In case the sources go away later, here it is for posterity sake:

"ac" => "whois.nic.ac", // Ascension Island.
// ad - Andorra - no whois server assigned.
"ae" => "whois.nic.ae", // United Arab Emirates.
"aero"=>"whois.aero",
"af" => "whois.nic.af", // Afghanistan.
"ag" => "whois.nic.ag", // Antigua And Barbuda.
"ai" => "whois.ai", // Anguilla.
"al" => "whois.ripe.net", // Albania.
"am" => "whois.amnic.net", // Armenia.
// an - Netherlands Antilles - no whois server assigned.
// ao - Angola - no whois server assigned.
// aq - Antarctica (New Zealand) - no whois server assigned.
// ar - Argentina - no whois server assigned.
"arpa" => "whois.iana.org",
"as" => "whois.nic.as", // American Samoa.
"asia" => "whois.nic.asia",
"at" => "whois.nic.at", // Austria.
"au" => "whois.aunic.net", // Australia.
// aw - Aruba - no whois server assigned.
"ax" => "whois.ax", // Aland Islands.
"az" => "whois.ripe.net", // Azerbaijan.
// ba - Bosnia And Herzegovina - no whois server assigned.
// bb - Barbados - no whois server assigned.
// bd - Bangladesh - no whois server assigned.
"be" => "whois.dns.be", // Belgium.
"bg" => "whois.register.bg", // Bulgaria.
"bi" => "whois.nic.bi", // Burundi.
"biz" => "whois.biz",
"bj" => "whois.nic.bj", // Benin.
// bm - Bermuda - no whois server assigned.
"bn" => "whois.bn", // Brunei Darussalam.
"bo" => "whois.nic.bo", // Bolivia.
"br" => "whois.registro.br", // Brazil.
"bt" => "whois.netnames.net", // Bhutan.
// bv - Bouvet Island (Norway) - no whois server assigned.
// bw - Botswana - no whois server assigned.
"by" => "whois.cctld.by", // Belarus.
"bz" => "whois.belizenic.bz", // Belize.
"ca" => "whois.cira.ca", // Canada.
"cat" => "whois.cat", // Spain.
"cc" => "whois.nic.cc", // Cocos (Keeling) Islands.
"cd" => "whois.nic.cd", // Congo, The Democratic Republic Of The.
// cf - Central African Republic - no whois server assigned.
"ch" => "whois.nic.ch", // Switzerland.
"ci" => "whois.nic.ci", // Cote d'Ivoire.
"ck" => "whois.nic.ck", // Cook Islands.
"cl" => "whois.nic.cl", // Chile.
// cm - Cameroon - no whois server assigned.
"cn" => "whois.cnnic.net.cn", // China.
"co" => "whois.nic.co", // Colombia.
"com" => "whois.verisign-grs.com",
"coop" => "whois.nic.coop",
// cr - Costa Rica - no whois server assigned.
// cu - Cuba - no whois server assigned.
// cv - Cape Verde - no whois server assigned.
// cw - Curacao - no whois server assigned.
"cx" => "whois.nic.cx", // Christmas Island.
// cy - Cyprus - no whois server assigned.
"cz" => "whois.nic.cz", // Czech Republic.
"de" => "whois.denic.de", // Germany.
// dj - Djibouti - no whois server assigned.
"dk" => "whois.dk-hostmaster.dk", // Denmark.
"dm" => "whois.nic.dm", // Dominica.
// do - Dominican Republic - no whois server assigned.
"dz" => "whois.nic.dz", // Algeria.
"ec" => "whois.nic.ec", // Ecuador.
"edu" => "whois.educause.edu",
"ee" => "whois.eenet.ee", // Estonia.
"eg" => "whois.ripe.net", // Egypt.
// er - Eritrea - no whois server assigned.
"es" => "whois.nic.es", // Spain.
// et - Ethiopia - no whois server assigned.
"eu" => "whois.eu",
"fi" => "whois.ficora.fi", // Finland.
// fj - Fiji - no whois server assigned.
// fk - Falkland Islands - no whois server assigned.
// fm - Micronesia, Federated States Of - no whois server assigned.
"fo" => "whois.nic.fo", // Faroe Islands.
"fr" => "whois.nic.fr", // France.
// ga - Gabon - no whois server assigned.
"gd" => "whois.nic.gd", // Grenada.
// ge - Georgia - no whois server assigned.
// gf - French Guiana - no whois server assigned.
"gg" => "whois.gg", // Guernsey.
// gh - Ghana - no whois server assigned.
"gi" => "whois2.afilias-grs.net", // Gibraltar.
"gl" => "whois.nic.gl", // Greenland (Denmark).
// gm - Gambia - no whois server assigned.
// gn - Guinea - no whois server assigned.
"gov" => "whois.nic.gov",
// gr - Greece - no whois server assigned.
// gt - Guatemala - no whois server assigned.
"gs" => "whois.nic.gs", // South Georgia And The South Sandwich Islands.
// gu - Guam - no whois server assigned.
// gw - Guinea-bissau - no whois server assigned.
"gy" => "whois.registry.gy", // Guyana.
"hk" => "whois.hkirc.hk", // Hong Kong.
// hm - Heard and McDonald Islands (Australia) - no whois server assigned.
"hn" => "whois.nic.hn", // Honduras.
"hr" => "whois.dns.hr", // Croatia.
"ht" => "whois.nic.ht", // Haiti.
"hu" => "whois.nic.hu", // Hungary.
// id - Indonesia - no whois server assigned.
"ie" => "whois.domainregistry.ie", // Ireland.
"il" => "whois.isoc.org.il", // Israel.
"im" => "whois.nic.im", // Isle of Man.
"in" => "whois.inregistry.net", // India.
"info" => "whois.afilias.net",
"int" => "whois.iana.org",
"io" => "whois.nic.io", // British Indian Ocean Territory.
"iq" => "whois.cmc.iq", // Iraq.
"ir" => "whois.nic.ir", // Iran, Islamic Republic Of.
"is" => "whois.isnic.is", // Iceland.
"it" => "whois.nic.it", // Italy.
"je" => "whois.je", // Jersey.
// jm - Jamaica - no whois server assigned.
// jo - Jordan - no whois server assigned.
"jobs" => "jobswhois.verisign-grs.com",
"jp" => "whois.jprs.jp", // Japan.
"ke" => "whois.kenic.or.ke", // Kenya.
"kg" => "www.domain.kg", // Kyrgyzstan.
// kh - Cambodia - no whois server assigned.
"ki" => "whois.nic.ki", // Kiribati.
// km - Comoros - no whois server assigned.
// kn - Saint Kitts And Nevis - no whois server assigned.
// kp - Korea, Democratic People's Republic Of - no whois server assigned.
"kr" => "whois.kr", // Korea, Republic Of.
// kw - Kuwait - no whois server assigned.
// ky - Cayman Islands - no whois server assigned.
"kz" => "whois.nic.kz", // Kazakhstan.
"la" => "whois.nic.la", // Lao People's Democratic Republic.
// lb - Lebanon - no whois server assigned.
// lc - Saint Lucia - no whois server assigned.
"li" => "whois.nic.li", // Liechtenstein.
// lk - Sri Lanka - no whois server assigned.
"lt" => "whois.domreg.lt", // Lithuania.
"lu" => "whois.dns.lu", // Luxembourg.
"lv" => "whois.nic.lv", // Latvia.
"ly" => "whois.nic.ly", // Libya.
"ma" => "whois.iam.net.ma", // Morocco.
// mc - Monaco - no whois server assigned.
"md" => "whois.nic.md", // Moldova.
"me" => "whois.nic.me", // Montenegro.
"mg" => "whois.nic.mg", // Madagascar.
// mh - Marshall Islands - no whois server assigned.
"mil" => "whois.nic.mil",
// mk - Macedonia, The Former Yugoslav Republic Of - no whois server assigned.
"ml" => "whois.dot.ml", // Mali.
// mm - Myanmar - no whois server assigned.
"mn" => "whois.nic.mn", // Mongolia.
"mo" => "whois.monic.mo", // Macao.
"mobi" => "whois.dotmobiregistry.net",
"mp" => "whois.nic.mp", // Northern Mariana Islands.
// mq - Martinique (France) - no whois server assigned.
// mr - Mauritania - no whois server assigned.
"ms" => "whois.nic.ms", // Montserrat.
// mt - Malta - no whois server assigned.
"mu" => "whois.nic.mu", // Mauritius.
"museum" => "whois.museum",
// mv - Maldives - no whois server assigned.
// mw - Malawi - no whois server assigned.
"mx" => "whois.mx", // Mexico.
"my" => "whois.domainregistry.my", // Malaysia.
// mz - Mozambique - no whois server assigned.
"na" => "whois.na-nic.com.na", // Namibia.
"name" => "whois.nic.name",
"nc" => "whois.nc", // New Caledonia.
// ne - Niger - no whois server assigned.
"net" => "whois.verisign-grs.net",
"nf" => "whois.nic.nf", // Norfolk Island.
"ng" => "whois.nic.net.ng", // Nigeria.
// ni - Nicaragua - no whois server assigned.
"nl" => "whois.domain-registry.nl", // Netherlands.
"no" => "whois.norid.no", // Norway.
// np - Nepal - no whois server assigned.
// nr - Nauru - no whois server assigned.
"nu" => "whois.nic.nu", // Niue.
"nz" => "whois.srs.net.nz", // New Zealand.
"om" => "whois.registry.om", // Oman.
"org" => "whois.pir.org",
// pa - Panama - no whois server assigned.
"pe" => "kero.yachay.pe", // Peru.
"pf" => "whois.registry.pf", // French Polynesia.
// pg - Papua New Guinea - no whois server assigned.
// ph - Philippines - no whois server assigned.
// pk - Pakistan - no whois server assigned.
"pl" => "whois.dns.pl", // Poland.
"pm" => "whois.nic.pm", // Saint Pierre and Miquelon (France).
// pn - Pitcairn (New Zealand) - no whois server assigned.
"post" => "whois.dotpostregistry.net",
"pr" => "whois.nic.pr", // Puerto Rico.
"pro" => "whois.dotproregistry.net",
// ps - Palestine, State of - no whois server assigned.
"pt" => "whois.dns.pt", // Portugal.
"pw" => "whois.nic.pw", // Palau.
// py - Paraguay - no whois server assigned.
"qa" => "whois.registry.qa", // Qatar.
"re" => "whois.nic.re", // Reunion (France).
"ro" => "whois.rotld.ro", // Romania.
"rs" => "whois.rnids.rs", // Serbia.
"ru" => "whois.tcinet.ru", // Russian Federation.
// rw - Rwanda - no whois server assigned.
"sa" => "whois.nic.net.sa", // Saudi Arabia.
"sb" => "whois.nic.net.sb", // Solomon Islands.
"sc" => "whois2.afilias-grs.net", // Seychelles.
// sd - Sudan - no whois server assigned.
"se" => "whois.iis.se", // Sweden.
"sg" => "whois.sgnic.sg", // Singapore.
"sh" => "whois.nic.sh", // Saint Helena.
"si" => "whois.arnes.si", // Slovenia.
"sk" => "whois.sk-nic.sk", // Slovakia.
// sl - Sierra Leone - no whois server assigned.
"sm" => "whois.nic.sm", // San Marino.
"sn" => "whois.nic.sn", // Senegal.
"so" => "whois.nic.so", // Somalia.
// sr - Suriname - no whois server assigned.
"st" => "whois.nic.st", // Sao Tome And Principe.
"su" => "whois.tcinet.ru", // Russian Federation.
// sv - El Salvador - no whois server assigned.
"sx" => "whois.sx", // Sint Maarten (dutch Part).
"sy" => "whois.tld.sy", // Syrian Arab Republic.
// sz - Swaziland - no whois server assigned.
"tc" => "whois.meridiantld.net", // Turks And Caicos Islands.
// td - Chad - no whois server assigned.
"tel" => "whois.nic.tel",
"tf" => "whois.nic.tf", // French Southern Territories.
// tg - Togo - no whois server assigned.
"th" => "whois.thnic.co.th", // Thailand.
"tj" => "whois.nic.tj", // Tajikistan.
"tk" => "whois.dot.tk", // Tokelau.
"tl" => "whois.nic.tl", // Timor-leste.
"tm" => "whois.nic.tm", // Turkmenistan.
"tn" => "whois.ati.tn", // Tunisia.
"to" => "whois.tonic.to", // Tonga.
"tp" => "whois.nic.tl", // Timor-leste.
"tr" => "whois.nic.tr", // Turkey.
"travel" => "whois.nic.travel",
// tt - Trinidad And Tobago - no whois server assigned.
"tv" => "tvwhois.verisign-grs.com", // Tuvalu.
"tw" => "whois.twnic.net.tw", // Taiwan.
"tz" => "whois.tznic.or.tz", // Tanzania, United Republic Of.
"ua" => "whois.ua", // Ukraine.
"ug" => "whois.co.ug", // Uganda.
"uk" => "whois.nic.uk", // United Kingdom.
"us" => "whois.nic.us", // United States.
"uy" => "whois.nic.org.uy", // Uruguay.
"uz" => "whois.cctld.uz", // Uzbekistan.
// va - Holy See (vatican City State) - no whois server assigned.
"vc" => "whois2.afilias-grs.net", // Saint Vincent And The Grenadines.
"ve" => "whois.nic.ve", // Venezuela.
"vg" => "whois.adamsnames.tc", // Virgin Islands, British.
// vi - Virgin Islands, US - no whois server assigned.
// vn - Viet Nam - no whois server assigned.
// vu - Vanuatu - no whois server assigned.
"wf" => "whois.nic.wf", // Wallis and Futuna.
"ws" => "whois.website.ws", // Samoa.
"xxx" => "whois.nic.xxx",
// ye - Yemen - no whois server assigned.
"yt" => "whois.nic.yt", // Mayotte.
"yu" => "whois.ripe.net"