Cloudron can't uninstall apps after access to domain got denied

[BrutalBirdie]

TL;DR

Can't delete apps for domain where I have no longer access to.
Stuck with apps which can not be uninstalled.

---

This sound logical and why should this be an issue?

Okay let me explain what happend here and why this should still work.
I am currently managing a big cloudron server for multible communitys.
I do not have direkt access to their domains (mostly via namecheap as manager)

Now a community decided to switch to a self hosted cloudron (pay 30$ for their own premium) which is good!
They basically tried cloudron premium for a month on the shared server to decide if they want to use this as well.
I helped them migrate to their own server and restore the backups.

Then they deleted me as a manager from namecheap, so they are the owner/manager alone.

But here is the issue. I did not uninstall the app, but only stopped them and turn the backups off for their migration.

Now I am sitting on 8 Apps which I can't uninstall since cloudron tries this

Nov 06 16:25:43 box:apptask track.X Unregistering subdomain: track.X
Nov 06 16:25:43 box:domains removeDNSRecord: track onX type A values [ 'X' ]
Nov 06 16:25:43 box:dns/namecheap del: track for zone X of type A with values ["X"]
Nov 06 16:25:44 box:apptask registerSubdomains: Remove error. Will retry. You don't seem to have edit permission for this domain.

---

After writing this issue I am asking myself, why is there no:
Try Cloudron premium for one month.

Well I know this can be abused, I create my 100 Cloudron Users install all apps and then simply dont pay. Something like that.

Could be negated by disabling all users and apps (except first 5 users and first 2 apps) after premium ended, but this behavior does not fit to the cloudron policy and I would not want such a death grip from this software.

But since I am advertising cloudron to many people and they want to try premium out first, I let them use a shared cloudron.
I think if they would have an option for trying cloudron premium for a limited time for free they would do that.

[BrutalBirdie]

@girish switching to manual DNS worked. Thanks

[girish]

@brutalbirdie A workaround is just to set the DNS Provider of the domain to be "manual". You can then delete the domain. When you get access back, put the DNS provider back. Would that work?

[BrutalBirdie]

@girish switching to manual DNS worked. Thanks

[nebulon]

While that workaround solves it fine, it is probably not obvious at all. I think there is room for improvement here. For a start, once uninstall failed, the "repair" action comes into picture, which is oddly named for that use-case. However it would allow to re-run the uninstall (ie the last failed task).

Now of course this would not solve any issue here, so maybe we should add some "force" option when uninstall fails. However for sake of consistency on the Cloudron itself, this option should not randomly skip resource cleanup errors, but I think the DNS records are not "owned" by the Cloudron and thus are ok to fail the cleanup?

To give an example, where the "force" option should not apply: Removing the app data folder. For whatever reason if that fails, the uninstall should not just run along, leaving unreferenced folder/files on the filesystem.

Just an idea though

[BrutalBirdie]

@nebulon had a similar idea about a notification that something failed but the user can force the uninstall.

[robi]

@nebulon there should be branching logic after Cloudron 'senses' that it doesn't own that DNS anymore.

Prompt for delete of an unmanaged resource in that state?

[andreasdueren]

I have a similar issue. But setting it to manual won't work, uninstall results in:

External Error - message: Invalid access token statusCode: 403 code:9109

[nebulon]

With manual backend, no API should be called, so either this is some other issue or double check the backend setting for the domain in question.

If you still see this issue, maybe you can post a bit more log history to see where in the code it fails.

[andreasdueren]

Looks like a previous attempt already removed the data folder?

Sep 26 10:49:38 box:taskworker Starting task 13697. Logs are at /home/yellowtent/platformdata/logs/5d825fe4-a4c1-47c5-bf67-2583d1638d00/apptask.log
Sep 26 10:49:38 box:apptask run: startTask installationState: pending_uninstall runState: stopped
Sep 26 10:49:38 box:tasks update 13697: {"percent":20,"message":"Deleting container"}
Sep 26 10:49:38 box:shell reload /usr/bin/sudo -S /home/yellowtent/box/src/scripts/restartservice.sh nginx
Sep 26 10:49:39 box:apptask deleteContainer: deleting app containers (app, scheduler)
Sep 26 10:49:39 box:shell removeLogrotateConfig /usr/bin/sudo -S /home/yellowtent/box/src/scripts/configurelogrotate.sh remove 5d825fe4-a4c1-47c5-bf67-2583d1638d00
Sep 26 10:49:39 box:tasks update 13697: {"percent":30,"message":"Teardown addons"}
Sep 26 10:49:39 box:services teardownAddons: Tearing down ["mysql","localstorage","sendmail","redis","ldap"]
Sep 26 10:49:39 box:services teardownAddons: Tearing down addon mysql with options {}
Sep 26 10:49:39 box:services teardownAddons: Tearing down addon localstorage with options {"ftp":{"uid":33,"uname":"www-data"}}
Sep 26 10:49:39 box:services teardownLocalStorage
Sep 26 10:49:39 box:shell clearVolume /usr/bin/sudo -S /home/yellowtent/box/src/scripts/clearvolume.sh rmdir /home/yellowtent/appsdata/5d825fe4-a4c1-47c5-bf67-2583d1638d00/data
Sep 26 10:49:39 box:services Tearing down sendmail
Sep 26 10:49:39 box:services teardownAddons: Tearing down addon sendmail with options {}
Sep 26 10:49:39 box:services teardownAddons: Tearing down addon redis with options {"optional":true}
Sep 26 10:49:39 box:shell removeVolume /usr/bin/sudo -S /home/yellowtent/box/src/scripts/rmaddondir.sh redis 5d825fe4-a4c1-47c5-bf67-2583d1638d00
Sep 26 10:49:39 box:services teardownAddons: Tearing down addon ldap with options {}
Sep 26 10:49:39 box:services Tearing down LDAP
Sep 26 10:49:39 box:tasks update 13697: {"percent":40,"message":"Cleanup file manager"}
Sep 26 10:49:39 box:tasks update 13697: {"percent":50,"message":"Deleting app data directory"}
Sep 26 10:49:39 box:tasks update 13697: {"percent":60,"message":"Deleting image"}
Sep 26 10:49:39 box:tasks update 13697: {"percent":70,"message":"Unregistering domains"}
Sep 26 10:49:39 box:network/generic getIPv4: querying ipv4.api.cloudron.io to get server IPv4
Sep 26 10:49:39 box:tasks update 13697: {"message":"Unregistering location: app.domain.tld"}
Sep 26 10:49:39 box:dns removeDNSRecords: app on domain1.tld type A values [ 'xx.xx.xx.xx' ]
Sep 26 10:49:39 box:dns removeDNSRecords: app on domain2.tld type A values [ 'xx.xx.xx.xx' ]
Sep 26 10:49:39 box:tasks update 13697: {"message":"Unregistering location: app2.domain2.tld"}
Sep 26 10:49:40 box:dns unregisterLocation: Error unregistering location A. retryable: false. message: Invalid access token statusCode: 403 code:9109
Sep 26 10:49:40 box:apptask run: app error for state pending_uninstall: BoxError: message: Invalid access token statusCode: 403 code:9109 at unregisterLocation (/home/yellowtent/box/src/dns.js:303:11) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async /home/yellowtent/box/src/dns.js:317:23 at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20) at async Object.unregisterLocations (/home/yellowtent/box/src/dns.js:316:9) at async uninstall (/home/yellowtent/box/src/apptask.js:772:5) { reason: 'External Error', details: {} }
Sep 26 10:49:40 box:tasks setCompleted - 13697: {"result":null,"error":{"stack":"BoxError: message: Invalid access token statusCode: 403 code:9109\n at unregisterLocation (/home/yellowtent/box/src/dns.js:303:11)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async /home/yellowtent/box/src/dns.js:317:23\n at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20)\n at async Object.unregisterLocations (/home/yellowtent/box/src/dns.js:316:9)\n at async uninstall (/home/yellowtent/box/src/apptask.js:772:5)","name":"BoxError","reason":"External Error","details":{},"message":"message: Invalid access token statusCode: 403 code:9109"}}
Sep 26 10:49:40 box:tasks update 13697: {"percent":100,"result":null,"error":{"stack":"BoxError: message: Invalid access token statusCode: 403 code:9109\n at unregisterLocation (/home/yellowtent/box/src/dns.js:303:11)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async /home/yellowtent/box/src/dns.js:317:23\n at async promiseRetry (/home/yellowtent/box/src/promise-retry.js:17:20)\n at async Object.unregisterLocations (/home/yellowtent/box/src/dns.js:316:9)\n at async uninstall (/home/yellowtent/box/src/apptask.js:772:5)","name":"BoxError","reason":"External Error","details":{},"message":"message: Invalid access token statusCode: 403 code:9109"}}
Sep 26 10:49:40 box:taskworker Task took 1.569 seconds
[no timestamp]  message: Invalid access token statusCode: 403 code:9109
[no timestamp]  message: Invalid access token statusCode: 403 code:9109
[no timestamp]  message: Invalid access token statusCode: 403 code:9109
[no timestamp]  failed to remove '/home/yellowtent/appsdata/5d825fe4-a4c1-47c5-bf67-2583d1638d00/data': No such file or directory
[no timestamp]  failed to remove '/home/yellowtent/appsdata/5d825fe4-a4c1-47c5-bf67-2583d1638d00/data': No such file or directory

[joseph]

@andreasdueren said in Cloudron can't uninstall apps after access to domain got denied:

Sep 26 10:49:40 box:dns unregisterLocation: Error unregistering location A. retryable: false. message: Invalid access token statusCode: 403 code:9109

The directory message is just a warning can be ignored. The above message is the real error. Are you sure you are not using any other domain in the app which in turn uses some DNS provider? Are all the DNS providers manual ?