@girish said in Cannot update due to one tiny app filling up 20GB drive:

@jdaviescoates you have to actually delete the swap file /apps.swap to get the space back.

Thanks! That did the job, I'm now updating. Finally!

@jdaviescoates you could set up a cron that gets usage info, and prints out a webpage you can monitor with Kuma for a keyword. When a threshold is met, you add/remove the keyword and fire off an alert via TG.

Similar for monitoring something with change detection app or n8n workflow.

@girish said in Disk usage issue:

https://git.cloudron.io/cloudron/box/-/commit/67cde5a62cf0394c8bf2d78ee3408e5995a220e7 is the fix if you want to patch it yourself.

Thanks, I wouldn't know where to start with that, but I guess useful for others who do 🙂

Have you seen https://docs.cloudron.io/troubleshooting/#recovery-after-disk-full with steps to bring a system back from such a disk full event?

Regarding Nextcloud, you should be able to restore it to the previous version then.

@subven 🤦

wordwrap!!! the problem was staring me in the face! But I could not see it because i did not scroll to the right...

Thanks for the help!!!!

I took a cert and key file from another server, and renamed the default cert to be whatever the error wanted.
first thedomain.com.cert, then thedomain.com.key. and so-forth until it loaded. It is working now! .

@girish yes, I just want to get notified either by Cloudron via email (because I'm not always logging in and seeing the notifications) or some other means (i.e. regularly checking an API endpoint) if disk space is running low. It's not about individual space usage of apps, just the df of the main disk

@shan 7.3.3 I believe.

@subven said in out of space error leading to missing certs:

@girish there is no way to trigger certificate renewal over the (SSH) console?

I'd like an answer to this question.. as I just ran into the missing cert problem too.

Having deleted all the conf/cert files, and gotten nginx started, the UI is still not accessible after box restart. All apps are inaccessible too.

box restart seems to recreate the /etc/nginx/applications/my.domain.conf BUT doesn't check if the /home/yellowtent/platformdata/nginx/certs/my.domain.host.cert is there.

How are they regenerated from the CLI?

@nebulon Thanks for confirming! 🙂

@girish great thanks, It works 😉

@ei8fdb right the graphs don't update immediately, but every 6 hours or so. Otherwise, the disk spins a lot 🙂

I will mark this thread as solved, but this problem will go away soonish. I think only ~20 apps are left to use the new base image. The 7.1 release also updates all the internal containers to use the latest image. Should all be done end of this month.

@scooke
@murgero

I am working on get my FreeNAS NFS mount working for the backups, but that seems to be a bit difficult to get the permissions working.

If you have experience with NFS, you might have an answer on my problem here...

https://forum.cloudron.io/topic/5927/cloudron-backups-on-truenas-nfs-mount/3?_=1636891033300

However thanks!

@girish

UPDATE: Finally found the culprit the was a huge backup file in /var/backups/snapshot#

Did the internal DNS restart steps, restarted docker and my sites are working again. Still having trouble accessing the cloudron admin page, complaint about certificates etc. will wait and see if this resolves itself.
Dimitri

UPDATE: /home/yellowtent/box/setup/start.sh did the trick cloudron dashboard also available again

@jdaviescoates said in Volume disk space strangeness:

/me goes to reduce the size of the volume back to a perfectly sufficient 10 GB...

Oh, bollocks. You can't decrease the size of Hetzner Cloud Volumes, only increase them. So I'd have to create a whole new one and then and then and then...can't be bothered with all that so I guess I'll just stomach the slightly increased cost of 15 GB over 10 GB.

Debugged this with @ZeZaung a bit. Somehow, it was possible to re-run cloudron-setup on the existing install and this reset the database even though the app containers and data were still there. I have made a fix which should hopefully mitigate the problem.

@girish Seems to be all clear. No got any notification in system area. So alert was it misunderstood?

Many thank's!

@jdaviescoates said in Nextcloud says 5.3GB used, Cloudron says Nextcloud is 19.16GB, why?:

That's "just" a difference of 2.66GB (which I guess could mostly be just Nextcloud files themselves)

So, assuming that is mostly just Nextcloud files themselves (I may do a clean install at some point just to see how big that is), I think this can be marked as solved.

Ah we did hit that kernel/initramfs issue with /boot getting filled up by Ubuntu already before. If it is 100% full even the autoremove fails.

Would be good to actually not just maybe warn about that, but see how the whole situation can be mitigated in the first place. Afterall it is not much use to have that many kernel versions around.

@girish & @marcusquinn thanks for the hints!

Finally the AWS guide did the trick, it was a 2 step approach (in my case):

sudo growpart /dev/sda 3

sudo resize2fs /dev/sda3