Finally managed to get backups stable with backblaze for a ~1TB cloudron with these settings (using encryption and rsync):

Some notes:

• During initial backups, memory was by far the most common failure. The amount needed was higher than I first expected for a given upload size and concurrency. I found (upload part size) x (upload concurrency) x 4 was roughly the amount needed. i.e. (64mb)*(20)*4 = ~5120mb.
• Scheduling backups at a time when server load is low and allotting as much memory as comfortable helped stability a lot. I ended up keeping my memory limit much higher than strictly necessary. This could probably be reduced, but stability is more important to me at the moment.
• In the graph above memory tended to spike once per backup- @girish maybe this is an issue that could be optimized?
• After initial upload, the memory required for subsequent rsyncs dropped off significantly as expected
• Uploading ~1TB of data with the above took approx. 24h

Curious to see what others find from optimizing these further, but glad to have a working baseline in the mean time.

I'm also curious to see what people's monthly bills tally up to. Waiting for the first month's bill to come in but B2 seems more cost effective than Wasabi so far!

For future reference i managed to clear it up by using a slightly modified version of Girish's search command. It seems like the length limit applies to path length not just the filename length

find . -type f -print|awk '{print length($0), $0}' | sort -n

Went into the web interface and deleted / renamed files longer than 156 bytes and finally cleared trashbin/versioned files

(wipes all deleted / old versions of files - careful):

sudo -u www-data php -f /app/code/occ versions:cleanup
sudo -u www-data php -f /app/code/occ trashbin:cleanup

@robi @girish agreed it would be awesome if there was an automated way to handle this in the future

Use case-
Nextcloud updates very frequently, meaning that even if you only intend to keep 1-2 app backups at a time, you can end up with 5 or 6 due to cloudron's 3 week app update policy: https://docs.cloudron.io/backups/#retention-policy

If you have a 1tb nextcloud, this triples the storage costs / box space and adds up really quick. I think this is especially unnecessary when many people will also have a physical drive synced via the nextcloud client as a third backup.

You can delete these backups manually from your storage provider if you remember to, but it would be awesome to have an option to customize this, so people can enable automatic updates+backups of large apps in cloudron without driving too much cost.

Wondering if there are any updates on whether mastodon will be in cloudron?

Would really like to see it

@girish Ubuntu 16.04.4 LTS, ah I just spotted this article: https://docs.cloudron.io/guides/upgrade-ubuntu-20/ will try upgrading and see if the issue persists

Funkwhale looks awesome! Hope one day we can see it or something like it in cloudron and ditch Spotify

Use case-
Nextcloud updates very frequently, meaning that even if you only intend to keep 1-2 app backups at a time, you can end up with 5 or 6 due to cloudron's 3 week app update policy: https://docs.cloudron.io/backups/#retention-policy

If you have a 1tb nextcloud, this triples the storage costs / box space and adds up really quick. I think this is especially unnecessary when many people will also have a physical drive synced via the nextcloud client as a third backup.

You can delete these backups manually from your storage provider if you remember to, but it would be awesome to have an option to customize this, so people can enable automatic updates+backups of large apps in cloudron without driving too much cost.

Finally managed to get backups stable with backblaze for a ~1TB cloudron with these settings (using encryption and rsync):

Some notes:

• During initial backups, memory was by far the most common failure. The amount needed was higher than I first expected for a given upload size and concurrency. I found (upload part size) x (upload concurrency) x 4 was roughly the amount needed. i.e. (64mb)*(20)*4 = ~5120mb.
• Scheduling backups at a time when server load is low and allotting as much memory as comfortable helped stability a lot. I ended up keeping my memory limit much higher than strictly necessary. This could probably be reduced, but stability is more important to me at the moment.
• In the graph above memory tended to spike once per backup- @girish maybe this is an issue that could be optimized?
• After initial upload, the memory required for subsequent rsyncs dropped off significantly as expected
• Uploading ~1TB of data with the above took approx. 24h

Curious to see what others find from optimizing these further, but glad to have a working baseline in the mean time.

I'm also curious to see what people's monthly bills tally up to. Waiting for the first month's bill to come in but B2 seems more cost effective than Wasabi so far!

For future reference i managed to clear it up by using a slightly modified version of Girish's search command. It seems like the length limit applies to path length not just the filename length

find . -type f -print|awk '{print length($0), $0}' | sort -n

Went into the web interface and deleted / renamed files longer than 156 bytes and finally cleared trashbin/versioned files

(wipes all deleted / old versions of files - careful):

sudo -u www-data php -f /app/code/occ versions:cleanup
sudo -u www-data php -f /app/code/occ trashbin:cleanup

@robi @girish agreed it would be awesome if there was an automated way to handle this in the future

Hi All, I am unable to complete backup of a nextcloud instance (~500gb) with backblaze. I know cloudron backup is not usually recommended for larger apps but I thought I'd give it a shot again now that more backup parameters are tweakable. The error I am seeing seems related to encrypted file name length, here it is below:

Error uploading snapshot/app_<nextcloud app id here>/KQtuToxscu6Wp+TQwes2r+Zt37fqVz9KBKw9r3zlGFo/rLwS54KpQJ5BUWx9Wf57gBjNtYl00DkPTpRIO7DHr6E/TQkmjgv8A-8ri6mQv26IOC3SBV98HjxGQYu7Wj942Bc/uROadiwRY44JGJ7Dx8MZs9Tl43w1dViTXgkUZQELVs4/gfgicTfaoUCSMjUN1MOOBnZV3VIOt-jmyeRQIYHGPgs/yROzi4S0rB1UDLZciqxuJMHauKsCowwWvn14ymMFMCo/zK0oecogUE+CWK3JQQCDlW2mmn2Ny-BzrmGk3QokTgk/jOMbp4bhfQklbq5PlI9VYAxfjWMDqO-kd04br5t9gp4/Bdj3Hxfg03RHtuW1vhWh30hEtbB9hS75XLlvUkpdn5M-orp0303DwHg86Kj0jH+Z/hVvdpoNTbKd5ttdU9ZK5JkxtksDlRByNTlLrKZ0Kd5w/7Z2EkbU6DDooF1el4rNzETh4Hi9hW4gqC1yNKEy2byo/s788Z0aGNJt5k5t4LUbO8bmitJ+Y2lTTeYbNV-aDguk/TyEoW0Ge6PJHTg6BeGnNuN4ATowyTxWeejACY22-tgeQA7KJlA7-LVNN-ANz+xc8/DYluRyzYVRZRwR+OtWyLsND675uSdAq7WOzPwbl7tH4/jLUYu3qxS96Fj5-MwYq0YowSytZATPSJkcoBi-JbVgY/Hnvhas87QQpcrWkXBVoITd-eMTXxw6Gh4xCov9sqVsc/0y6Kh1Lt99UO6RdBTII7kgHHoPVLeSu9r-6HWmzi5MSBa9ioD6J9pkSDgI6bPq6a/7Z2EkbU6DDooF1el4rNzETh4Hi9hW4gqC1yNKEy2byo/Hnvhas87QQpcrWkXBVoITd-eMTXxw6Gh4xCov9sqVsc/1GQ+V1wtz+KCmZU1hEcQb0Wf0ZxKEAZhoejV31Cr6Lg/+tqCfG-K-nTRG3LiFmWOSqRg1HmzpiqY2ytGbHNF8Xxh4xhfznEEAfrMKhJIKDFl. Message: File name in UTF8 must be no more than 1024 bytes HTTP Code: InvalidRequest

@girish Ubuntu 16.04.4 LTS, ah I just spotted this article: https://docs.cloudron.io/guides/upgrade-ubuntu-20/ will try upgrading and see if the issue persists

Thanks for the quick response @mehdi !

Noted, switching to using black hole in that case

Hey All,

I get an error when attempting to link couchpotato and transmission in the downloader settings menu with these inputs:

Host: https://transmission.my-example-cloudron.com
RPC url: transmission
user: [cloudron username]
password [cloudron password]

These seem to be work fine in SickChill but CP returns this error:

ERROR[.downloaders.transmission] Config properties are not filled in correctly, port is missing.

Adding a port to the end of the transmission url doesn't seem to solve it. Anyone else having this issue?

Hi All,

I've seen an issue twice in the last few months where /run/udev/data will fill up the /run partition overtime, eventually crashing the dashboard. Restarting the server seems to clear this folder and the server comes back online, but obviously it would be best not to have the server go down every few weeks.

Below is output of some disk usage commands. Please let me know if there's any other info I can provide!

Best,
Ethan

root@localhost:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            2.0G     0  2.0G   0% /dev
tmpfs           393M  393M     0 100% /run
/dev/sda2        20G   11G  8.0G  57% /
tmpfs           2.0G     0  2.0G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/sda1       487M   36M  422M   8% /boot
/dev/sda3       1.8T  795G  927G  47% /home
overlay         1.8T  795G  927G  47% /home/docker/overlay2/9a5e282fdcd9ec4681c0d1c0fb15f7a7ff7eda0ece5e4b9e0a68762692eae35e/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/0bdf5e13f8a83ca3ebebc87212922638eb4d9ded7d7accaa3e20ddeda5dced92/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/45613b260840f06be967ef5fdf248007199891fa2f7f9d4847e6ba6398804db4/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/476f326bbc83b1900d923072d7ba3e263b27e9467d4cf4a6e1920bf33ce03e58/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/a237010796d9a5da3b27631eca065d2e103da31dcaba2e1877d7c4fade19d218/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/7169515c216da9e1913b8c39ea96296790571f7c022c988496b76ae49ba73436/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/193a0fd036a1b7deaba4fbcf59b7cb64e37178df637b7cd65c041d55151bdf3e/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/a29651b2ee737f945aadbf52e025e5a1bb0fad78b4a2eed0f1e50f51eec1b268/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/8d142cc736c9e0512dc809a99e45158f9a9ba2552fb741935a2de26ea2a80165/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/942491a3968de0663f8116d94afe22f0e0e8755676e7c4f2d82e8649fb587857/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/0ca4d556581ee79190f7948caa097be42fc9cf7c8e928f759a7af8022c682fb9/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/242df83b4d74aa35ca500b2eda722b96210fcbffa1cce39a6332d4d2d23fc8ac/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/a196773d462838d7b38a8f44d9b44557022ed24784bcc2f700331fa471fc5272/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/dd4da96ca820fbe61e77a27cb02346742d4c1afc710097b74d8f52d31e8b0517/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/bc6b0bea1be9dae8eb85b73b38f54c7504216b1941cf9f1e9a528c442696c6d5/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/37fbe6e6b9b452c17f1f8de2a5267018f85c6a3983f46998052f80278dbe50b1/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/a2b82c8008d0f0e210866b2c77864d26b2b1fcb1d8ea195ecf81f50f7bdbb4ed/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/9ad83b3b2ab5055882503162f952c9c442a10fbc17c80d09f62084c42f23bfe6/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/cfd2bd3867197e60d4eebf99e8df3423ab5dde864b3191fdf7b9393afe22b3ef/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/d7e51bc7bae2d8cdc16799e7b9579e8a1f85c443d35acd6b028c07e2f8576b74/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/02dcfe4cdb631a7121cb8a37efccefe70ef5adbef9aa1f2ccaff2bd09fa48002/merged
overlay         1.8T  795G  927G  47% /home/docker/overlay2/aba28e860dab3232388fee0d24a7e36dbbbe9bffa8738742fe2461b74f42569e/merged
tmpfs           393M     0  393M   0% /run/user/0

root@localhost:/run# du -a | sort -n -r | head -n 5
 
402000	.
400320	./udev/data
400320	./udev
708	./docker/runtime-runc/moby
708	./docker/runtime-runc

Hi @mehdi, Wondering if this feature is included in the app / if these folders can be set to Cloudron volumes

Thanks Girish that is it- I'll have to manually move over my calendar / contacts it looks like but the emails are there