Cloudron Backups to BackBlaze And RC Backup

[Usman]

Hello,

I am posting it here since it's related to "General Backup" process as well as specific to App backup.
I am using BackBlaze for Backups.
General Question:
In my BackBlaze bucket, I see "snapshot" directory as well as date wise directory (for daily backups). I see my apps being backed up at both locations, under Snapshot with container name and under date directory with App name.
(so looks like 2 backups files for each run which looks to be same)
Why is that ? Looks like un-necessary upload and space usage ? (Or m I missing something)

App Question:
I am using RocketChat app. I downloaded the backup from Backblaze, extracted it and I see one "mongodbdump" file without any extension. And then a data directory with mongodump.tar (which looks to be mongodump of the app instance)

What exactly is "mongodbdump" which is taking about 300MB.

Just trying to understand Cloudron backup in a better way and making sure the backups are properly done in case something bad happens.

Thanks

[joseph]

@Usman said in Cloudron Backups to BackBlaze And RC Backup:

Why is that ? Looks like un-necessary upload and space usage ? (Or m I missing something)

The idea is that files are incrementally uploaded into snapshot. Then, a server side copy is done into the timestamped directory i.e no upload. s3 has API to to do server side copies. The timestamped directories are complete backups and work standalone i.e they don't rely on the snapshot directory. You should not delete or change files in the snapshot directory. Otherwise, Cloudron has no idea if you changed it and will assume previously uploaded files are still there.

I am using RocketChat app. I downloaded the backup from Backblaze, extracted it and I see one "mongodbdump" file without any extension.

Rocket.chat uses mongodb as the database and mongodbdump is the dump of the database. I guess this is tgz format. But it's to be treated as a single blob so the file extension is not important.

And then a data directory with mongodump.tar (which looks to be mongodump of the app instance)

This is created by the app itself, maybe. This is not created by Cloudron. Maybe the app is also creating backups of it's own. You can disable this.

[Usman]

@joseph said in Cloudron Backups to BackBlaze And RC Backup:

The idea is that files are incrementally uploaded into snapshot. Then, a server side copy is done into the timestamped directory i.e no upload. s3 has API to to do server side copies. The timestamped directories are complete backups and work standalone i.e they don't rely on the snapshot directory. You should not delete or change files in the snapshot directory. Otherwise, Cloudron has no idea if you changed it and will assume previously uploaded files are still there.

I am not deleting anything. Right now I am testing Cloudron (and hoping to use it more for my setups)
Since I am using "BackBlaze" for backups and that options is provided by Cloudron, there is no option/setting for me to make any changes in this process, everything is done by Cloudron here.
The "Ideal" situation would be to just backup "complete app backups" to remote storage and keep the "snapshot" backups on the server (if that is incremental backup to create a full app backup)

Right now, a 2 GB app backup is causing 4GB storage. (one 2 gb file in snapshot and one 2gb file in Date directory.

Rocket.chat uses mongodb as the database and mongodbdump is the dump of the database. I guess this is tgz format. But it's to be treated as a single blob so the file extension is not important.

I understand that RC is using MongoDB. And I double checked, that "mongodbdump" is indeed a tgz file which contains the same backup data that is under the "data" directory as "mongodump.tar" file.
Again, one mongoDB database is backed up twice. (having it one copy should be enough)

This is created by the app itself, maybe. This is not created by Cloudron. Maybe the app is also creating backups of it's own. You can disable this.

There is no "backup" options in RocketChat itself. The only option I have configured is Backups in cloudron. So this is done by Cloudron Backup procedure.

My main concern here is, unnecessary data transfer and unnecessary space usage in BackBlaze (or any remote storage). Right now, the dbs are small (1 GB in one app, 100mb in 2nd app) so it probably not making it an issue. But if the db will grow (which it will), the backup process might start giving trouble,

So in short, one RC app is backed up twice (under snapshot and then under date directory) and the mongodb database is backed up twice in there (one in tar and one is tgz). Seems inconvenient

Like i said, planning to use Cloudron more so such things matter.

[joseph]

@Usman said in Cloudron Backups to BackBlaze And RC Backup:

Right now, a 2 GB app backup is causing 4GB storage. (one 2 gb file in snapshot and one 2gb file in Date directory.

Yes, this is the case. But also, the system is optimized for having many backups. If you have 30 full backups, this is just 62GB. The 2GB is negligible in this context.

There is no "backup" options in RocketChat itself. The only option I have configured is Backups in cloudron. So this is done by Cloudron Backup procedure.

I don't see this in our Rocket.chat backup atleast. There is no second mongodb dump.

Top level:

Inside data: