Bloated n8n backup

[joseph]

@umnz are you able to check the size of the file in your backup storage directly? Does it match what Cloudron is reporting?

[umnz]

@Joseph yes, the size matches.

There appears to be no rhyme or reason to these backup sizes after updating. Nightly is showing a smaller file size now.

The reason it is noticeable is updating an app is taking 15-20 minutes because of the snapshot+upload to Cloudflare R2. 6GB takes a lot longer than 800mb.

[joseph]

@umnz you should download one of the backups from Cloudflare UI and check what is taking so much space inside the tarball.

[umnz]

When I try to download the backup from the Cloudron interface I get this error:

{
  "status": "Conflict",
  "message": "only tgz backups can be downloaded"
}

Is this because they are encrypted? .tar.gz.enc

I will try the Cloudflare UI. But still curious to understand why that error occurs.

[joseph]

@umnz I think that was a bug, it's fixed in 9.0.14 .

[joseph]

@umnz also if you open webterminal of n8n and then cd /app/data and du -hcs . what size do you get?

[umnz]

@joseph said in Bloated n8n backup:

du -hcs .

I get 1.5G.

Be mindful since posting the above I have restarted my Cloudron instance which had been up (for 5 days) since automatically updating to 9.0.13.

Cloudflare won't let me download files over 1G directly so I'll look to update to 9.0.14 now.

[umnz]

Actually I'm going to hold off on updating to 9.0.14 because I don't want to risk it turning into a different problem. I'll wait for 9.0.14 to come out of pre-release or do it this weekend when activity is low.

[umnz]

@joseph said in Bloated n8n backup:

du -hcs .

Sorry, I misread this. I navigated to cd app/data/ and it's 906MB. My other post was running that from app/code

[umnz]

Just an update - I have changed absolutely nothing but I'm seeing the nightly backup size get smaller.

[joseph]

Oh, that's great. I suspect some file inside n8n is growing in size and then gets compacted by some cron or something. If it grows again, please analyze the backup and check what file is growing.

[p44]

hi all, hi @joseph I think there is some problem going on, because I too had the opposite problem: the size decreased after some backup of LibreChat. Please have a look:

We’re talking about changes during 24-48 hours...

[joseph]

@p44 have you checked inside the backup what is changing in size?

[p44]

@joseph What do you mean exactly? What I should check "inside the backup”?

[james]

Hello @p44

I think, this might be no issue at all rather expected between each LibreChat version change.
I have checked the changelog of LibreChat app version 1.0.1 to 1.0.2 since that is the big size change.
Since Version the app version 1.0.2 upgraded LibreChat to 0.8.1 this also required a new meilisearch version the meilisearch DB file was reinitialized which is a necessary step.
This could explain the size change between the two version and would also make sense with all the other backups slowly increasing in size since the meilisearch db is filled over time.

If you can download the backup with the larger size and the latest that is smaller.
Cross compare them to see if it is truly just the meilisearch DB file, that would confirm it.

[p44]

@james As I showed in the video, size decreased... not increased...

[james]

Hello @p44
Yes, exactly.

I did not want to imply with

the big size change

that the size change was incremental but the difference was substential.
Should have phrased that better.

[umnz]

Also note, I am waiting for 9.0.14 to be stable release before I can update and then download a backup. Currently it's not possible to download a backup in 9.0.13 from Cloudflare R2.