Limit CPU Usage to S3
-
Hey all,
Trying to downgrade my server to 1 CPU core. Although i have noticed when S3 backups up daily, 100% CPU is utilized for 2-3 hours while it backups.
On the other side, 2 CPU cores uses around 50% CPU usage.
Any ideas?
Thanks
-
I guess it's consuming CPU because gzip of the backup does consume some CPU. I guess one way to disable gzip is switching to rsync which will try to upload files without zipping them. But note that S3 charges per HTTP request and depending on how many files you have this can be expensive. An easy fix to that might be to choose some other S3 backup service like say DO Spaces which doesn't charge per request.
-
@dylightful I can recommend rsync to Wasabi, tried all the rest and this works best for something you probably won't have to ever think about again. AWS used to be good but I don't recommend them nowadays.
-
@marcusquinn said in Limit CPU Usage to S3:
@dylightful I can recommend rsync to Wasabi, tried all the rest and this works best for something you probably won't have to ever think about again. AWS used to be good but I don't recommend them nowadays.
Do you have a speed and/or performance difference in mind between storage format tarball and rsync to wasabi?
I want to switch to wasabi from google cloud storage and now asked my self, after reading your comment, if I should go with tarball or rsync.
-
@BrutalBirdie I'm happy with rsync. I tested Tarball for interest but then you're paying in full with CPUs & bandwidth every day with that. Whichever you choose I'm sure you'll be happy with Wasabi (I should get an affiliate link
).Anything else I think is a separate debate for rsync vs tarball vs both vs multi-cloud backups etc.
-
@marcusquinn said in Limit CPU Usage to S3:
@BrutalBirdie I'm happy with rsync. I tested Tarball for interest but then you're paying in full with CPUs & bandwidth every day with that. Whichever you choose I'm sure you'll be happy with Wasabi (I should get an affiliate link
).Anything else I think is a separate debate for rsync vs tarball vs both vs multi-cloud backups etc.
Nice idea, does wasabi even have something like affiliate links?
-
@BrutalBirdie I don't see a standard affiliate referral scheme but MSP seems the closest on their partners page
-
@marcusquinn said in Limit CPU Usage to S3:
@BrutalBirdie I don't see a standard affiliate referral scheme but MSP seems the closest on their partners page
Looks like this is more for companies / businesses. Now some questions:
- what benefit does a company have when being a partner with wasabi?
- Do the customers of said partner have to pay less?
- Would it be beneficial for Cloudron to have (or to be) a partner at wasabi to offer their customers a better price or service? Bundle like idea, Cloudron Premium + 5$ with wasabi already preconfigured?
I don't know. Only if this would be optional.
I can imagine a new premium customer would like the option for a plug'n'play backup solution.
But then GDPR/DSGVO yada yada.Is this even a good idea?
-
@BrutalBirdie it's a good idea as an option. they offer US & EU regions.
-
Now that the Privacy Shield has been repelled, it's very much in question whether or not any US-based provider can be GDPR-compliant, even if they offer EU regions, because of the Cloud Act.
-
@mehdi Very much interested in this topic. I love my American friends, but the EU certainly isn't without capability to respond to the data-mining free-for-all antics of the last decade. I share occasional posts on Twitter on these subjects if anyone's interested.
