Help with Wasabi mounting
-
@privsec We tried this once and it was unbearably slow. Personally I'd look for ways to reduce the Nextcloud storage needs and use a native mounted drive from the host, and just offer an Archive Wasabi S3 bucket for people to move things no longer needed for regular access to that to save space needs on the main NC.
@marcusquinn Cyber Duck/Mountain Duck is very good as a platform to making Wasabi/S3 storage quite accessible to users if they need.
-
@privsec We tried this once and it was unbearably slow. Personally I'd look for ways to reduce the Nextcloud storage needs and use a native mounted drive from the host, and just offer an Archive Wasabi S3 bucket for people to move things no longer needed for regular access to that to save space needs on the main NC.
@marcusquinn
I can not get a mounted drive on my host to actually work. I have tried both Rclone and S3FS, both gave me the same result of no errors, but no syncing.I have decided (atleast for now till I find a better solution) to give each user on nextcloud a wasabi bucket as a external storage drive, and restrict only them to be able to access that bucket.
I am now working on trying to figure out how to remove the built in files for nextcloud to just have the external storage folder listed with the nextcloud folders/files within the wasabi drive.
In regards to Cyber Duck/Mountain Duck, whats the difference between the external storage connection in nextcloud and them ? Is there a performance difference?
-
@marcusquinn
I can not get a mounted drive on my host to actually work. I have tried both Rclone and S3FS, both gave me the same result of no errors, but no syncing.I have decided (atleast for now till I find a better solution) to give each user on nextcloud a wasabi bucket as a external storage drive, and restrict only them to be able to access that bucket.
I am now working on trying to figure out how to remove the built in files for nextcloud to just have the external storage folder listed with the nextcloud folders/files within the wasabi drive.
In regards to Cyber Duck/Mountain Duck, whats the difference between the external storage connection in nextcloud and them ? Is there a performance difference?
@privsec Sorry, I mean just a mounted local host storage, not mounting external S3, which will have such high latency that it makes NC painful to use.
-
@marcusquinn
I can not get a mounted drive on my host to actually work. I have tried both Rclone and S3FS, both gave me the same result of no errors, but no syncing.I have decided (atleast for now till I find a better solution) to give each user on nextcloud a wasabi bucket as a external storage drive, and restrict only them to be able to access that bucket.
I am now working on trying to figure out how to remove the built in files for nextcloud to just have the external storage folder listed with the nextcloud folders/files within the wasabi drive.
In regards to Cyber Duck/Mountain Duck, whats the difference between the external storage connection in nextcloud and them ? Is there a performance difference?
@privsec I realise host local storage isn't cheap, and Wasabi is cheap, which is why we tried doing what you are once. Maybe you'll have a different experience but we just found the juice wasn't worth the squeeze, especially with large numbers of small files.
It might work for something like Video storage where there's fewer files but I just don't fancy the slow user experience costs and would always opt for speed over price since time is so much more expensive if users are spending too much waiting on their file services to show results.
-
@marcusquinn
I can not get a mounted drive on my host to actually work. I have tried both Rclone and S3FS, both gave me the same result of no errors, but no syncing.I have decided (atleast for now till I find a better solution) to give each user on nextcloud a wasabi bucket as a external storage drive, and restrict only them to be able to access that bucket.
I am now working on trying to figure out how to remove the built in files for nextcloud to just have the external storage folder listed with the nextcloud folders/files within the wasabi drive.
In regards to Cyber Duck/Mountain Duck, whats the difference between the external storage connection in nextcloud and them ? Is there a performance difference?
@privsec Just reading this, not sure if you've read? I'm not saying i can't be done, just that we gave up. Maybe this helps? I'm still reading: https://autoize.com/s3-compatible-storage-for-nextcloud/
-
From reading the above article, is sounds like it might need to be that Cloudron has 2 x versions of Nextcloud packaged, or at least an option in installing to make it setup for S3 storage.
Probably needs one of the @appdev team to have a read and confirm or deny.
I agree it would be interesting and our past experience might have been what this article above solves.
Let's get more opinions on this, someone else might also have a need and interest and be willing to try themselves too.
Web Design https://www.evergreen.je
Development https://brandlight.org
Life https://marcusquinn.com -
From reading the above article, is sounds like it might need to be that Cloudron has 2 x versions of Nextcloud packaged, or at least an option in installing to make it setup for S3 storage.
Probably needs one of the @appdev team to have a read and confirm or deny.
I agree it would be interesting and our past experience might have been what this article above solves.
Let's get more opinions on this, someone else might also have a need and interest and be willing to try themselves too.
@marcusquinn Ill have to give that article above a read.
Anything I can do to help make launching costs low is sought after, while keeping user experience high
-
From reading the above article, is sounds like it might need to be that Cloudron has 2 x versions of Nextcloud packaged, or at least an option in installing to make it setup for S3 storage.
Probably needs one of the @appdev team to have a read and confirm or deny.
I agree it would be interesting and our past experience might have been what this article above solves.
Let's get more opinions on this, someone else might also have a need and interest and be willing to try themselves too.
@marcusquinn said in Help with Wasabi mounting:
From reading the above article, is sounds like it might need to be that Cloudron has 2 x versions of Nextcloud packaged, or at least an option in installing to make it setup for S3 storage
Ihmo no second app necessary, the drawback is though that the s3 backend can only be enabled before any data is added to the app.
I am however sceptical if adding network latency for file access is such a smart idea. Especially for a php application.
-
@marcusquinn said in Help with Wasabi mounting:
From reading the above article, is sounds like it might need to be that Cloudron has 2 x versions of Nextcloud packaged, or at least an option in installing to make it setup for S3 storage
Ihmo no second app necessary, the drawback is though that the s3 backend can only be enabled before any data is added to the app.
I am however sceptical if adding network latency for file access is such a smart idea. Especially for a php application.
-
TBH I can see the advantages for having a separate Nextcloud instance with just the Files app enabled as web interface for S3 to allow users to have it available as a slower cold-storage option, with having a faster standard NC setup for their daily hot-storage needs.
So, I wouldn't dismiss this aim, just with speed cautions, and the cautions in the article that the storage would then be unbrowsable directly without the NC database metadata interface, so it also has mass disruption risks if that were lost.
-
@privsec Just reading this, not sure if you've read? I'm not saying i can't be done, just that we gave up. Maybe this helps? I'm still reading: https://autoize.com/s3-compatible-storage-for-nextcloud/
@marcusquinn said in Help with Wasabi mounting:
@privsec Just reading this, not sure if you've read? I'm not saying i can't be done, just that we gave up. Maybe this helps? I'm still reading: https://autoize.com/s3-compatible-storage-for-nextcloud/
This is along the same lines as my advanced object store usage option above.
I didn't know this was possible from within NC, but it appears they engineered it in. Very smart & clever.
They are essentially caching everything recent in /tmp and background syncing to the object store over time.
The user experience is good as it's all mostly local, and the capacity is greatly extended as the Object Store is vast.I've designed this for other use cases at IBM, and wrote their Redbook on it, hence advanced prior knowledge.
For this article to be applied in Cloudron, we'd need a new packaged NC App that is configured for this before the setup/install. Then one can point it at a local Minio instance or external S3 object store.
-
@marcusquinn said in Help with Wasabi mounting:
@privsec Just reading this, not sure if you've read? I'm not saying i can't be done, just that we gave up. Maybe this helps? I'm still reading: https://autoize.com/s3-compatible-storage-for-nextcloud/
This is along the same lines as my advanced object store usage option above.
I didn't know this was possible from within NC, but it appears they engineered it in. Very smart & clever.
They are essentially caching everything recent in /tmp and background syncing to the object store over time.
The user experience is good as it's all mostly local, and the capacity is greatly extended as the Object Store is vast.I've designed this for other use cases at IBM, and wrote their Redbook on it, hence advanced prior knowledge.
For this article to be applied in Cloudron, we'd need a new packaged NC App that is configured for this before the setup/install. Then one can point it at a local Minio instance or external S3 object store.
@robi We have tried the setup of using S3 as objectstorage before in nextcloud and it just doesn't perform well. It also makes a very large number of s3 requests and for things like AWS this costs some real $$$. Other services like DO Spaces will promptly rate limit. Overall, the whole setup is unstable. It looks good on paper but doesn't work reliably for serious use.
for example: https://help.nextcloud.com/t/high-aws-s3-costs-due-to-nextcloud-requests/68687 and https://help.nextcloud.com/t/high-traffic-to-primary-storage-s3-from-nextcloud/83185 .
-
@robi We have tried the setup of using S3 as objectstorage before in nextcloud and it just doesn't perform well. It also makes a very large number of s3 requests and for things like AWS this costs some real $$$. Other services like DO Spaces will promptly rate limit. Overall, the whole setup is unstable. It looks good on paper but doesn't work reliably for serious use.
for example: https://help.nextcloud.com/t/high-aws-s3-costs-due-to-nextcloud-requests/68687 and https://help.nextcloud.com/t/high-traffic-to-primary-storage-s3-from-nextcloud/83185 .
@girish yes, the article explicitly warns about this, pointing out that AWS is not a good place to do this as it's not a part of the bandwidth alliance.
This is not a problem with NC, but of the underlying architecture chosen.
There is a way to make this work well, but one must understand all the pieces, as if you ignore one, it will be the lowest common denominator and bottleneck.
It is an Enterprise Deployment afterall.
-
@robi We have tried the setup of using S3 as objectstorage before in nextcloud and it just doesn't perform well. It also makes a very large number of s3 requests and for things like AWS this costs some real $$$. Other services like DO Spaces will promptly rate limit. Overall, the whole setup is unstable. It looks good on paper but doesn't work reliably for serious use.
for example: https://help.nextcloud.com/t/high-aws-s3-costs-due-to-nextcloud-requests/68687 and https://help.nextcloud.com/t/high-traffic-to-primary-storage-s3-from-nextcloud/83185 .
@girish This article suggests there's an old and a new way to do this, so I wonder if we're all talking about the same thing?
Web Design https://www.evergreen.je
Development https://brandlight.org
Life https://marcusquinn.com -
@girish This article suggests there's an old and a new way to do this, so I wonder if we're all talking about the same thing?
@marcusquinn said in Help with Wasabi mounting:
@girish This article suggests there's an old and a new way to do this, so I wonder if we're all talking about the same thing?
old = adding an additional S3 store from within NC running locally.
new = setting up NC with S3 as primary storage. -
@marcusquinn said in Help with Wasabi mounting:
@girish This article suggests there's an old and a new way to do this, so I wonder if we're all talking about the same thing?
old = adding an additional S3 store from within NC running locally.
new = setting up NC with S3 as primary storage.@robi @marcusquinn Both methods have been there for a some time. I have investigated this a year or so ago, for example, this and this. Back then, it was not really stable. It will create a lot of oid:xx files in the object backend and AFAIK there is no easy way to change an installation from one storage to another (it's setup once at install time). All this means, that as @robi pointed out, this is really targeted at enterprises and people having full time dedicated sysadmins and nextcloud support contract. Don't get me wrong, I am sure it can be made to work but it's not something we want to support (best to pay nextcloud for this)
-
@robi @marcusquinn Both methods have been there for a some time. I have investigated this a year or so ago, for example, this and this. Back then, it was not really stable. It will create a lot of oid:xx files in the object backend and AFAIK there is no easy way to change an installation from one storage to another (it's setup once at install time). All this means, that as @robi pointed out, this is really targeted at enterprises and people having full time dedicated sysadmins and nextcloud support contract. Don't get me wrong, I am sure it can be made to work but it's not something we want to support (best to pay nextcloud for this)
@girish Fair enough. Thanks.
-
@marcusquinn said in Help with Wasabi mounting:
@girish This article suggests there's an old and a new way to do this, so I wonder if we're all talking about the same thing?
old = adding an additional S3 store from within NC running locally.
new = setting up NC with S3 as primary storage.@robi I've managed to get this working using Digital Ocean Spaces on my Cloudron Nextcloud installation by following the instructions at this resource. This involved simply editing the config/config.php file to include the following:
'objectstore' => array ( 'class' => '\\OC\\Files\\ObjectStore\\S3', 'arguments' => array ( 'bucket' => 'yourbucketname', 'key' => 'YOURKEY', 'secret' => 'YOURSECRET', 'hostname' => 'nyc3.digitaloceanspaces.com', 'region' => 'nyc3', 'port' => 443, 'use_ssl' => true, 'use_path_style' => true, ), ), );I added this to the very end of my config.php, and before the last closing );
Functionality tested and working. Uploads and downloads happen directly within the DO Spaces bucket. Configuration persists between reboots.
I am new to Cloudron, so I am not certain if these changes will persist after updates to the Nextcloud app or Cloudron.
-
@robi I've managed to get this working using Digital Ocean Spaces on my Cloudron Nextcloud installation by following the instructions at this resource. This involved simply editing the config/config.php file to include the following:
'objectstore' => array ( 'class' => '\\OC\\Files\\ObjectStore\\S3', 'arguments' => array ( 'bucket' => 'yourbucketname', 'key' => 'YOURKEY', 'secret' => 'YOURSECRET', 'hostname' => 'nyc3.digitaloceanspaces.com', 'region' => 'nyc3', 'port' => 443, 'use_ssl' => true, 'use_path_style' => true, ), ), );I added this to the very end of my config.php, and before the last closing );
Functionality tested and working. Uploads and downloads happen directly within the DO Spaces bucket. Configuration persists between reboots.
I am new to Cloudron, so I am not certain if these changes will persist after updates to the Nextcloud app or Cloudron.
