Latest attempt - why oh why does this just not work????
-
Hi everyone,
I thought I'd take another stab at getting my Peertube to use Object Storage, using a Cloudron-installed Minio instance (on a different server). Here is what I have in my production.yaml:object_storage: enabled: true driver: 'minio' pathstyle: 'true' endpoint: 'https://minioapi.server.com' (it used to be minio-api... but I removed the - like another poster did, and then at least the upload part would start. Before, not always, as I messed with different settings.) region: 'ca-central-1' upload_acl: # Set this ACL on each uploaded object of public/unlisted videos public: 'public' # Set this ACL on each uploaded object of private/internal videos # PeerTube can proxify requests to private objects so your users can access them private: 'private' protocol: 'HTTPS' max_upload_part: '2GB' web_videos: bucketName: 'vids' prefix: 'web/' # Use the same bucket as for web videos but with a different prefix streaming_playlists: bucketName: 'vids' prefix: 'streaming/' credentials: access_key_id: 'longkey' secret_access_key: 'longerkey'
ar 31 16:31:09 [example.com:443] 2024-03-31 14:31:09.337 info: Creating external storage move job for video 1b4a83e8-c474-4b28-bcf2-4ccacadc15e1. Mar 31 16:31:09 [example.com:443] 2024-03-31 14:31:09.352 info: Moving video 1b4a83e8-c474-4b28-bcf2-4ccacadc15e1 to object storage in job 13. Mar 31 16:31:09 [example.com:443] 2024-03-31 14:31:09.957 info: Initialized S3 client https://minioapi.server.com with region ca-central-1. Mar 31 16:31:09 [example.com:443] 2024-03-31 14:31:09.968 info: Processing transcoding job builder in job af6ed39e-1e1f-40e4-af70-20b84f2c2a1d. Mar 31 16:31:10 172.18.0.1 - - [31/Mar/2024:14:31:10 +0000] "GET / HTTP/1.1" 200 6492 "-" "Mozilla (CloudronHealth)" Mar 31 16:31:10 [example.com:443] 2024-03-31 14:31:10.282 info: 172.18.0.1 - - [31/Mar/2024:14:31:10 +0000] "GET / HTTP/1.0" 200 18062 "-" "Mozilla (CloudronHealth)" Mar 31 16:31:12 [example.com:443] 2024-03-31 14:31:12.270 error: Cannot move video https://example.com/videos/watch/1b4a83e8-c474-4b28-bcf2-4ccacadc15e1 storage. { Mar 31 16:31:12 "err": { Mar 31 16:31:12 "stack": "CredentialsProviderError: Could not load credentials from any providers\n at /app/code/server/node_modules/@aws-sdk/credential-provider-node/dist-cjs/defaultProvider.js:13:11\n at /app/code/server/node_modules/@smithy/property-provider/dist-cjs/chain.js:12:39\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async coalesceProvider (/app/code/server/node_modules/@smithy/property-provider/dist-cjs/memoize.js:14:24)\n at async SignatureV4.credentialProvider (/app/code/server/node_modules/@smithy/property-provider/dist-cjs/memoize.js:33:24)\n at async SignatureV4.signRequest (/app/code/server/node_modules/@smithy/signature-v4/dist-cjs/SignatureV4.js:106:29)\n at async /app/code/server/node_modules/@aws-sdk/middleware-signing/dist-cjs/awsAuthMiddleware.js:16:18\n at async /app/code/server/node_modules/@smithy/middleware-retry/dist-cjs/retryMiddleware.js:27:46\n at async /app/code/server/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/flexibleChecksumsMiddleware.js:57:20\n at async /app/code/server/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/region-redirect-endpoint-middleware.js:14:24\n at async /app/code/server/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/region-redirect-middleware.js:9:20\n at async /app/code/server/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:7:26\n at async Promise.all (index 0)\n at async Upload.__uploadUsingPut (/app/code/server/node_modules/@aws-sdk/lib-storage/dist-cjs/Upload.js:66:26)\n at async Upload.__doConcurrentUpload (/app/code/server/node_modules/@aws-sdk/lib-storage/dist-cjs/Upload.js:128:28)\n at async Promise.all (index 0)\n at async Upload.__doMultipartUpload (/app/code/server/node_modules/@aws-sdk/lib-storage/dist-cjs/Upload.js:211:9)\n at async Upload.done (/app/code/server/node_modules/@aws-sdk/lib-storage/dist-cjs/Upload.js:39:16)\n at async uploadToStorage (file:///app/code/server/dist/core/lib/object-storage/shared/object-storage-helpers.js:152:23)\n at async moveWebVideoFiles (file:///app/code/server/dist/core/lib/job-queue/handlers/move-to-object-storage.js:39:25)\n at async moveToJob (file:///app/code/server/dist/core/lib/job-queue/handlers/shared/move-video.js:19:13)\n at async Object.processMoveToObjectStorage [as move-to-object-storage] (file:///app/code/server/dist/core/lib/job-queue/handlers/move-to-object-storage.js:16:5)\n at async Object.wrapPromiseFun (file:///app/code/server/dist/core/lib/plugins/hooks.js:8:24)\n at async Worker.processJob (/app/code/server/node_modules/bullmq/dist/cjs/classes/worker.js:350:28)\n at async Worker.retryIfFailed (/app/code/server/node_modules/bullmq/dist/cjs/classes/worker.js:535:24)", Mar 31 16:31:12 "message": "Could not load credentials from any providers", Mar 31 16:31:12 "tryNextLink": false, Mar 31 16:31:12 "name": "CredentialsProviderError", Mar 31 16:31:12 "$metadata": { Mar 31 16:31:12 "attempts": 1, Mar 31 16:31:12 "totalRetryDelay": 0 Mar 31 16:31:12 } Mar 31 16:31:12 } Mar 31 16:31:12 } Mar 31 16:31:12 [example.com:443] 2024-03-31 14:31:12.287 error: Cannot execute job 13 in queue move-to-object-storage. { Mar 31 16:31:12 "payload": { Mar 31 16:31:12 "videoUUID": "1b4a83e8-c474-4b28-bcf2-4ccacadc15e1", Mar 31 16:31:12 "isNewVideo": true, Mar 31 16:31:12 "previousVideoState": 2 Mar 31 16:31:12 }, Mar 31 16:31:12 "err": { Mar 31 16:31:12 "stack": "CredentialsProviderError: Could not load credentials from any providers\n at /app/code/server/node_modules/@aws-sdk/credential-provider-node/dist-cjs/defaultProvider.js:13:11\n at /app/code/server/node_modules/@smithy/property-provider/dist-cjs/chain.js:12:39\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async coalesceProvider (/app/code/server/node_modules/@smithy/property-provider/dist-cjs/memoize.js:14:24)\n at async SignatureV4.credentialProvider (/app/code/server/node_modules/@smithy/property-provider/dist-cjs/memoize.js:33:24)\n at async SignatureV4.signRequest (/app/code/server/node_modules/@smithy/signature-v4/dist-cjs/SignatureV4.js:106:29)\n at async /app/code/server/node_modules/@aws-sdk/middleware-signing/dist-cjs/awsAuthMiddleware.js:16:18\n at async /app/code/server/node_modules/@smithy/middleware-retry/dist-cjs/retryMiddleware.js:27:46\n at async /app/code/server/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/flexibleChecksumsMiddleware.js:57:20\n at async /app/code/server/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/region-redirect-endpoint-middleware.js:14:24\n at async /app/code/server/node_modules/@aws-sdk/middleware-sdk-s3/dist-cjs/region-redirect-middleware.js:9:20\n at async /app/code/server/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:7:26\n at async Promise.all (index 0)\n at async Upload.__uploadUsingPut (/app/code/server/node_modules/@aws-sdk/lib-storage/dist-cjs/Upload.js:66:26)\n at async Upload.__doConcurrentUpload (/app/code/server/node_modules/@aws-sdk/lib-storage/dist-cjs/Upload.js:128:28)\n at async Promise.all (index 0)\n at async Upload.__doMultipartUpload (/app/code/server/node_modules/@aws-sdk/lib-storage/dist-cjs/Upload.js:211:9)\n at async Upload.done (/app/code/server/node_modules/@aws-sdk/lib-storage/dist-cjs/Upload.js:39:16)\n at async uploadToStorage (file:///app/code/server/dist/core/lib/object-storage/shared/object-storage-helpers.js:152:23)\n at async moveWebVideoFiles (file:///app/code/server/dist/core/lib/job-queue/handlers/move-to-object-storage.js:39:25)\n at async moveToJob (file:///app/code/server/dist/core/lib/job-queue/handlers/shared/move-video.js:19:13)\n at async Object.processMoveToObjectStorage [as move-to-object-storage] (file:///app/code/server/dist/core/lib/job-queue/handlers/move-to-object-storage.js:16:5)\n at async Object.wrapPromiseFun (file:///app/code/server/dist/core/lib/plugins/hooks.js:8:24)\n at async Worker.processJob (/app/code/server/node_modules/bullmq/dist/cjs/classes/worker.js:350:28)\n at async Worker.retryIfFailed (/app/code/server/node_modules/bullmq/dist/cjs/classes/worker.js:535:24)", Mar 31 16:31:12 "message": "Could not load credentials from any providers", Mar 31 16:31:12 "tryNextLink": false, Mar 31 16:31:12 "name": "CredentialsProviderError", Mar 31 16:31:12 "$metadata": { Mar 31 16:31:12 "attempts": 1, Mar 31 16:31:12 "totalRetryDelay": 0 Mar 31 16:31:12 }
The main problem seems to be
CredentialsProviderError: Could not load credentials from any providers\n
Now, is this just because I don't have the exactly correct syntax? What would that be? What gets capitalized, what gets '', what gets indented and how many spaces? Or have I missed some settings on the bucket? It's public with read-write settings, and at least one user who is listed as having access. Or is it CORS??? I thought for sure this would work with a Cloudron-installed Minio.
If you DO have a working Cloudron-based Peertube successfully linked to a Minio instance, perhaps you could share your .env?
-
@scooke said in Latest attempt - why oh why does this just not work????:
If you DO have a working Cloudron-based Peertube successfully linked to a Minio instance, perhaps you could share your .env?
I've not tried it with a Cloudron Minio yet, but I do have it working with Scaleway Object Storage.
This is what I've got (which I now see appears to have auto-changed - presumably during some update - since I actually did it):
object_storage: enabled: true # Scaleway endpoint endpoint: 's3.fr-par.scw.cloud' region: 'fr-par' videos: bucket_name: 'bucket-name' prefix: 'videos/' # Use the same bucket as for webtorrent videos but with a different prefix streaming_playlists: bucket_name: 'bucket-name' prefix: 'streaming-playlists/' credentials: access_key_id: 'XXX' secret_access_key: 'XXX'
-
@jdaviescoates I might just move on over to Scaleway too then. Thanks.