<![CDATA[Backups to Storj suddenly fails: "Error listing objects in snapshot/app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf. Message: ListObjects(V2): listing too many items for gateway-side filtering"]]>As of this morning, my backups to Storj suddenly fails. This is the error message I get.

Error listing objects in snapshot/app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf. Message: ListObjects(V2): listing too many items for gateway-side filtering using arbitrary delimiter/prefix HTTP Code: NotImplemented

Any ideas?

]]>https://forum.cloudron.io/topic/11612/backups-to-storj-suddenly-fails-error-listing-objects-in-snapshot-app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf-message-listobjects-v2-listing-too-many-items-for-gateway-side-filteringRSS for NodeFri, 17 Apr 2026 02:33:58 GMTFri, 26 Apr 2024 09:02:37 GMT60<![CDATA[Reply to Backups to Storj suddenly fails: "Error listing objects in snapshot/app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf. Message: ListObjects(V2): listing too many items for gateway-side filtering" on Sat, 04 May 2024 08:10:02 GMT]]>I put storj separately at https://forum.cloudron.io/topic/11670/add-storj-as-backup-option

]]>https://forum.cloudron.io/post/87962https://forum.cloudron.io/post/87962Sat, 04 May 2024 08:10:02 GMT<![CDATA[Reply to Backups to Storj suddenly fails: "Error listing objects in snapshot/app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf. Message: ListObjects(V2): listing too many items for gateway-side filtering" on Sat, 04 May 2024 08:07:12 GMT]]>@thoresson mm, ok. Usually, each storage provider has their own quirk (like the one you reported). I think we have to test with storj properly to see all the issues. I think this was requested in https://forum.cloudron.io/topic/5171/add-storj-and-filebase-as-backup-options/ . We have to test this when we add that storage provider.

]]>https://forum.cloudron.io/post/87959https://forum.cloudron.io/post/87959Sat, 04 May 2024 08:07:12 GMT<![CDATA[Reply to Backups to Storj suddenly fails: "Error listing objects in snapshot/app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf. Message: ListObjects(V2): listing too many items for gateway-side filtering" on Fri, 26 Apr 2024 11:40:42 GMT]]>Got this reply from Storj @girish @nebulon

Could you change some settings in this software to use a delimiter for prefixes? By default it's `/` (a forward slash). Without delimiting you will hit the listing limit if you would use S3.  
As an alternative you may use a native Storj protocol, it doesn't have this limit, but I'm not sure that you can use it in this software.  
What's supposed to do? if backups, then I can suggest to use a normal ones, who can use prefixes correctly (and may reduce your storage costs by using a larger segments (less or equal to 64MiB)): [https://docs.storj.io/dcs/third-party-tools#backups](https://docs.storj.io/dcs/third-party-tools#backups)  
   
Our S3 compatibility you can see there: [https://docs.storj.io/dcs/api/s3/s3-compatibility](https://docs.storj.io/dcs/api/s3/s3-compatibility)  
However, there is a limitation, how much objects you can list using the not limited prefix (like a prefix without a trailing `/`).
]]>https://forum.cloudron.io/post/87526https://forum.cloudron.io/post/87526Fri, 26 Apr 2024 11:40:42 GMT<![CDATA[Reply to Backups to Storj suddenly fails: "Error listing objects in snapshot/app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf. Message: ListObjects(V2): listing too many items for gateway-side filtering" on Fri, 26 Apr 2024 10:06:55 GMT]]>@nebulon Sure – if I only can figure out how to see how many objects there are. 🙂 Do you know?

@girish Yeah, using rsync. Can try tgz once I've checked in with Storj's support.

]]>https://forum.cloudron.io/post/87517https://forum.cloudron.io/post/87517Fri, 26 Apr 2024 10:06:55 GMT<![CDATA[Reply to Backups to Storj suddenly fails: "Error listing objects in snapshot/app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf. Message: ListObjects(V2): listing too many items for gateway-side filtering" on Fri, 26 Apr 2024 10:00:38 GMT]]>If you are using rsync, maybe you can try tgz. Looks like S3 API has some incompatibility with too many files?

]]>https://forum.cloudron.io/post/87515https://forum.cloudron.io/post/87515Fri, 26 Apr 2024 10:00:38 GMT<![CDATA[Reply to Backups to Storj suddenly fails: "Error listing objects in snapshot/app_0c3fb7c9-77c7-4d96-a90a-b75b5d1562bf. Message: ListObjects(V2): listing too many items for gateway-side filtering" on Fri, 26 Apr 2024 09:46:34 GMT]]>oh seems like Storj does not implement some S3 feature for filtering. Can you check how many objects you have in that path in your bucket?

Can you maybe contact them to let them know and i maybe get some info if they will add this?

]]>https://forum.cloudron.io/post/87512https://forum.cloudron.io/post/87512Fri, 26 Apr 2024 09:46:34 GMT