Thanks for the explanation.

This is currently not possible and not really planned either so far. Maybe you can use the mountpoint backup location on your system as storing backups on the main disk isn't a good idea in the long run anyways https://docs.cloudron.io/backups/#filesystem-mountpoint

Another example here too (a much longer one): Dec 22 23:24:14 WARN [qtp24334184-588] 07:24:14,205 org.apache.tika.server.core.resource.TikaResource tika/: Text extraction failed (=?utf-8?B?Tm8gMjEgeGlhb3FpbmcgemhhbmcuZG9jeA==?=) Dec 22 23:24:14 org.apache.tika.exception.TikaException: TIKA-198: Illegal IOException from org.apache.tika.parser.pkg.PackageParser@6166fb02 Dec 22 23:24:14 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:304) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:298) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:204) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.server.core.resource.TikaResource.parse(TikaResource.java:363) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.server.core.resource.TikaResource.lambda$produceText$1(TikaResource.java:509) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.jaxrs.provider.BinaryDataProvider.writeTo(BinaryDataProvider.java:176) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.jaxrs.utils.JAXRSUtils.writeMessageBody(JAXRSUtils.java:1651) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.serializeMessage(JAXRSOutInterceptor.java:249) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.processResponse(JAXRSOutInterceptor.java:122) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.jaxrs.interceptor.JAXRSOutInterceptor.handleMessage(JAXRSOutInterceptor.java:84) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:90) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:244) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:80) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:223) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1381) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:178) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1303) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:149) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.Server.handle(Server.java:563) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.HttpChannel$RequestDispatchable.dispatch(HttpChannel.java:1598) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:753) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:501) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:287) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:421) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:390) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:277) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.run(AdaptiveExecutionStrategy.java:199) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149) [tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at java.base/java.lang.Thread.run(Thread.java:840) [?:?] Dec 22 23:24:14 Caused by: org.apache.commons.io.TaggedIOException: invalid code lengths set Dec 22 23:24:14 at org.apache.commons.io.input.TaggedInputStream.handleIOException(TaggedInputStream.java:93) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:228) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.gagravarr.tika.OggDetector.detect(OggDetector.java:68) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.java:84) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:179) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:71) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbedded(ParsingEmbeddedDocumentExtractor.java:111) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.java:481) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parseEntries(PackageParser.java:386) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser._parse(PackageParser.java:336) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:259) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:298) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 ... 41 more Dec 22 23:24:14 Caused by: org.apache.commons.io.TaggedIOException: invalid code lengths set Dec 22 23:24:14 at org.apache.commons.io.input.TaggedInputStream.handleIOException(TaggedInputStream.java:93) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:228) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:224) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:224) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.gagravarr.tika.OggDetector.detect(OggDetector.java:68) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.java:84) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:179) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:71) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbedded(ParsingEmbeddedDocumentExtractor.java:111) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.java:481) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parseEntries(PackageParser.java:386) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser._parse(PackageParser.java:336) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:259) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:298) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 ... 41 more Dec 22 23:24:14 Caused by: java.util.zip.ZipException: invalid code lengths set Dec 22 23:24:14 at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.readFromInflater(ZipArchiveInputStream.java:1083) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.readDeflated(ZipArchiveInputStream.java:998) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.read(ZipArchiveInputStream.java:929) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:244) ~[?:?] Dec 22 23:24:14 at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:263) ~[?:?] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:224) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:224) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:224) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.gagravarr.tika.OggDetector.detect(OggDetector.java:68) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.java:84) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:179) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:71) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbedded(ParsingEmbeddedDocumentExtractor.java:111) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.java:481) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parseEntries(PackageParser.java:386) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser._parse(PackageParser.java:336) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:259) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:298) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 ... 41 more Dec 22 23:24:14 Caused by: java.util.zip.DataFormatException: invalid code lengths set Dec 22 23:24:14 at java.base/java.util.zip.Inflater.inflateBytesBytes(Native Method) ~[?:?] Dec 22 23:24:14 at java.base/java.util.zip.Inflater.inflate(Inflater.java:378) ~[?:?] Dec 22 23:24:14 at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.readFromInflater(ZipArchiveInputStream.java:1081) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.readDeflated(ZipArchiveInputStream.java:998) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.compress.archivers.zip.ZipArchiveInputStream.read(ZipArchiveInputStream.java:929) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:244) ~[?:?] Dec 22 23:24:14 at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:263) ~[?:?] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:224) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:224) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:224) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.gagravarr.tika.OggDetector.detect(OggDetector.java:68) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.java:84) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:179) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:71) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbedded(ParsingEmbeddedDocumentExtractor.java:111) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.java:481) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parseEntries(PackageParser.java:386) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser._parse(PackageParser.java:336) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:259) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:298) ~[tika-server-standard.jar:3.0.0] Dec 22 23:24:14 ... 41 more Dec 22 23:24:14 Dec 23, 2024 7:24:14 AM org.apache.cxf.jaxrs.utils.JAXRSUtils logMessageHandlerProblem Dec 22 23:24:14 SEVERE: Problem with writing the data, class org.apache.tika.server.core.resource.TikaResource$$Lambda$391/0x00007f41fc2ac740, ContentType: text/plain

That's great, thanks @girish! It seems to work now. I enabled it and followed the steps in the new documentation you linked to, and then ran doveadm -c /run/dovecot.conf index -A '*' from the mail container which started indexing everything. Just waiting for the indexing to finish to confirm it'll be good.

I would say lets close this here. I do not see any failed cleanup jobs even multiple jobs try todo the same.

@nebulon Not so clear. The volume disconnected again. The "Remount Volume" button did not help but "Edit Volume" and saving (unchanged settings) worked.

Omg, for whatever reason the TLD did change from .de to .com (I have .de but not .com) so the failure was totally correct. No problems, nothing to see here, please close the case Happy holidays!

Thank you for this. I ended up creating a DO Spaces bucket, putting the backup there, and then restoring from that. To make it work, I had to use the Dry Run option and then manually sync the DNS. But now it is working. I wish that Hostinger had something akin to DO Spaces, but it doesn't, so I am still trying to figure out my strategy for offsite backups. Thanks @nebulon for your help!

Fixed now . https://git.cloudron.io/platform/box/-/commit/17fee930027896c405e8bbe3fbe56204c18c2670 . Thanks for reporting

@berryd great you sorted it out!

@timconsidine said in Domain cert renewal - when ?: I don’t even remember my own name. Your name is Timcon S'idine, just in case.

@scooke I’m interested in knowing too as I’d like to add a few more things to my set up like a hardware firewall, rpi nas, and blueiris (nvr cam system) that can be accessed remotely but also uses my ip and that definitely will conflict with the Cloudron server. But like you said, why over complicate things? It’s the only reason why i didn’t look deeper into this.

@crazybrad said in Unmounting volume leaves the "backup volume": Listmonk: Docker Error: Unable to pull image cloudron/app.listmonk.cloudronapp:202411131609490000. Please check the network or if the image needs authentication. statusCode: 500 This is the ipv6 rate limit issue which many are hitting. dockerhub seems to pushed some bad implementation of rate limits and it's wholesale cutting off network providers from accessing dockerhub. We have a workaround in place now for the new release.

On a brand new DO Ubuntu server, the first pull fails. root@my:~# TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token) curl --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5429 0 5429 0 0 17471 0 --:--:-- --:--:-- --:--:-- 17512 HTTP/1.1 200 OK content-length: 527 content-type: application/vnd.docker.distribution.manifest.v2+json docker-content-digest: sha256:c2d41d2ba6d8b7b4a3ffec621578eb4d9a0909df29dfa2f6fd8a2e5fd0836aed docker-distribution-api-version: registry/2.0 etag: "sha256:c2d41d2ba6d8b7b4a3ffec621578eb4d9a0909df29dfa2f6fd8a2e5fd0836aed" date: Sat, 14 Dec 2024 15:41:35 GMT strict-transport-security: max-age=31536000 ratelimit-limit: 100;w=21600 ratelimit-remaining: 0;w=21600 docker-ratelimit-source: 2604:a880:2:d1::

Right, yes, that would make sense!

Thank you! I'm working through the platform updates now. I didn't realize it was so out of date, as I had automatic updates turned on. Discovered that the same issue causing my backups to fail was also preventing the platform updates from completing. Manually stopping the Snipe-IT app allowed the updates to complete successfully, so I'm assuming this will be resolved, but will follow up here if not. Really appreciate the assistance.

It was happening across multiple servers at the same time yesterday. Looks to be resolved now.

The issue is now resolved. The problem was with the GitLab database, where we found 160 GB of logs. The database dump was failing because there wasn’t enough free disk space. We have purged the database, and backups are now successful. Thank you for your support.

Thanks for the heads up. Added for the next release.