Regenerate video thumbnails

[mdc773]

Sure yes I was able to get this to work but I lost a lot of thumbnails I can't get back. Anyway possible for me to set up sending thumbnails to my objective storage.

[mdc773]

See my issue that I spoke about in my first post was that my peertube failed to send uploads to my Idrive-e3. I don't know what happened all I noticed was that cloudron said my main drive was full so when I check system info I seen that my peertube took up all the space randomly I literally have 2.5tb of videos on my E3 but strange how everything decided to save on the main drive. When my config states that it needs to be stored on the objective storage. So when I looked for a backup I only had a backup from a week ago sadly I didn't have a recent one so when I reverted back to the backup from a week ago I lost lots of thumbnails but makes no sense because some of those videos you've been on there for months. Now I don't know what to do on getting back theses thumbnails I'm talking like of videos

[girish]

@mdc773 You can run the command directly without credentials.

Initially, I have no thumbnails:

So, I recreate them like this:

root@28ad76ef-962c-4fd8-a2b0-00d25cec8109:/app/code/server# gosu cloudron:cloudron npm run regenerate-thumbnails

> peertube@5.2.1 regenerate-thumbnails
> node ./dist/scripts/regenerate-thumbnails.js

Processing video Cloudron Test Video.

I got the thumbnail back:

[mdc773]

Thanks for your findings, i actually figured out this way of doing it as well but i find some thumbnails but then fails on some.. plus I am having the problem again when my peertube fails to send the videos to my objective storage (idrive-e3) so what happen is peertube save the videos locally and its about 1.5TB then it takes up all the space and the app stops..... this the seconded time this happened so idk if the problem is idrive or cloudron

[mdc773]

[mdc773]

This post is deleted!

[robi]

Btw, it's "object storage".

It's an interesting failure scenario that should be investigated with the peertube guys upstream.

Perhaps they need better retries and resilience on their s3 backend code and options for not saving locally if that fails.

[mdc773]

thanks will do
and yeah lol object storage

[jdaviescoates]

@mdc773 said in Regenerate video thumbnails:

so what happen is peertube save the videos locally and its about 1.5TB then it takes up all the space and the app stops..... this the seconded time this happened so idk if the problem is idrive or cloudron

I was having similar problems with large videos when I was trying to run PeerTube on a server that didn't really have enough RAM / CPU to power it. Since I've been on a more powerful server everything has been working fine. I'd guess you're hitting a similar problem and what you need is a server upgrade.

[jdaviescoates]

@robi said in Regenerate video thumbnails:

options for not saving locally if that fails

I'm not sure, but I'd guess it does all the CPU intensive transcoding locally before uploading to S3.

[mdc773]

I appreciate everyone help even @jdaviescoates but hey i'm not 100% sure but i have a EX101 hetzner Intel Core i9-13900 24 Core with 128gb ram cant be the server peertube fails and i need to find the exact log.. at the moment i plan to simulate the problem with a massive upload :?

[jdaviescoates]

@mdc773 certainly sounds like server power is not the issue!

Just to be sure though, have you also cranked up the app resources in the app settings?

[girish]

Maybe I missed it, what is the error message now?

[mdc773]

well i am trying to simulate this at the moment..

[mdc773]

@jdaviescoates yes i have

[mdc773]

warn[10/24/2023, 3:39:37 PM] Object storage max upload part seems to have a big value (1073741824 bytes). Consider using a lower one (like 100MB). 

I have set this up long time ago i don't fully understand max upload part but I've set to 100MB like it says same problem peertube support is very slow they have a matrix chat but barley reply

i really need some help my idea for object storage was to save local space which i think this why all of us do this

[girish]

@mdc773 said in Regenerate video thumbnails:

i don't fully understand max upload part

On S3 compatible storage, you can upload large files, like 5TB. It would be infeasible to upload such large files in a single shot. So, they have APIs to upload files in "parts". Like you upload 10k parts of 5GB each for example to get to 5TB. The number of parts and size of each part depends on the S3 storage.

Usually, I leave upload part size to something like 10MB-50MB. There is actually nothing wrong with 100MB part size, but for some reason peertube is complaining about it.