Peertube export not working...

[DanTheMan]

Hi all,

I recently ran the export feature that is now presented in Peertube.
https://docs.joinpeertube.org/use/setup-account#account-import-export

Now i started the export job a couple of times, but it keeps failing when i want to include the videos in my account on (mine) Peertube install.
Checking the logs in Peertube i discovered the following:

Job: 7
Type: create-user-export
Processed on 25-5-24 21:16:25,192
Finished on 25-5-24 21:46:03,979

{
"userExportId": 7
}

Error: ENOENT: no such file or directory, stat '/app/data/storage/tmp_persistent/user-export-2-2024-05-25T19:16:25.187Z.zip'

I can see in the filemanager (production.yaml) of Peertube, that it has a line to directory tmp:
/app/data/storage/tmp/

But it's different from the error location where it wants to write the export to:
/app/data/storage/tmp_persistent/

So my guess is that the directory is not available in Peertube? And the export keeps failing is the result of that?
As i double checked the directory, i discovered that the directory exists in Cloudron....

By the way im not using any S3 storage for my Peertube installation, all is local storage on Cloudron itself.

[DanTheMan]

scrolling through the logs also shows this error on a different line:

Job: 4
Type: create-user-export
Processed on 25-5-24 15:39:17,223
Finished on 25-5-24 16:06:06,783

{
"userExportId": 4
}

Error
at Query.run (/app/code/server/node_modules/sequelize/lib/dialects/postgres/query.js:50:25)
at /app/code/server/node_modules/sequelize/lib/sequelize.js:315:28
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async PostgresQueryInterface.update (/app/code/server/node_modules/sequelize/lib/dialects/abstract/query-interface.js:355:12)
at async UserExportModel.save (/app/code/server/node_modules/sequelize/lib/model.js:2490:35)
at async file:///app/code/server/dist/core/helpers/database-utils.js:31:13
at async /app/code/server/node_modules/sequelize/lib/sequelize.js:507:18

Not sure what that's all about and/or if it's related in some way...

[girish]

For me this "just worked" (tm)

I was able to download archive as well. Even got some email notification when the archive was ready to download.

[DanTheMan]

Thanks @girish for checking.

I tried it with an other account (test-account) on the same Peertube instance.
Strangely this test-account gives an succesfull export.

So my thought is that the export from the account that throws the error, simply is too big to export maybe?
(around 19GB)

I'm now trying to delete some videos to get to a smaller volume and see if that helps...

Maybe it's worth mentioning this error i see in postgresql on Cloudron itself:
ERROR: value "18519529041" is out of range for type integer<27>1 2024-06-01T13:41:40+02:00 cloudronserver postgresql 920 postgresql - 2024-06-01 11:41:40.171 UTC [2226] userdxxxxxxx4b34b2e2b12ab16a57eb@xxxxxxdbd6d0b666 CONTEXT: unnamed portal parameter $1 = '...'<27>1 2024-06-01T13:41:40+02:00 cloudronserver postgresql 920 postgresql - 2024-06-01 11:41:40.171 UTC [2226] userdxxxxxxx4b34b2e2b12ab16a57eb@xxxxxxdbd6d0b666 STATEMENT: UPDATE "userExport" SET "size"=$1,"state"=$2,"error"=$3,"updatedAt"=$4 WHERE "id" = $5

[luckow]

and then

[DanTheMan]

That one is actually really easy to solve. Only thing is, that my Peertube instance is in Dutch language and i can't find an option to change it 🥴

Anyway, i shall do my best to help you get there, because there's a setting for that...

As Peertube admin logged into your instance,
Go to:

Administration > Configuration > Basic >

In there you scroll (almost) all the way down, until you reached the "user import/export" settings.
There you will also find the value for max take out.

Set it to your desired value and voila

[girish]

@DanTheMan the postgres error seems like an app issue. Maybe worth reporting upstream.

[luckow]

@DanTheMan same here

[DanTheMan]

Thanks for the report

definitely a somewhere...

I already did some deep diving into the error, but it's kinda complicated...

I found an older post that explains a little about the error in relation with postgresql..
https://stackoverflow.com/questions/52238781/postgres-integer-out-of-range-why-this-error-occur

And some pure details:
https://pganalyze.com/docs/log-insights/app-errors/U130#:~:text=Explanation%3A,large value into the database.

I guess it's better (like Girish mentioned) to report this upstream with the developers of Peertube...

@girish I'm no database expert, but isn't this something that has to be done on the database that holds Peertube on Cloudron, Postgresql?
Like to adjust the integer to the appropriate values of the database of Peertube?

I'm gonna report it on their Github and keep this forum post updated when I have more info from there...

[girish]

@DanTheMan said in Peertube export not working...:

Like to adjust the integer to the appropriate values of the database of Peertube?

My reading of the error is that the app (peertube) contructed a sql query and the query has a bug in it. Let me know what the upstream devs say, happy to fix accordingly.

[DanTheMan]

I reported upstream and this is the outcome:
https://github.com/Chocobozzz/PeerTube/commit/e7ebcb16acbd43043fd7f95e8918b8ab50595796

Should be fixed within the next update(s) of Peertube

I'll keep an eye open 🧐

[DanTheMan]

Peertube has a new release, the export problem should be fixed in this one

"Fix big user export file size"
https://github.com/Chocobozzz/PeerTube/releases/tag/v6.2.0#:~:text=Fix big user export file size

[nebulon]

The release is out by now. Does it work again?

[DanTheMan]

I saw this post as still being "unsolved".

Yesterday i succefully downloaded a exported backup from my Peertube instance (35GB).

So that's working fine now

Did not yet import it to another instance. But will try to do that in a short matter of time...