Transcode after import fails

[girish]

OK, I have made a PR upstream - https://github.com/Chocobozzz/PeerTube/pull/3198. In the meantime, I have pushed a new package, can you check if transcoding works with that?

I found other sporadic errors:

• "Killed" - this means you have to give peertube more memory
• "Conversion failed" - not sure why this happens but it happens with some youtube videos.

[imc67]

@girish again not succesfull (sorry ) :

• I gave the app 8GB memory (was already all the time 8GB)
• the jobs are successful
• the video preview (still) is visible in the PeerTube GUI
• playing doesn't work at all in the PeerTube GUI and the 3 resolutions are not in the 'menu'
• I can see by processor use and upload speed in Zabbix that "something is happening"
• generated files are everywhere (except videos folder, see screenshots)
• the HLS mp4 files I can download them via SCP and play them correctly in the correct resolution
• import via YT URL or upload via file results in the same situation
• I doubt if the error logs are related to this problem, it feels more like that the Peertube GUI "is looking to the wrong place" for the video files?
• however, if I download the video via the Peertube GUI, the correct path is there and the download is succesful

Screenshots of the Filemanager folders:

Error log:

warn[16-10-2020 10:43:20] Cannot authenticate.

{
  "err": {
    "statusCode": 401,
    "status": 401,
    "code": 401,
    "message": "Invalid token: access token has expired",
    "name": "invalid_token",
    "stack": "invalid_token: Invalid token: access token has expired\n    at new InvalidTokenError (/app/code/server/node_modules/oauth2-server/lib/errors/invalid-token-error.js:25:14)\n    at AuthenticateHandler.validateAccessToken (/app/code/server/node_modules/oauth2-server/lib/handlers/authenticate-handler.js:227:11)\n    at AuthenticateHandler.<anonymous> (/app/code/server/node_modules/oauth2-server/lib/handlers/authenticate-handler.js:77:19)\n    at PassThroughHandlerContext.finallyHandler (/app/code/server/node_modules/bluebird/js/release/finally.js:57:23)\n    at PassThroughHandlerContext.tryCatcher (/app/code/server/node_modules/bluebird/js/release/util.js:16:23)\n    at Promise._settlePromiseFromHandler (/app/code/server/node_modules/bluebird/js/release/promise.js:547:31)\n    at Promise._settlePromise (/app/code/server/node_modules/bluebird/js/release/promise.js:604:18)\n    at Promise._settlePromise0 (/app/code/server/node_modules/bluebird/js/release/promise.js:649:10)\n    at Promise._settlePromises (/app/code/server/node_modules/bluebird/js/release/promise.js:729:18)\n    at _drainQueueStep (/app/code/server/node_modules/bluebird/js/release/async.js:93:12)\n    at _drainQueue (/app/code/server/node_modules/bluebird/js/release/async.js:86:9)\n    at Async._drainQueues (/app/code/server/node_modules/bluebird/js/release/async.js:102:5)\n    at Immediate.Async.drainQueues [as _onImmediate] (/app/code/server/node_modules/bluebird/js/release/async.js:15:14)\n    at processImmediate (internal/timers.js:456:21)"
  }
}

warn[16-10-2020 10:43:20] Cannot authenticate.

{
  "err": {
    "statusCode": 401,
    "status": 401,
    "code": 401,
    "message": "Invalid token: access token has expired",
    "name": "invalid_token",
    "stack": "invalid_token: Invalid token: access token has expired\n    at new InvalidTokenError (/app/code/server/node_modules/oauth2-server/lib/errors/invalid-token-error.js:25:14)\n    at AuthenticateHandler.validateAccessToken (/app/code/server/node_modules/oauth2-server/lib/handlers/authenticate-handler.js:227:11)\n    at AuthenticateHandler.<anonymous> (/app/code/server/node_modules/oauth2-server/lib/handlers/authenticate-handler.js:77:19)\n    at PassThroughHandlerContext.finallyHandler (/app/code/server/node_modules/bluebird/js/release/finally.js:57:23)\n    at PassThroughHandlerContext.tryCatcher (/app/code/server/node_modules/bluebird/js/release/util.js:16:23)\n    at Promise._settlePromiseFromHandler (/app/code/server/node_modules/bluebird/js/release/promise.js:547:31)\n    at Promise._settlePromise (/app/code/server/node_modules/bluebird/js/release/promise.js:604:18)\n    at Promise._settlePromise0 (/app/code/server/node_modules/bluebird/js/release/promise.js:649:10)\n    at Promise._settlePromises (/app/code/server/node_modules/bluebird/js/release/promise.js:729:18)\n    at _drainQueueStep (/app/code/server/node_modules/bluebird/js/release/async.js:93:12)\n    at _drainQueue (/app/code/server/node_modules/bluebird/js/release/async.js:86:9)\n    at Async._drainQueues (/app/code/server/node_modules/bluebird/js/release/async.js:102:5)\n    at Immediate.Async.drainQueues [as _onImmediate] (/app/code/server/node_modules/bluebird/js/release/async.js:15:14)\n    at processImmediate (internal/timers.js:456:21)"
  }
}

warn[16-10-2020 10:45:18] Warning in tracker.

{
  "err": {
    "stack": "Error: Unknown infoHash 00dfd0272f82ae6cdb0061f5af6fb395b06ba4fa requested by ip **.**.**.**\n    at Server.<anonymous> (/app/code/server/dist/server/controllers/tracker.js:54:20)\n    at Generator.next (<anonymous>)\n    at fulfilled (/app/code/server/node_modules/tslib/tslib.js:111:62)",
    "message": "Unknown infoHash 00dfd0272f82ae6cdb0061f5af6fb395b06ba4fa requested by ip **.**.**.**"
  }
}

[girish]

@imc67 That's fine Good news is I think this is a different transcoding issue and the read only filesystem thing is sorted out, it's atleast converting now. Does this happen for all videos ? I found https://github.com/Chocobozzz/PeerTube/issues/2990 which may be related?

[imc67]

@girish I give up on this piece of software, it's not worth all the waiting and debugging to get it to work.

On 27th of July I wrote:

Hi @girish , I discovered PeerTube since it was announced here and was very enthusiastic because it fits in our needs. However during the past 6 weeks I'm beginning to loose faith in this project.

The project has a very slow "beat", they write they have only 1 (paid) developer. We had to wait 5 weeks to get an essential bug (not able to config) solved. After your last update it looked fine and I started to setup everything but then discovered a "stupid" bug again: videos don't play on iPad (https://github.com/Chocobozzz/PeerTube/issues/2941)! The developer knows this for at least 3 weeks but there seems no investigation/progress. Most of our viewers use iPad so we can't start (again) ...
There is also a very annoying issue that themes are alternating switching during browsing, so 1 click you're in dark mode and the next one in light mode (https://github.com/Chocobozzz/PeerTube/issues/3007). Beside that there are 350 other open issues ...

We're now 3 months later, you did an awful lot of work but it's still not working but it's not your fault!

It's not your mentioned issue 2990 because the video simply doesn't play, no matter how long you wait.

I will uninstall the app and wait for a better alternative ........

[girish]

No problem, I understand . We are probably very early adopters of PeerTube at this point.

[robi]

Would dtube be a viable replacement?

https://github.com/dtube
looks much more actively developed

[girish]

@imc67 said in Transcode after import fails:

@girish I give up on this piece of software, it's not worth all the waiting and debugging to get it to work.

On a positive note, Debian put in some money to help improve now Peertube - https://bits.debian.org/2020/10/debian-donation-peertube.html

[imc67]

@girish I really need to use "something" like PeerTube soon and I couldn't believe why it still doesn't work with HLS!

I did some testing and research:

1. On the Cloudron demo I installed PeerTube (also there I had this issue https://forum.cloudron.io/topic/3642/new-installation-not-able-to-log-in but solved it, I really think you should give it at least 1GB App memory and 1GB Redis memory to start)
2. I only switched on 2 kind of transcoding options and HLS (no WebTorrents)
3. Uploaded a mp4
4. waited for transcoding
5. Video did not play ....

So same issue with our own PeerTube instance on our own Cloudron.

Searching through their Git I discovered solved issues that might be related to Cloudron?

1.

https://github.com/Chocobozzz/PeerTube/issues/3181

The PeerTube developer asks here "Do you have a reverse proxy in front of docker? Because it's seems the server does not respect the byte range request", that was the case and they solved it.

Could it be a similar issue on Cloudron?

2.
Another issue on the Cloudron version: ffmpeg is not the right version!

Current:

ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
libavutil      55. 78.100 / 55. 78.100
libavcodec     57.107.100 / 57.107.100
libavformat    57. 83.100 / 57. 83.100
libavdevice    57. 10.100 / 57. 10.100
libavfilter     6.107.100 /  6.107.100
libavresample   3.  7.  0 /  3.  7.  0
libswscale      4.  8.100 /  4.  8.100
libswresample   2.  9.100 /  2.  9.100
libpostproc    54.  7.100 / 54.  7.100

Needed: You need ffmpeg >= 4.1 if you enable HLS (https://github.com/Chocobozzz/PeerTube/issues/2469#issuecomment-584563107)

[girish]

@imc67 Thanks for the detailed investigation! I have pushed a new package with ffmpeg 4.3

[imc67]

@girish thank you very much for your quick response and update! However .....

... it still doesn't work (although now it does show the start of the video!). For the last hour I've checked all of their Git and got inspired to look into my browser consoles (Safari on MacOS and Firefox on MacOS) and discovered in both a similar error that might be Cloudron related (but of course I'm not sure):

Safari:

Firefox:

It looks like Peertube in the local browser needs a kind of connection to the server but it's forbidden:

GET wss://videos.[domainname].[tld]/tracker/socket | [HTTP/1.1 403 Forbidden 64ms]

[girish]

@imc67 Can you try to see if the instances at https://joinpeertube.org/instances#instances-list work? Will help me understand if this is a packaging issue or an upstream issue.

I am not an expert on HLS. Can you tell me how I can test this? Does it require a specific kind of video or does it fail with any video ?

[imc67]

@girish I've found just at random a video and it plays well, I'm pretty sure it's playing in HLS, see screenshot:

https://tube.tardis.world/videos/watch/a56df39d-05d8-4a51-9393-b549ffd7167c

Screenshot of Network usage in browser (Firefox):
here you can see it's playing the 1080.m3u8 file (HLS), as far as I understand HLS this file "contains indexes" of the mp4 so it only retrieves from the browser at the right timing the right "slice" from the mp4 to play locally without retrieving first all the GB's of the mp4.

@girish "Does it require a specific kind of video or does it fail with any video ?" In Peertube settings you can choose for (only) HLS and Peertube makes these special files.

This is how it looks like for 1 video:

Configuration in Peertube:

[girish]

@imc67 thanks, I will have some time tomorrow to investigate this.

[imc67]

@girish said in Transcode after import fails:

@imc67 thanks, I will have some time tomorrow to investigate this.

I’m really curious if you could find the wss issue?

[girish]

@imc67 Too busy this week, has to be next week. FWIW, it's most likely not a wss issue (since we have apps like rocket.chat, cloudron dashboard, mattermost already using wss) but some PeerTube related issue.

[girish]

@imc67 Been trying this out. I can also see 403 error to wss. There is also some hls.js error with bufferStalledError. But the video plays anyway (in safari). Exact same as https://github.com/Chocobozzz/PeerTube/issues/655 (does safari block these by default maybe?)

Edit: I see same behavior in firefox on linux as well.

[imc67]

@girish said in Transcode after import fails:

does safari block these by default maybe?)

HLS was invented by Apple so I can't imagine they block it.

This video plays HLS very well: https://tube.tardis.world/videos/watch/a56df39d-05d8-4a51-9393-b549ffd7167c

[imc67]

@girish said in Transcode after import fails:

video plays anyway (in safari).

Does it play HLS or WebTorrent? Because for testing I also switched WebTorrent ON and then the video does play in Safari but in WebTorrent not HLS.

[girish]

@imc67 said in Transcode after import fails:

Does it play HLS or WebTorrent?

How can I make out? I actually disabled WebTorrent just like your initial transcoding configuration. Only HLS with P2P is enabled. But I would like to check how it's playing the video.

[girish]

@imc67 said in Transcode after import fails:

This video plays HLS very well: https://tube.tardis.world/videos/watch/a56df39d-05d8-4a51-9393-b549ffd7167c

Just to double check. This is not a Cloudron installation right? That video is 10GB for download, so I imagine it will take me forever to download and upload it into my test instance.