terminal goes blank/crashed?

[jamescridland]

I'm opening the Mastodon terminal from Cloudron, and it allows me to use some bin/tootctl commands perfectly happily.

However, if you try doing a relatively taxing one - like this one for example...

RAILS_ENV=production bin/tootctl media refresh --domain=mastodon.nz --verbose

...then it runs for 30 seconds or so, and then the terminal screen goes blank (with the header still visible). I think it crashes most of the app - it seems to make everything grind to a halt and even Cloudron seems to be inoperable for a few minutes.

This is a bit of a pain, to be honest. Is there any way I can find out what's going wrong? I can't see anything obvious in the logs, but I'm unsure which log to look at.

I've given Mastodon 2GB and 75% CPU capacity.

[subven]

Is your hardware appropriate? Double the RAM for Mastodon and try again. Open app logs + box (Cloudron) logs and keep them open to monitor the output for errors or warnings. SSH the server and use journalctl -f to follow local logs. You can also use tools like htop or docker stats to look for hanging processes and bottlenecks.

[jamescridland]

@subven said in terminal goes blank/crashed?:

Is your hardware appropriate?

That one question doesn't appear to have an obvious answer - I don't know. (Nowhere does it tell me what appropriate hardware might be!)

But I don't see anything in the log files; I'll do the journalctl test next time I give it a go.

[nebulon]

@jamescridland if your terminal also gets disconnected, most likely it gets killed by the platform due to memory limit. Also be aware, that the memory limit is just the upper max limit, and it can also be that your system overall runs temporarily out of memory itself and your process gets killed.

[jamescridland]

@nebulon Cloudron automatically gave Mastodon 512MB, which clearly wasn't enough. I've given it 2GB. That seems not enough either! Yikes.

The box is a 2 GB Memory / 1 AMD vCPU / 50 GB Disk

[jdaviescoates]

@jamescridland said in terminal goes blank/crashed?:

@nebulon Cloudron automatically gave Mastodon 512MB, which clearly wasn't enough. I've given it 2GB. That seems not enough either! Yikes.

The box is a 2 GB Memory / 1 AMD vCPU / 50 GB Disk

Yeah, methinks you'll need a bigger VPS than that for Mastodon.

Seemingly postgresql needs a fair bit of memory too.

Also, 50GB wont be enough if this is enough for a public instance, it will very quickly fill up with all the media from the federated timeline.

[nebulon]

My personal instance works fine with 1Gb memory limit set, I think in this case that command just consumes a lot of memory temporarily, so possibly increasing the memory limit temporarily is fine as well. Also with the latest package, the disk space consumption is in our instances drastically reduced to a few GBs, it surely depends on the amount of users and the grade of federation though.

[subven]

@jamescridland keep in mind that other Cloudron services also use memory even if they are not needed (sftp, turn..). Such a small VPS also lacks CPU performance so you might need a bigger instance or try to minimize ressource use in general.

I was able to run Cloudron + Minio on a 512mb storage VPS but that was a use case where ressource usage was not demanding and without spikes.

[jamescridland]

@subven In this case, the Mastodon server is a personal instance. The disk space isn't an issue (now!) - I'm shunting all the media into an S3 bucket.

I'm aware that other Cloudron services need memory. Because there's swap on the box as well, I'm giving Mastodon 2GB, which is seemingly fine as long as it doesn't run any of those maintenance scripts.

But, if Mastodon isn't capable of being run in 2GB properly, my own suggestion would be to ( at least) warn Cloudron users with that sort of memory that it won't run properly.

[nebulon]

@jamescridland maybe to take a step back here, what is the reason to run such commands? The media cache is already maintained by the running app ondemand.