Cloudron update to 7.5.1 - Success!
-
Usually the disk is the limiting speed in all this. Then comes RAM and then CPU. What is the disk speed ? (It's in the System page in the graphs)
-
@mhgcic said in Cloudron update to 7.5.1 - Success!:
I could increase the RAM to see if that helps.
Regardless if it is what's causing the issues you're currently facing, I'd definitely get more RAM if I were you.
Each instance of Mastodon needs at least 1.5GB and each instance of Discourse at least 2GB. Already that is at least 10.5GB RAM you need before you've included all your other apps plus Cloudron and Ubuntu themselves.
And these are bare minimums. If you have users actually actively using these apps they'll need more.
-
@jdaviescoates said in Cloudron update to 7.5.1 - Success!:
Regardless if it is what's causing the issues you're currently facing, I'd definitely get more RAM if I were you.
One reason to love SSDnodes is their generous amounts of RAM.
-
You can also restart a single app and just time how much time it takes to restart it. Maybe try something which is not discourse. Discourse takes notoriously long to restart since it has to compile assets and also it's a rails app.
-
@LoudLemur said in Cloudron update to 7.5.1 - Success!:
One reason to love SSDnodes is their generous amounts of RAM.
I've read too many negative reviews (on here and elsewhere) about SSDnodes to be interested in using them. In my view "too good to be true" pricing is basically exactly that, i.e. too good to be true. There are loads of posts on here with people having issues with them (although admittedly also others that seem to be very happy with them)
My plan is to move to a dedicated Hetzner server like this one https://www.hetzner.com/dedicated-rootserver/ax41-nvme which comes with 64GB RAM that isn't shared or using any virtualization.
-
@jdaviescoates said in Cloudron update to 7.5.1 - Success!:
@LoudLemur said in Cloudron update to 7.5.1 - Success!:
One reason to love SSDnodes is their generous amounts of RAM.
I've read too many negative reviews (on here and elsewhere) about SSDnodes to be interested in using them.
Boom!
Agree 100%My plan is to move to a dedicated Hetzner server like this one https://www.hetzner.com/dedicated-rootserver/ax41-nvme which comes with 64GB RAM that isn't shared or using any virtualization.
That's wisdom, moreover their prices are very good for that high-quality tech and service. The auction section has monthly prices that will put you on your...
-
I'm surprised to see the Ryzen 3600 processor on there. That's from the desktop CPU lineup. It's a shame they don't have any affordable dedi's in the US though.
BTW @jdaviescoates
It looks like the AX41-NVME has a processing time of several working days in case you missed it.Just noticed there are more than one listing with the same name. NEVERMIND
-
@humptydumpty said in Cloudron update to 7.5.1 - Success!:
BTW @jdaviescoates It looks like the AX41-NVME has a processing time of several working days in case you missed it. Just noticed there are more than one listing with the same name. NEVERMIND
I was gonna say:
-
@jdaviescoates Oooh! That looks very tempting! Nice!
-
I restarted each discourse on its own, each one too about 60 Seconds - 1 minute 20 Seconds to restart.
So I added RAM 64GB still the same issue, so I watched the logs it seems to possibly be network issues.
Aug 30 06:05:192023-08-30 05:05:19,293 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. Aug 30 06:05:192023-08-30 05:05:19,305 INFO Included extra file "/etc/supervisor/conf.d/nginx.conf" during parsing Aug 30 06:05:192023-08-30 05:05:19,305 INFO Included extra file "/etc/supervisor/conf.d/puma.conf" during parsing Aug 30 06:05:192023-08-30 05:05:19,305 INFO Included extra file "/etc/supervisor/conf.d/sidekiq.conf" during parsing Aug 30 06:05:192023-08-30 05:05:19,305 INFO Included extra file "/etc/supervisor/conf.d/streaming.conf" during parsing Aug 30 06:05:19Unlinking stale socket /var/run/supervisor.sock Aug 30 06:05:192023-08-30 05:05:19,668 INFO RPC interface 'supervisor' initialized Aug 30 06:05:192023-08-30 05:05:19,668 CRIT Server 'unix_http_server' running without any HTTP authentication checking Aug 30 06:05:192023-08-30 05:05:19,668 INFO supervisord started with pid 1 Aug 30 06:05:20=> Healtheck error: Error: connect ECONNREFUSED 172.18.19.180:8000 Aug 30 06:05:202023-08-30 05:05:20,671 INFO spawned: 'puma' with pid 16 Aug 30 06:05:202023-08-30 05:05:20,673 INFO spawned: 'sidekiq' with pid 17 Aug 30 06:05:202023-08-30 05:05:20,676 INFO spawned: 'streaming' with pid 18 Aug 30 06:05:202023-08-30 05:05:20,759 INFO spawned: 'nginx' with pid 19 Aug 30 06:05:212023-08-30 05:05:21,812 INFO success: puma entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) Aug 30 06:05:212023-08-30 05:05:21,814 INFO success: sidekiq entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) Aug 30 06:05:212023-08-30 05:05:21,816 INFO success: streaming entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) Aug 30 06:05:212023-08-30 05:05:21,817 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) Aug 30 06:05:26172.18.0.1 - - [30/Aug/2023:05:05:26 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:05:262023/08/30 05:05:26 [error] 26#26: *1 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:05:28172.18.0.1 - - [30/Aug/2023:05:05:28 +0000] "GET /sw.js HTTP/1.1" 200 32160 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:05:30172.18.0.1 - - [30/Aug/2023:05:05:30 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:05:302023/08/30 05:05:30 [error] 26#26: *4 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:05:30=> Healtheck error got response status 502 Aug 30 06:05:36172.18.0.1 - - [30/Aug/2023:05:05:36 +0000] "HEAD / HTTP/1.1" 502 0 "https://community.landingstrip.social" "Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)" Aug 30 06:05:362023/08/30 05:05:36 [error] 26#26: *6 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "HEAD / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social", referrer: "https://community.landingstrip.social" Aug 30 06:05:40172.18.0.1 - - [30/Aug/2023:05:05:40 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:05:402023/08/30 05:05:40 [error] 26#26: *8 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:05:40=> Healtheck error got response status 502 Aug 30 06:05:40[16] Puma starting in cluster mode... Aug 30 06:05:40[16] * Puma version: 5.6.5 (ruby 3.0.4-p208) ("Birdie's Version") Aug 30 06:05:40[16] * Min threads: 5 Aug 30 06:05:40[16] * Max threads: 5 Aug 30 06:05:40[16] * Environment: production Aug 30 06:05:40[16] * Master PID: 16 Aug 30 06:05:40[16] * Workers: 2 Aug 30 06:05:40[16] * Restarts: (✔) hot (✖) phased Aug 30 06:05:40[16] * Preloading application Aug 30 06:05:41WARN Starting streaming API server master with 1 workers Aug 30 06:05:49172.18.0.1 - - [30/Aug/2023:05:05:49 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:05:492023/08/30 05:05:49 [error] 26#26: *10 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:05:50=> Healtheck error got response status 502 Aug 30 06:05:50172.18.0.1 - - [30/Aug/2023:05:05:50 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:05:502023/08/30 05:05:50 [error] 26#26: *12 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:05:50172.18.0.1 - - [30/Aug/2023:05:05:50 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:05:50172.18.0.1 - - [30/Aug/2023:05:05:50 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:05:502023/08/30 05:05:50 [error] 26#26: *15 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:05:52172.18.0.1 - - [30/Aug/2023:05:05:52 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:00=> Healtheck error got response status 502 Aug 30 06:06:00172.18.0.1 - - [30/Aug/2023:05:06:00 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:06:002023/08/30 05:06:00 [error] 26#26: *18 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:06:01WARN Starting worker 1 Aug 30 06:06:01172.18.0.1 - - [30/Aug/2023:05:06:01 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:012023/08/30 05:06:01 [error] 26#26: *20 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:06:01WARN Worker 1 now listening on 127.0.0.1:4000 Aug 30 06:06:02172.18.0.1 - - [30/Aug/2023:05:06:02 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:04172.18.0.1 - - [30/Aug/2023:05:06:04 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:042023/08/30 05:06:04 [error] 26#26: *23 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:06:06172.18.0.1 - - [30/Aug/2023:05:06:05 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:10=> Healtheck error got response status 502 Aug 30 06:06:10172.18.0.1 - - [30/Aug/2023:05:06:10 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:06:102023/08/30 05:06:10 [error] 26#26: *26 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:06:13warning: parser/current is loading parser/ruby30, which recognizes 3.0.5-compliant syntax, but you are running 3.0.4. Aug 30 06:06:13Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri. Aug 30 06:06:14172.18.0.1 - - [30/Aug/2023:05:06:14 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:142023/08/30 05:06:14 [error] 26#26: *28 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:06:15172.18.0.1 - - [30/Aug/2023:05:06:15 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:202023/08/30 05:06:20 [error] 26#26: *31 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:06:20172.18.0.1 - - [30/Aug/2023:05:06:20 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:06:20=> Healtheck error got response status 502 Aug 30 06:06:21warning: parser/current is loading parser/ruby30, which recognizes 3.0.5-compliant syntax, but you are running 3.0.4. Aug 30 06:06:21Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri. Aug 30 06:06:30172.18.0.1 - - [30/Aug/2023:05:06:30 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:06:302023/08/30 05:06:30 [error] 26#26: *33 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:06:30=> Healtheck error got response status 502 Aug 30 06:06:36172.18.0.1 - - [30/Aug/2023:05:06:36 +0000] "HEAD / HTTP/1.1" 502 0 "https://community.landingstrip.social" "Mozilla/5.0+(compatible; UptimeRobot/2.0; http://www.uptimerobot.com/)" Aug 30 06:06:362023/08/30 05:06:36 [error] 26#26: *35 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "HEAD / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social", referrer: "https://community.landingstrip.social" Aug 30 06:06:362023-08-30T05:06:36.765Z pid=17 tid=69h WARN: `config.options[:key] = value` is deprecated, use `config[:key] = value`: ["/app/code/lib/mastodon/redis_config.rb:38:in `<top (required)>'", "/app/code/config/application.rb:55:in `require_relative'"] Aug 30 06:06:392023-08-30T05:06:39.520Z pid=17 tid=69h INFO: Booting Sidekiq 6.5.8 with Sidekiq::RedisConnection::RedisAdapter options {:driver=>:hiredis, :url=>"redis://:REDACTED@redis-c5a20c30-0d28-420e-8d7c-6ec465053032:6379/0", :namespace=>nil} Aug 30 06:06:40172.18.0.1 - - [30/Aug/2023:05:06:40 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:06:402023/08/30 05:06:40 [error] 26#26: *37 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:06:40=> Healtheck error got response status 502 Aug 30 06:06:45172.18.0.1 - - [30/Aug/2023:05:06:45 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:47172.18.0.1 - - [30/Aug/2023:05:06:47 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:472023/08/30 05:06:47 [error] 26#26: *40 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:06:49172.18.0.1 - - [30/Aug/2023:05:06:49 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:49172.18.0.1 - - [30/Aug/2023:05:06:49 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:492023/08/30 05:06:49 [error] 26#26: *43 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:06:50172.18.0.1 - - [30/Aug/2023:05:06:50 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:06:502023/08/30 05:06:50 [error] 26#26: *45 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:06:50=> Healtheck error got response status 502 Aug 30 06:06:50172.18.0.1 - - [30/Aug/2023:05:06:50 +0000] "GET / HTTP/1.1" 502 894 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:502023/08/30 05:06:50 [error] 26#26: *47 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:3000/", host: "community.landingstrip.social" Aug 30 06:06:50172.18.0.1 - - [30/Aug/2023:05:06:50 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:06:52172.18.0.1 - - [30/Aug/2023:05:06:52 +0000] "GET /sw.js HTTP/1.1" 304 0 "https://community.landingstrip.social/sw.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36" Aug 30 06:07:002023/08/30 05:07:00 [error] 26#26: *51 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social" Aug 30 06:07:00172.18.0.1 - - [30/Aug/2023:05:07:00 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:07:00=> Healtheck error got response status 502 Aug 30 06:07:10=> Healtheck error got response status 502 Aug 30 06:07:10172.18.0.1 - - [30/Aug/2023:05:07:10 +0000] "GET /about HTTP/1.1" 502 894 "-" "Mozilla (CloudronHealth)" Aug 30 06:07:102023/08/30 05:07:10 [error] 26#26: *53 connect() failed (111: Unknown error) while connecting to upstream, client: 172.18.0.1, server: , request: "GET /about HTTP/1.1", upstream: "http://127.0.0.1:3000/about", host: "community.landingstrip.social"
This goes on for different times with each app.
-
BTW - Update to 7.5.2 was flawless
-
@jdaviescoates said in Cloudron update to 7.5.1 - Success!:
@mhgcic said in Cloudron update to 7.5.1 - Success!:
I could increase the RAM to see if that helps.
Regardless if it is what's causing the issues you're currently facing, I'd definitely get more RAM if I were you.
Each instance of Mastodon needs at least 1.5GB and each instance of Discourse at least 2GB. Already that is at least 10.5GB RAM you need before you've included all your other apps plus Cloudron and Ubuntu themselves.
And these are bare minimums. If you have users actually actively using these apps they'll need more.
We know this and monitor daily, and we have software that lets us know that RAM needs to be increased whether it is on a server level or on an app level.
We upgrade as and when needed, We also use throttling so if extra RAM or CPU is needed it is given until we decide on its upgrade needs.
After testing it had nothing to do with CPU or RAM it seems to be networking or a service possibly.