Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    New set of alerts in my log after getting red pop alerts covering my window

    Support
    graphs warning redis
    4
    15
    437
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • scooke
      scooke last edited by girish

      I reckon these two errors are connected to the red alerts I've suddenly been getting. This, plus the fact that I also keep getting warnings about the cloudron reaching it's resource limit (I have 24GB ram but none of the memory graphs show any total getting even near 5GB), might have something to do with the red warnings.

      Jun 08 09:15:04 15:M 08 Jun 07:15:04.776 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
      Jun 08 09:15:04 15:M 08 Jun 07:15:04.777 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
      

      Should I do the two suggested fixes here?

      A life lived in fear is a life half-lived

      1 Reply Last reply Reply Quote 0
      • girish
        girish Staff last edited by

        @scooke Those are just warnings from redis, you can ignore them. What are the red alerts you are getting? What does it say? Finally, are the resource limit alerts the warning that you see when installing a new app?

        1 Reply Last reply Reply Quote 0
        • scooke
          scooke last edited by

          Hi,
          Thank you for replying. That is good to know what those lines mean, but if they are connected to the red alerts can I do anything about it? These red alert pop-ups can cover my entire screen at times and I have to click each one to make them disappear. Screenshot 2020-06-09 10.10.28.png

          The installation warning just tells me that I need a server with more memory, but like I said, none of the graphs ever show my apps going above 2GB, most between 250 and 500 MB... I have a feeling the red alerts are related to this.Screenshot 2020-06-09 10.13.32.png

          A life lived in fear is a life half-lived

          mehdi 1 Reply Last reply Reply Quote 0
          • mehdi
            mehdi App Dev @scooke last edited by

            @scooke How much memory does your server have?

            1 Reply Last reply Reply Quote 0
            • scooke
              scooke last edited by

              a nice 24 GB ram, with 240GB ssd.

              A life lived in fear is a life half-lived

              mehdi 1 Reply Last reply Reply Quote 0
              • mehdi
                mehdi App Dev @scooke last edited by

                @scooke You can see total memory usage in the "System" page, under the "System Memory" tab. How high is it ? Because yeah 24GB should be good for quite a few apps. How many have you got ?

                scooke 1 Reply Last reply Reply Quote 0
                • scooke
                  scooke @mehdi last edited by scooke

                  @mehdi Hovering at just under 4GB.
                  Screenshot 2020-06-09 10.48.32.png So It appears I have about 20GB left for the apps. My VPS dashboard tells me I'm using 8GB out of the 24GB, which tallies with the 4GB being used and the seeming total of all the other apps.

                  I have 34 apps installed! A few big ones would be NC, Pixel, Matrix and Riot, 3 Ghost apps and 10 WP, and Emby. But like I said, none of them ever go above 2GB at any time (I have checked by starting a process, like importing songs to Emby or Updating themes in WP, etc., and watching the memory dashboard and there just isn't ever any noticeable increase.

                  The total ram allocated for all these apps equals about 46GB (one app has 5888MB assigned and two have 4096MB assigned, 10 have 2048MB assigned, as examples - an earlier discussion I had either on Support or on Chat suggested that going over the physical RAM of the machine wouldn't automatically mean is affects overall performance in light of the fact that few of these ever reach their max allotted.

                  One WP site with 4096MB allotted, as an example, apparently hasn't used more than 36MB over the last 7 days, according to its graph. I suppose I could lower this one, and get rid of the Emby which has the 5888 allotted...

                  A life lived in fear is a life half-lived

                  1 Reply Last reply Reply Quote 0
                  • girish
                    girish Staff last edited by

                    @scooke The warning appears when you just add up allocated RAM for all apps and see if it's sort of near the system RAM. It's just a frontend/UI check, the backend will happily install how many ever apps you want as long as you think the system can handle it. So, you can ignore that warning if you think there is enough memory available.

                    scooke 1 Reply Last reply Reply Quote 0
                    • scooke
                      scooke @girish last edited by

                      @girish Hi again, apologies for not posting more detail about the red pop ups. They have continued. I logged on to my cloudron this morning and my screen was covered with them, and I had to manually click on each to remove them. I found what appears to be a connected error in the logs. I'll share both a screen shot of all the red alerts, and the logs, here. If it is a matter of increasing ram for a redis node, that is fine but I have no idea which redis! Clicking 30 times is not the end of the world, but it is a little bothersome after awhile.
                      If these red pop-ups are simple warnings of the added-up allocated ram, is there a way to change the warning? The associated log entry doesn't read to me like it has been adding memory up; it looks like some connection is failing. Thanks for the help.

                      Jun 11 10:32:16 ERROR ServiceUnavailableError: Response timeout
                      at IncomingMessage.<anonymous> (/home/yellowtent/box/node_modules/connect-timeout/index.js:84:8)
                      at IncomingMessage.emit (events.js:198:13)
                      at Timeout.<anonymous> (/home/yellowtent/box/node_modules/connect-timeout/index.js:49:11)
                      at ontimeout (timers.js:436:11)
                      at tryOnTimeout (timers.js:300:5)
                      at listOnTimeout (timers.js:263:5)
                      at Timer.processTimers (timers.js:223:10) [ /home/yellowtent/box/node_modules/express/lib/application.js:630:43 ]
                      

                      Screenshot 2020-06-11 10.30.57.png

                      A life lived in fear is a life half-lived

                      1 Reply Last reply Reply Quote 0
                      • nebulon
                        nebulon Staff last edited by

                        This looks like the main Cloudron process is not responding in time. This either is an issue that the server itself is too busy. Can you open up the webinspector in the browser when that happens to see, which REST api call fails as well as do you see further hints in the logs?
                        less /home/yellowtent/platformdata/logs/box.log in this case

                        scooke 1 Reply Last reply Reply Quote 0
                        • scooke
                          scooke @nebulon last edited by

                          @nebulon said in New set of alerts in my log after getting red pop alerts covering my window:

                          At the moment there are lines after lines after lines of:
                          Jun 11 15:03:43 box:ldap user search: dn ou=users, dc=cloudron, scope sub, filter (&(|(objectclass=user))(displayname=*)(|(displayname=*)(mail=*))) (from 172.18.0.47:34340)
                          I don't recall seeing this repeated like this before.

                          A life lived in fear is a life half-lived

                          mehdi 1 Reply Last reply Reply Quote 0
                          • nebulon
                            nebulon Staff last edited by

                            This is shown everytime an app tries to authenticate or search for users via LDAP. Unless these are there in the 100s within short times like minutes, I don't think that is related.

                            1 Reply Last reply Reply Quote 0
                            • mehdi
                              mehdi App Dev @scooke last edited by

                              @scooke If there are really a lot of these lines in a very short time, maybe one of your apps is under DDoS attack ? Or someone is trying to brute-force a password ?

                              scooke 1 Reply Last reply Reply Quote 0
                              • nebulon
                                nebulon Staff last edited by

                                @scooke not sure if there is a better way currently, but you could run docker ps | grep 34340 on the server to find out which app is issuing those LDAP searches and then maybe temporarily stop that app to see if the timeout issue is related.

                                1 Reply Last reply Reply Quote 0
                                • scooke
                                  scooke @mehdi last edited by

                                  @nebulon about 15-20 every second. When it stops there is a line box:apphealthmonitor app health: 28 alive / 4 dead. Nothing about accessing any sites though.

                                  A life lived in fear is a life half-lived

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post
                                  Powered by NodeBB