Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved Potential Issue - "exe" and/or netns-create stuck?

    Support
    3
    17
    267
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • doodlemania2
      doodlemania2 App Dev last edited by

      This morning my CR ground to a halt. I thought it was getting VPS bound since I've been "popular" the past few days, but they said all good so I kept digging. In the VM, there's a bin running REALLY hot. In top it shows up as "exe" which made me worry a bit:

      1563 root 20 0 4926308 2.9g 28356 R 102.3 6.2 1:07.44 exe

      So I ps-ax'd the PID and it came back as
      1563 ? Rl 0:26 set-ipv6 /var/run/docker/netns/3939c4953582 all false

      After a bit, that PID goes away and is replaced by another PID just like it. VM is up but CR is down.

      Did a reboot, still persists. Any ideas?

      girish robi 2 Replies Last reply Reply Quote 0
      • girish
        girish Staff @doodlemania2 last edited by

        @doodlemania2 You should look into where that exe binary is coming from, never seen anything like it. I think you can also see /proc/1563/cmdline (from memory)

        doodlemania2 1 Reply Last reply Reply Quote 0
        • doodlemania2
          doodlemania2 App Dev @girish last edited by

          @girish yeah, it's just running really hot this type of command over and over:
          netns-create/var/run/docker/netns/f4e6ea807391

          htop gives a slightly better visual:
          b08ffe49-32e8-4dd2-81f7-f9873eb743bc-image.png

          Something about set-ipv6 maybe? (Not using IPv6 on this CR I don't think)

          1 Reply Last reply Reply Quote 0
          • robi
            robi @doodlemania2 last edited by

            @doodlemania2 how is docker configured? is ipv6 enabled?

            https://docs.docker.com/config/daemon/ipv6/

            If not, perhaps enable it and see if the process calms down the CPU usage.

            Life of Advanced Technology

            doodlemania2 1 Reply Last reply Reply Quote 0
            • doodlemania2
              doodlemania2 App Dev @robi last edited by

              @robi enabled and restarted - still remain stuck - still digging, wondering if docker somehow went wonky. VPS team also looking.

              girish 1 Reply Last reply Reply Quote 0
              • girish
                girish Staff @doodlemania2 last edited by

                @doodlemania2 where is your server hosted ? (don't tell me netcup...)

                doodlemania2 1 Reply Last reply Reply Quote 0
                • doodlemania2
                  doodlemania2 App Dev @girish last edited by

                  @girish haha, nah, ssd'nodes that @robi introduced me to - they're just darn close to amazing!

                  They are still looking but I am starting to wonder if docker is just corrupted somehow.

                  girish 1 Reply Last reply Reply Quote 1
                  • girish
                    girish Staff @doodlemania2 last edited by

                    @doodlemania2 can I take a look at the server? Can you write to support@ ? I want to see what the app is that is making it get stuck. In the other thread, wp developer app was suspected.

                    doodlemania2 1 Reply Last reply Reply Quote 1
                    • doodlemania2
                      doodlemania2 App Dev @girish last edited by

                      @girish sure thing - will enable from terminal!

                      doodlemania2 1 Reply Last reply Reply Quote 1
                      • doodlemania2
                        doodlemania2 App Dev @doodlemania2 last edited by

                        @doodlemania2 @doodlenode:~$ sudo cloudron-support --enable-ssh
                        [sudo] password for derek:
                        Enabling ssh access for the Cloudron support team...Done

                        girish 1 Reply Last reply Reply Quote 1
                        • girish
                          girish Staff @doodlemania2 last edited by

                          @doodlemania2 wow, that server is pretty clogged up. I keep getting kicked out of ssh. But I did see a lot of "exe". Maybe we should disable docker and reboot to check if it helps? systemctl disable docker if you can

                          doodlemania2 1 Reply Last reply Reply Quote 0
                          • doodlemania2
                            doodlemania2 App Dev @girish last edited by

                            @girish sorry for delay, got sidetracked on another thing - docker disabled and restarted!

                            doodlemania2 1 Reply Last reply Reply Quote 0
                            • doodlemania2
                              doodlemania2 App Dev @doodlemania2 last edited by

                              @girish okay - I think I fixed it - the /etc/hosts file was almost 40GB in size of garbage (shrug)
                              I emptied it out and replaced it with just default localhost items and restarted everything. MUCH happier it is.
                              Thank you so much for looking at it with me - it was in a very bad way!

                              Now to see if I can figure out why hosts file got gummed up

                              girish 1 Reply Last reply Reply Quote 1
                              • Topic has been marked as a question  girish girish 
                              • Topic has been marked as solved  girish girish 
                              • girish
                                girish Staff @doodlemania2 last edited by

                                @doodlemania2 that sounds.. worrying. how did the hosts file get corrupt ? how did you figure you should look into that file?

                                doodlemania2 1 Reply Last reply Reply Quote 0
                                • doodlemania2
                                  doodlemania2 App Dev @girish last edited by

                                  @girish well, the thing that kept getting overwhelmed was netns ip6 and i also noticed that my tailscale agent was running at like 4.6GB in memory, so thought, okay, let me ping something. it took a solid 40 seconds to even return a ping but it was correct. so that's when i looked at hosts.

                                  did a cat to /dev/null on it and boom, all back to good. i am thinking that some dns thing went haywire - but I don't think it's CR's fault cause I've never seen CR try to write to hosts. Will dig a bit more but I also have an LTS upgrade due, so perhaps there's a bug someplace in Ubuntu with bind.

                                  robi 1 Reply Last reply Reply Quote 1
                                  • robi
                                    robi @doodlemania2 last edited by

                                    @doodlemania2 would have been good to get a sample/clue of what was in the hosts file with head or tail.

                                    Life of Advanced Technology

                                    doodlemania2 1 Reply Last reply Reply Quote 0
                                    • doodlemania2
                                      doodlemania2 App Dev @robi last edited by

                                      @robi yeah, I tried to cat / tail it, was unicode garbage. me thinks corrupted

                                      1 Reply Last reply Reply Quote 0
                                      • First post
                                        Last post
                                      Powered by NodeBB