Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


    Cloudron Forum

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular

    Solved "Platform data" and "Other" disk usage

    Support
    cloudron disk usage logs
    9
    24
    845
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • yusf
      yusf last edited by girish

      With the new disk usage visualization I can see that "Platform data" takes up a whopping 19GB on a Cloudron instance of mine. "Other" is also hogging. Is that reasonable?

      54328e9e-a4aa-476a-923d-49d007047965-bild.png

      1 Reply Last reply Reply Quote 0
      • nebulon
        nebulon Staff last edited by

        This does look a bit excessive. Can you dig a bit further there and see which part of platform data consumes much?
        You can see that with du -sh /home/yellowtent/platformdata/* via SSH.

        1 Reply Last reply Reply Quote 0
        • yusf
          yusf last edited by

          I got helped out by gramakri. It was some 19GB log file taking up space. I will know where to look next time.

          Are there other optimizations possible? For instance, outdated docker images?

          1 Reply Last reply Reply Quote 0
          • ?
            A Former User last edited by

            @yusf: Can you post your solution? Where is the log file located and what did you do with it (just delete it)? What about the "Other"?

            1 Reply Last reply Reply Quote 0
            • d19dotca
              d19dotca last edited by

              I just noticed mine at 10.24 GB for "Other". What is that and how do we clear it up (if it's even safe to do so)?

              Side note: Would be great if the feature graph would also do the math to show how much remaining space exists so we don't have to do the math ourselves. 😉 I think I'll file that feature request separately later on.

              --
              Dustin Dauncey
              www.d19.ca

              1 Reply Last reply Reply Quote 0
              • girish
                girish Staff last edited by

                @d19dotca Please check the size of /home/yellowtent/platformdata/logs. We have a logrotate issue which is not cleaning up logs properly. You can delete some of the old files there safely.

                yusf 1 Reply Last reply Reply Quote 0
                • W
                  Warez last edited by

                  I have the same problem.

                  I identified:

                  47G /var/lib/docker

                  30G /var/backups

                  Backups i can understand but Docker folder is kind of big.

                  Screenshot from 2019-11-06 21.57.25.png

                  Any ideea?

                  du -sh /home/yellowtent/platformdata/*

                  4.0K /home/yellowtent/platformdata/INFRA_VERSION
                  4.0K /home/yellowtent/platformdata/VERSION
                  4.0K /home/yellowtent/platformdata/acme
                  28K /home/yellowtent/platformdata/addons
                  12K /home/yellowtent/platformdata/backup
                  24K /home/yellowtent/platformdata/collectd
                  67M /home/yellowtent/platformdata/graphite
                  12K /home/yellowtent/platformdata/logrotate.d
                  344K /home/yellowtent/platformdata/logs
                  511M /home/yellowtent/platformdata/mongodb
                  132M /home/yellowtent/platformdata/mysql
                  60K /home/yellowtent/platformdata/nginx
                  77M /home/yellowtent/platformdata/postgresql
                  1.9M /home/yellowtent/platformdata/redis
                  4.0K /home/yellowtent/platformdata/sessions
                  108K /home/yellowtent/platformdata/update

                  1 Reply Last reply Reply Quote 0
                  • girish
                    girish Staff last edited by

                    @Warez Did you use du -hcs to get docker storage size? Because the du tool does not work correctly with the overlayfs that docker uses. Instead use docker system df output.

                    1 Reply Last reply Reply Quote 0
                    • W
                      Warez last edited by

                      TYPE TOTAL ACTIVE SIZE RECLAIMABLE
                      Images 10 9 2.591GB 1.33GB (51%)
                      Containers 10 9 0B 0B
                      Local Volumes 22 18 21.79MB 31.76kB (0%)
                      Build Cache 0 0 0B 0B

                      1 Reply Last reply Reply Quote 0
                      • girish
                        girish Staff last edited by

                        @Warez The 2.5GB there seems OK. Our docker app images are not disk usage optimized.

                        1 Reply Last reply Reply Quote 0
                        • W
                          Warez last edited by

                          I understand, but why does cloudron report Other 82.82 GB?

                          1 Reply Last reply Reply Quote 0
                          • nebulon
                            nebulon Staff last edited by

                            "Other" might not be the best term in hindsight. Basically it is just the sum of everything else on the system. For example the Linux system/distro data itself, as well as data which is otherwise placed outside of Cloudron known folders, if any.

                            If anyone has a better term for that, to convey this information, I am happy to adjust it, if we find a good one.

                            M 1 Reply Last reply Reply Quote 0
                            • M
                              msbt App Dev @nebulon last edited by

                              @nebulon what about just "System" 😉

                              1 Reply Last reply Reply Quote 1
                              • J
                                jk last edited by

                                Hi,

                                I have another question regardiing this. The following is the output of docker system df.

                                $ docker system df
                                TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
                                Images              34                  16                  9.857GB             6.508GB (66%)
                                Containers          21                  18                  0B                  0B
                                Local Volumes       160                 36                  5.274GB             3.964GB (75%)
                                Build Cache         0                   0                   0B                  0B
                                

                                As you can see, docker claims that a lot of storage is reclaimable.

                                Is it safe to run docker volume prune and docker image prune?

                                1 Reply Last reply Reply Quote 2
                                • girish
                                  girish Staff last edited by

                                  The prune commands should be safe to run. Unless there is some Cloudron bug, it shouldn't remove anything.

                                  J 1 Reply Last reply Reply Quote 0
                                  • J
                                    jk @girish last edited by

                                    @girish thank you.

                                    That worked for docker volume prune. The other command (docker image prune) did not actually clean anything up though, but the output of docker system df stays with 66% reclaimable.

                                    Any further ideas?

                                    1 Reply Last reply Reply Quote 0
                                    • yusf
                                      yusf @girish last edited by

                                      @girish said in "Platform data" and "Other" disk usage:

                                      We have a logrotate issue which is not cleaning up logs properly.

                                      I still need to continuously watch these log files not filling up the machine completely. Can you take another look at the log rotation feature?

                                      1 Reply Last reply Reply Quote 1
                                      • girish
                                        girish Staff last edited by

                                        @yusf Do you know which log is taking up space? Maybe we miss a logrotation rule somewhere. You can find this by du -hcs /home/yellowtent/platformdata/logs/*

                                        d19dotca 1 Reply Last reply Reply Quote 0
                                        • d19dotca
                                          d19dotca @girish last edited by d19dotca

                                          @girish For what it's worth, I ran that command and found the total size was 872 MB. Is that expected? It seems like a lot to me to be nearly a GB in size just for text files. I'd expect that to be more around the 200 MB mark or so, but I'm not sure of the logrotate rules and how much gets logged generally nor how quickly.

                                          I noticed one of the larger log files was around 47 MB on its own, and it looks to be from my Radicale app. It seems to be logging in DEBUG mode, which I presume is not intended? I'm guessing for production systems we'd like to keep logs in a normal INFO mode, right?

                                          --
                                          Dustin Dauncey
                                          www.d19.ca

                                          1 Reply Last reply Reply Quote 0
                                          • girish
                                            girish Staff last edited by

                                            mm, OK. I looked into your own installation - https://paste.cloudron.io/eqixarerox.css (~800MB)

                                            logrotate configs are in /home/yellowtent/platformdata/logrotate.d

                                            • Some apps like 2f004a04-e256-426b-9485-e06a31adc2dd were missing configs. This is because when we added the new configs many releases ago, we didn't go back and re-generate it for all the apps. To fix this, simply enable and disable recovery mode and then the logrotate config will be generated.

                                            • For turn, the logrotate was not added when we added it in the previous release. I have fixed this now for next release.

                                            • For all apps/services, we keep up to 2 logs of 10M. This is why most of them are < 20M. So, this is mostly working as expected.

                                            • For the rest, I see that log files are > 20M. I wonder why... investigating.

                                            1 Reply Last reply Reply Quote 0
                                            • girish
                                              girish Staff last edited by girish

                                              Found out why. If a log file is > 20M at rotation time, then it's never truncated. Looks like this is just how logrotate works.

                                              1 Reply Last reply Reply Quote 0
                                              • necrevistonnezr
                                                necrevistonnezr last edited by

                                                https://paste.cloudron.io/upozogutut.sql
                                                For me it adds up to 699 MB

                                                Is there a way to clean up those logs in between?

                                                1 Reply Last reply Reply Quote 0
                                                • girish
                                                  girish Staff last edited by

                                                  @necrevistonnezr If you go into /home/yellowtent/platformdata/logrotate, do you see 07e4e251-ed90-42ca-9ba3-e015b108f1e6 and c5c137fa-0221-4d42-b418-10e544cebad1 there? If not, go those apps (you can figure this out either by looking at the logs a bit or in the dashboard's URL) and start recovery mode and end recovery mode. This will generate the logrotate configs for them.

                                                  Once, the configs are in place, future logs will get rotated. But this does not help the current logs. You can just truncate them like:

                                                  truncate -s0 /home/yellowtent/platformdata/logs/07e4e251-ed90-42ca-9ba3-e015b108f1e6/app.log
                                                  
                                                  necrevistonnezr 1 Reply Last reply Reply Quote 0
                                                  • necrevistonnezr
                                                    necrevistonnezr @girish last edited by

                                                    @girish Thanks, worked!

                                                    1 Reply Last reply Reply Quote 0
                                                    • First post
                                                      Last post
                                                    Powered by NodeBB