Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Offical apps | Community apps | Demo | Docs | Install
  1. Cloudron Forum
  2. Support
  3. MySQL binlogs growing unbounded — 145 GB, no expiry set?

MySQL binlogs growing unbounded — 145 GB, no expiry set?

Scheduled Pinned Locked Moved Solved Support
disk usageubuntu 26.04ubuntumysqldist upgrade
32 Posts 5 Posters 514 Views 5 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J
    J
    jayonrails
    translator
    wrote last edited by
    #1

    Note: analysis for this report has been done with support of ai (claude).

    Environment

    • Cloudron on Ubuntu 26.04., root partition 295 GB
    • MySQL addon: registry.docker.com/cloudron/mysql:3.8.0
    • Affected app: FreeScout
      • Image: cloudron/net.freescout.cloudronapp:202609190520200000
      • App ID: 0a2837d5-…
      • DB: fc0ce74f…

    Problem

    The root partition went from roughly 50% to 89% within a few hours.
    The cause is MySQL binlogs inside the addon container:

    /var/lib/mysql 146 GB total
    binlog.* 145 GB (1486 files, 101 MB each)

    The actual databases only account for about 1 GB. binlog.000001
    dates from 2026-09-01, so every binlog since initial setup is still
    being retained with nothing expiring. binlog_expire_logs_seconds
    appears not to be set in the Cloudron MySQL configuration.

    Write rate

    A new 101 MB binlog is currently created roughly every 9 minutes,
    i.e. about 4 GB/hour (timestamps 16:43 → 22:37 = 40 files).

    The source is FreeScout: its activity_log table has grown to 12 GB
    (activity_logs.ibd), and the nightly mysqldump for that same app
    is now 10 GB.

    Questions

    1. Is it intended that the MySQL addon ships without a binlog expiry?
      On installations without replication, the logs will inevitably
      fill the disk.
    2. What is the supported way to set binlog_expire_logs_seconds
      persistently, so it survives container restarts and app updates?
    3. Are the binlogs required by Cloudron's own backup mechanism, or
      can they be purged safely?
    4. Is the excessive logging in FreeScout's activity_log known
      behaviour for the package, and is there any housekeeping for it?

    I would expect it to be handeled automatically, like it has been for the most previous years. I assume it is a bug in the platform rather than behaivour that is expected to happen.

    I am not sure if this could be a bug when upgrading from Ubuntu 24.04 to Ubuntu 26.04. or because of a Cloudron change?

    I have activated the support login on this server, so staff can check it on their own, if they prefer.

    What has been ruled out

    Docker itself is unremarkable: container logs max 21 MB, build cache
    0 B, volumes in the MB range. Journal 93 MB, /var/log 341 MB. I don't have any other idea where this could originate from.

    1 Reply Last reply
    1
    • jamesJ
      jamesJ
      james
      Staff
      wrote last edited by james
      #25

      Hello @jayonrails and @crbear
      I have merged the two topics into this one.


      I have reproduced the issue, and it comes from upgrading Ubuntu 24 to Ubuntu 26.
      A documentation update is deployed as of writing this.

      The issue is that Ubuntu upgrade does uninstall the MySQL server on the system and both of you did correctly follow the guide and run:

      systemctl stop box
      apt install mysql-server
      systemctl start box
      

      This installed the MySQL server with the apt package default configuration and not the Cloudron configurations.
      To fix this please run the following command from your ssh root/sudo session:

      bash /home/yellowtent/box/setup/start.sh
      

      This will ensure all services on the root system including MySQL are configured correctly.


      Thank you again for bringing this to our attention.

      1 Reply Last reply
      2
      • jamesJ
        jamesJ
        james
        Staff
        wrote last edited by
        #2

        Hello @jayonrails

        @jayonrails said:

        Is it intended that the MySQL addon ships without a binlog expiry?
        On installations without replication, the logs will inevitably
        fill the disk.

        The Cloudron MySQL Service is shipped with skip-log-bin and I just checked there are no bin logs stored for multiple Cloudron servers.

        @jayonrails said:

        What is the supported way to set binlog_expire_logs_seconds
        persistently, so it survives container restarts and app updates?

        Since skip-log-bin is the default I am currently unsure where these are coming from.
        I would suspect Freescout to be the culprit here.
        The default configuration for Freescout and the activity_log is:

        # https://github.com/freescout-help-desk/freescout/blob/88734eec10a898682a8dc0e846459f477d60600a/config/activitylog.php#L8
        'enabled' => env('ACTIVITY_LOGGER_ENABLED', true),
        
        # https://github.com/freescout-help-desk/freescout/blob/88734eec10a898682a8dc0e846459f477d60600a/config/activitylog.php#L14
        'delete_records_older_than_days' => env('ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS', 30),
        

        So by default it is enabled and should remove records older than 30 days.
        Did you configure a different value for ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS in your /app/data/env file?

        Also, could it be that there is somthing creating massive ammounts of logs for the activity_log and that is actually spamming the database?
        My thought is something like, if there is something spamming 101 MB every 9 minutes and if only older than 30 days are cleaned up, only after 30 days it would slowly be cleaned up.

        1 Reply Last reply
        0
        • J
          J
          jayonrails
          translator
          wrote last edited by
          #3

          Hello @james
          thanks for your research.

          I did not touch the config file for at least some years, but I cannot guarantee that it has never been touched.

          ####################################################################################################
          ## If you want to use web installer **DO NOT** create `.env` file manually. 
          ## If `.env` file exists in the root of your app, web installer won't run.
          ##
          ## Every time you are making changes in .env file, in order changes to take an effect you need to run:
          ## php artisan freescout:clear-cache
          #####################################################################################################
          
          # Application URL
          APP_URL=https://wont-tell.de
          
          # Use HTTPS protocol and redirect to HTTPS
          #APP_FORCE_HTTPS=true
          
          # If you are using HTTPS, feel free to uncomment this line to improve security
          #SESSION_SECURE_COOKIE=true
          
          # Enter your proxy address here if freescout.net is not available from your server
          # (access to freescout.net is required to obtain official modules)
          #APP_PROXY=
          
          # Custom headers to add to all outgoing emails.
          #APP_CUSTOM_MAIL_HEADERS="IsTransactional:True;X-Custom-Header:value"
          
          # Timezones: https://github.com/freescout-helpdesk/freescout/wiki/PHP-Timezones
          # Comment it to use default timezone from php.ini
          APP_TIMEZONE=Europe/Berlin
          
          DB_CONNECTION=mysql
          DB_HOST=mysql
          DB_PORT=3306
          DB_DATABASE=censored
          DB_USERNAME=censored
          # Maximum password length is 50 characters
          DB_PASSWORD=censored
          
          # Run the following console command to generate the key: php artisan key:generate
          # Otherwise application will show the following error: "Whoops, looks like something went wrong"
          APP_KEY=base64:km/censored
          APP_FORCE_HTTPS = true
          APP_DISABLE_UPDATING = true
          
          # Uncomment to see errors in your browser, don't forget to comment it back when debugging finished
          #APP_DEBUG=true
          
          APP_CUSTOM_NUMBER=false
          
          APP_USER_PERMISSIONS=
          
          APP_EMAIL_CONV_HISTORY=none
          
          APP_EMAIL_USER_HISTORY=full
          
          APP_LOCALE=de
          
          FASTERSEARCH_URL="https://ms-censored.fra.meilisearch.io"
          
          FASTERSEARCH_API_KEY=censored
          
          APP_ALERT_LOGS=
          
          APP_ALERT_LOGS_PERIOD=hour
          
          export FILE_STORAGE=local
          export FILE_STORAGE_UPLOAD_MAX_SIZE=26214400
          
          APP_REMOTE_HOST_WHITE_LIST=mail,172.18.0.0/16,fd00::/8,127.0.1.1,"my.censored.de",172.18.0.1"
          
          

          After adding the changes you suggested, I noticed that it does not match the styling of the other entries. After starting, I got the following error:

          Sep 22 00:27:54 In Loader.php line 273:
          Sep 22 00:27:54                                                                  
          Sep 22 00:27:54                                                                  
          Sep 22 00:27:54   Dotenv values containing spaces must be surrounded by quotes.  
          Sep 22 00:27:54 => Clear cache folders
          Sep 22 00:27:54 => Ensure directories
          Sep 22 00:27:54 => Ensure permissions
          Sep 22 00:27:54 => Run migrations
          Sep 22 00:27:54 => Set configs
          Sep 22 00:27:54 PHP Warning:  Trying to access array offset on null in /app/code/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php on line 159
          Sep 22 00:27:55 2026-09-21T22:27:55Z 
          Sep 22 00:27:55 2026-09-21T22:27:55Z 
          

          I have then changed it:

          
          # https://github.com/freescout-help-desk/freescout/blob/88734eec10a898682a8dc0e846459f477d60600a/config/activitylog.php#L8
          ACTIVITY_LOGGER_ENABLED=true
          
          # https://github.com/freescout-help-desk/freescout/blob/88734eec10a898682a8dc0e846459f477d60600a/config/activitylog.php#L14
          ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS=30
          

          ...and cleared cache.

          root@0a2837d5-d303-4d2b-b9f4-c35199425682:/app/code# php artisan freescout:clear-cache
          The compiled services & packages files have been removed.
          Cache cleared successfully.
          Compiled views cleared!
          Configuration cache cleared!
          Configuration cached successfully!
          Cleared: JS and CSS builds
          Created: /js/builds/vars.js
          

          @james sagte:

          Also, could it be that there is somthing creating massive ammounts of logs for the activity_log and that is actually spamming the database?
          My thought is something like, if there is something spamming 101 MB every 9 minutes and if only older than 30 days are cleaned up, only after 30 days it would slowly be cleaned up.

          I also thought about that. After some research I have found out that there is mailboxes / support boxes within FreeScout that try to connect to an IMAP / SMTP mailbox which no longer exists.

          Bildschirmfoto 2026-09-22 um 00.41.20.png

          Also I have noticed that the meiliseach instance is no longer there. It has been disabled but the respective entries in the config file have not been remove - although the module has been disabled. When I try to uninstall the module, the page reloads, but the module is still there. It does show a confirmation that the module has been deleted.

          Also I have noticed that after making the changes to the config file and cleaning the cache, the authentication with SMTP did not work. I checked credentials and other data and when saving, I got the error notice that the hostname is not allowed.

          Then I took a deeper look into APP_REMOTE_HOST_WHITE_LIST and changed it as follows:

          APP_REMOTE_HOST_WHITE_LIST=mail,172.18.0.0/16,fd00::/8,127.0.1.1,172.17.0.1,172.17.0.0/16,172.18.0.1,fe80::/10,fe80::be24:11ff:fe61:d810,my.example.com
          

          my.example.com being the hostname of my Cloduron.

          After saving, restarting the application and clearing cache, this worked again.

          When I was sure everything was working again, I cleared all the log files in FreeScout that I could find. Then I disabled the app again, because now the most serious question is: how can I remove the data from the mySQL binlogs?

          When that is done and 145 GB more disk space is available, I would to start the application again and check for log files and if the rotation works correctly, also I would like to check if it get's this big again.

          1 Reply Last reply
          2
          • J
            J
            jayonrails
            translator
            wrote last edited by
            #4

            Addition: I would like to know when "ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS" has been implemented. My FreeScout installtion probably didn't have that when I first installed it. There might be other people that could run into this problem; it could have been a bug some years ago?

            It might be clever to re-install it and do an import, however I have not researched yet if an import is possible and if I can keep the ticket IDs the samde and transfer all data including attachments.

            1 Reply Last reply
            1
            • jamesJ
              jamesJ
              james
              Staff
              wrote last edited by
              #5

              Hello @jayonrails

              @jayonrails said:

              After adding the changes you suggested, I noticed that it does not match the styling of the other entries. After starting, I got the following error:

              The code block I have posted with the URLs are source code and not environment variables.
              I should have been more clear, sorry.
              The environment variables names are ACTIVITY_LOGGER_ENABLED and ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS.


              In your /app/data/env you have shared

              @jayonrails said:

              export FILE_STORAGE=local
              export FILE_STORAGE_UPLOAD_MAX_SIZE=26214400

              there are these two entries which look incorrect.
              No other entries in the /app/data/env have the export $SOME_ENVIRONMENT_VARIABLE=$VALUE style.
              Also I did not find anything in the upstream Freescout code that uses these two environment variables.
              You might want to look into that as well.


              @jayonrails said:

              how can I remove the data from the mySQL binlogs?

              Can you please share where exactly you have found these binlogs?
              Are they inside the freescout container or on the host system?
              When on the host system please connect to your server with ssh and share the output of the following command:

              ls -lah /home/yellowtent/platformdata/mysql/
              # and
              du -hd 0 /home/yellowtent/platformdata/mysql
              

              @jayonrails said:

              I would like to know when "ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS" has been implemented.

              We Cloudron have never implemented ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS.
              This is a default from Freescout and has been like that as far as I can tell for 8 years.

              @jayonrails said:

              My FreeScout installtion probably didn't have that when I first installed it.

              With my statement above, no, it has always been like that unless configured otherwise.

              jdaviescoatesJ J 2 Replies Last reply
              0
              • jamesJ james

                Hello @jayonrails

                @jayonrails said:

                After adding the changes you suggested, I noticed that it does not match the styling of the other entries. After starting, I got the following error:

                The code block I have posted with the URLs are source code and not environment variables.
                I should have been more clear, sorry.
                The environment variables names are ACTIVITY_LOGGER_ENABLED and ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS.


                In your /app/data/env you have shared

                @jayonrails said:

                export FILE_STORAGE=local
                export FILE_STORAGE_UPLOAD_MAX_SIZE=26214400

                there are these two entries which look incorrect.
                No other entries in the /app/data/env have the export $SOME_ENVIRONMENT_VARIABLE=$VALUE style.
                Also I did not find anything in the upstream Freescout code that uses these two environment variables.
                You might want to look into that as well.


                @jayonrails said:

                how can I remove the data from the mySQL binlogs?

                Can you please share where exactly you have found these binlogs?
                Are they inside the freescout container or on the host system?
                When on the host system please connect to your server with ssh and share the output of the following command:

                ls -lah /home/yellowtent/platformdata/mysql/
                # and
                du -hd 0 /home/yellowtent/platformdata/mysql
                

                @jayonrails said:

                I would like to know when "ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS" has been implemented.

                We Cloudron have never implemented ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS.
                This is a default from Freescout and has been like that as far as I can tell for 8 years.

                @jayonrails said:

                My FreeScout installtion probably didn't have that when I first installed it.

                With my statement above, no, it has always been like that unless configured otherwise.

                jdaviescoatesJ
                jdaviescoatesJ
                jdaviescoates
                wrote last edited by
                #6

                @james said:

                No other entries in the /app/data/env have the export $SOME_ENVIRONMENT_VARIABLE=$VALUE style.

                If I'm not mistaken, that style used to be how things were routinely done on Cloudron(?), so this is probably just left over from it being an old install? 🤔

                I use Cloudron with Gandi & Hetzner

                J 1 Reply Last reply
                2
                • jamesJ james

                  Hello @jayonrails

                  @jayonrails said:

                  After adding the changes you suggested, I noticed that it does not match the styling of the other entries. After starting, I got the following error:

                  The code block I have posted with the URLs are source code and not environment variables.
                  I should have been more clear, sorry.
                  The environment variables names are ACTIVITY_LOGGER_ENABLED and ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS.


                  In your /app/data/env you have shared

                  @jayonrails said:

                  export FILE_STORAGE=local
                  export FILE_STORAGE_UPLOAD_MAX_SIZE=26214400

                  there are these two entries which look incorrect.
                  No other entries in the /app/data/env have the export $SOME_ENVIRONMENT_VARIABLE=$VALUE style.
                  Also I did not find anything in the upstream Freescout code that uses these two environment variables.
                  You might want to look into that as well.


                  @jayonrails said:

                  how can I remove the data from the mySQL binlogs?

                  Can you please share where exactly you have found these binlogs?
                  Are they inside the freescout container or on the host system?
                  When on the host system please connect to your server with ssh and share the output of the following command:

                  ls -lah /home/yellowtent/platformdata/mysql/
                  # and
                  du -hd 0 /home/yellowtent/platformdata/mysql
                  

                  @jayonrails said:

                  I would like to know when "ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS" has been implemented.

                  We Cloudron have never implemented ACTIVITY_LOGGER_DELETE_RECORDS_OLDER_THAN_DAYS.
                  This is a default from Freescout and has been like that as far as I can tell for 8 years.

                  @jayonrails said:

                  My FreeScout installtion probably didn't have that when I first installed it.

                  With my statement above, no, it has always been like that unless configured otherwise.

                  J
                  J
                  jayonrails
                  translator
                  wrote last edited by
                  #7

                  @james sagte:

                  The code block I have posted with the URLs are source code and not environment variables.
                  I should have been more clear, sorry.

                  no problem at all :))

                  @james sagte:

                  there are these two entries which look incorrect.
                  No other entries in the /app/data/env have the export $SOME_ENVIRONMENT_VARIABLE=$VALUE style.
                  Also I did not find anything in the upstream Freescout code that uses these two environment variables.
                  You might want to look into that as well.

                  I will check this!

                  @james sagte:

                  Can you please share where exactly you have found these binlogs?
                  Are they inside the freescout container or on the host system?

                  root@my:~# # Biggest items inside /var/lib/mysql, one level deep
                  du -xh --max-depth=1 /var/lib/mysql | sort -h | tail -15
                  36K	/var/lib/mysql/mysql
                  116K	/var/lib/mysql/sys
                  804K	/var/lib/mysql/#innodb_temp
                  1.7M	/var/lib/mysql/performance_schema
                  101M	/var/lib/mysql/#innodb_redo
                  213M	/var/lib/mysql/box
                  173G	/var/lib/mysql
                  root@my:~# 
                  root@my:~# # Binlogs vs everything else, side by side
                  du -ch /var/lib/mysql/binlog.* | tail -1
                  du -sh --exclude='binlog.*' /var/lib/mysql
                  173G	total
                  424M	/var/lib/mysql
                  root@my:~# # Oldest and newest, with timestamps
                  ls -lh --time-style=long-iso /var/lib/mysql/binlog.* | head -5
                  ls -lh --time-style=long-iso /var/lib/mysql/binlog.* | tail -5
                  
                  # The index MySQL maintains — one line per binlog it knows about
                  wc -l /var/lib/mysql/binlog.index
                  head -3 /var/lib/mysql/binlog.index
                  -rw-r----- 1 mysql mysql  180 2026-09-01 14:36 /var/lib/mysql/binlog.000001
                  -rw-r----- 1 mysql mysql  180 2026-09-01 14:38 /var/lib/mysql/binlog.000002
                  -rw-r----- 1 mysql mysql  180 2026-09-01 14:38 /var/lib/mysql/binlog.000003
                  -rw-r----- 1 mysql mysql  441 2026-09-11 17:40 /var/lib/mysql/binlog.000004
                  -rw-r----- 1 mysql mysql 101M 2026-09-11 17:49 /var/lib/mysql/binlog.000005
                  -rw-r----- 1 mysql mysql 101M 2026-09-22 17:23 /var/lib/mysql/binlog.001764
                  -rw-r----- 1 mysql mysql 101M 2026-09-22 17:32 /var/lib/mysql/binlog.001765
                  -rw-r----- 1 mysql mysql 101M 2026-09-22 17:41 /var/lib/mysql/binlog.001766
                  -rw-r----- 1 mysql mysql  79M 2026-09-22 17:48 /var/lib/mysql/binlog.001767
                  -rw-r----- 1 mysql mysql  28K 2026-09-22 17:41 /var/lib/mysql/binlog.index
                  1767 /var/lib/mysql/binlog.index
                  ./binlog.000001
                  ./binlog.000002
                  ./binlog.000003
                  root@my:~# docker exec mysql ls -lh /var/lib/mysql/ | head -5
                  docker exec mysql df -h /var/lib/mysql
                  total 295M
                  -rw-r----- 1 mysql mysql 6.0M Sep 22 17:48 #ib_16384_0.dblwr
                  -rw-r----- 1 mysql mysql  14M Sep 21 22:47 #ib_16384_1.dblwr
                  drwxr-x--- 2 mysql mysql 4.0K Sep 22 17:22 #innodb_redo
                  drwxr-x--- 2 mysql mysql 4.0K Sep 11 17:40 #innodb_temp
                  Filesystem      Size  Used Avail Use% Mounted on
                  /dev/vda3       295G  290G  4.7G  99% /var/lib/mysql
                  
                  

                  @james sagte:

                  ls -lah /home/yellowtent/platformdata/mysql/

                  and

                  du -hd 0 /home/yellowtent/platformdata/mysql

                  root@my:~# ls -lah /home/yellowtent/platformdata/mysql/
                  total 295M
                  -rw-r-----  1 102        uuidd      6.0M Sep 22 17:45 '#ib_16384_0.dblwr'
                  -rw-r-----  1 102        uuidd       14M Sep 21 22:47 '#ib_16384_1.dblwr'
                  drwxr-x---  2 102        uuidd      4.0K Sep 22 17:22  '#innodb_redo'
                  drwxr-x---  2 102        uuidd      4.0K Sep 11 17:40  '#innodb_temp'
                  drwxr-xr-x 14 102        uuidd      4.0K Sep 11 17:40  .
                  drwxr-xr-x 23 yellowtent yellowtent 4.0K Sep  1 21:03  ..
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:14  a627b901205146af
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:33  a78a01e8ce480b13
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:32  a79f97d58ed20a71
                  -rw-r-----  1 102        uuidd        56 Sep  1 21:00  auto.cnf
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:31  c894b11c3cecf0d3
                  -rw-------  1 102        uuidd      1.7K Sep  1 21:00  ca-key.pem
                  -rw-r--r--  1 102        uuidd      1.1K Sep  1 21:00  ca.pem
                  -rw-r--r--  1 102        uuidd      1.1K Sep  1 21:00  client-cert.pem
                  -rw-------  1 102        uuidd      1.7K Sep  1 21:00  client-key.pem
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:31  df2322f70b8c510e
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:29  fc0ce74fabc98ba9
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:32  fd74cc63fc19b991
                  -rw-r-----  1 102        uuidd      3.9K Sep 11 17:39  ib_buffer_pool
                  -rw-r-----  1 102        uuidd       12M Sep 22 17:45  ibdata1
                  -rw-r-----  1 102        uuidd       12M Sep 11 17:40  ibtmp1
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:00  mysql
                  -rw-r-----  1 102        uuidd       26M Sep 22 17:44  mysql.ibd
                  -rw-r-----  1 102        uuidd       125 Sep  1 21:00  mysql_upgrade_history
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:00  performance_schema
                  -rw-------  1 102        uuidd      1.7K Sep  1 21:00  private_key.pem
                  -rw-r--r--  1 102        uuidd       452 Sep  1 21:00  public_key.pem
                  -rw-r--r--  1 102        uuidd      1.1K Sep  1 21:00  server-cert.pem
                  -rw-------  1 102        uuidd      1.7K Sep  1 21:00  server-key.pem
                  drwxr-x---  2 102        uuidd      4.0K Sep  1 21:00  sys
                  -rw-r-----  1 102        uuidd      208M Sep 22 17:45  undo_001
                  -rw-r-----  1 102        uuidd       16M Sep 22 17:45  undo_002
                  root@my:~# du -hd 0 /home/yellowtent/platformdata/mysql
                  12G	/home/yellowtent/platformdata/mysql
                  
                  1 Reply Last reply
                  0
                  • jdaviescoatesJ jdaviescoates

                    @james said:

                    No other entries in the /app/data/env have the export $SOME_ENVIRONMENT_VARIABLE=$VALUE style.

                    If I'm not mistaken, that style used to be how things were routinely done on Cloudron(?), so this is probably just left over from it being an old install? 🤔

                    J
                    J
                    jayonrails
                    translator
                    wrote last edited by
                    #8

                    @jdaviescoates sagte:

                    If I'm not mistaken, that style used to be how things were routinely done on Cloudron(?), so this is probably just left over from it being an old install?

                    Yes, the installation is around 3 years old I would guess.

                    1 Reply Last reply
                    1
                    • jamesJ
                      jamesJ
                      james
                      Staff
                      wrote last edited by
                      #9

                      Hello @jayonrails
                      Since you did run the commands I have asked for against /var/lib/mysql and not /home/yellowtent/platformdata/mysql/ I now understand that this is about the Cloudron MySQL server on the root system itself and not the MySQL service.

                      This changes things and I will have to look into it again.

                      1 Reply Last reply
                      1
                      • J
                        J
                        jayonrails
                        translator
                        wrote last edited by
                        #10

                        Perfect. Since I am running out of space now, is there a safe way to do a backup (which I would share with you to investigate) and get rid of the bullshit data so I don't have to move to another server tonight?

                        1 Reply Last reply
                        1
                        • jamesJ
                          jamesJ
                          james
                          Staff
                          wrote last edited by
                          #11

                          Hello @jayonrails

                          Before we do a clean-up, please run the following command and share the output with me.

                          mysql -uroot -ppassword -e "SHOW BINARY LOGS;"
                          
                          1 Reply Last reply
                          0
                          • J
                            J
                            jayonrails
                            translator
                            wrote last edited by
                            #12

                            I cannot attach a textfile nor add the whole lines of output here because this is too big.

                            https://paste.cloudron.io/p/bat-cobra-emu

                            1 Reply Last reply
                            1
                            • jamesJ
                              jamesJ
                              james
                              Staff
                              wrote last edited by
                              #13

                              Hello @jayonrails
                              Thanks for that.

                              For a quick clean-up you can run the following command:

                              mysql -uroot -ppassword -e "RESET BINARY LOGS AND GTIDS;"
                              

                              After that please check again with the following command if they got removed:

                              mysql -uroot -ppassword -e "SHOW BINARY LOGS;"
                              

                              After that we need to understand why your MySQL server even has bin logging enabled.
                              But let's attempt the quick clean-up first.

                              1 Reply Last reply
                              0
                              • J
                                J
                                jayonrails
                                translator
                                wrote last edited by
                                #14

                                Seems fine for now. Would you like to aggree on a meeting so we could analyze this together?

                                root@my:~# mysql -uroot -ppassword -e "RESET BINARY LOGS AND GTIDS;"
                                mysql: [Warning] Using a password on the command line interface can be insecure.
                                root@my:~# mysql -uroot -ppassword -e "SHOW BINARY LOGS;"
                                mysql: [Warning] Using a password on the command line interface can be insecure.
                                +---------------+-----------+-----------+
                                | Log_name      | File_size | Encrypted |
                                +---------------+-----------+-----------+
                                | binlog.000001 |   1895520 | No        |
                                +---------------+-----------+-----------+
                                
                                1 Reply Last reply
                                1
                                • jamesJ
                                  jamesJ
                                  james
                                  Staff
                                  wrote last edited by
                                  #15

                                  Hello @jayonrails

                                  I will have to check some things first, please give me 10-15 minutes.

                                  1 Reply Last reply
                                  1
                                  • jamesJ
                                    jamesJ
                                    james
                                    Staff
                                    wrote last edited by james
                                    #16

                                    Hello @jayonrails

                                    Since 26.02.2021 all Cloudron installation ship the MySQL system server with:

                                    skip-log-bin
                                    

                                    in the /etc/mysql/my.cnf file.

                                    Is your Cloudron installation older than 26.02.2021?

                                    Please check your /etc/mysql/my.cnf file if skip-log-bin is set or if for example:

                                    expire_logs_days = 10
                                    max_binlog_size = 100M
                                    

                                    or

                                    binlog_expire_logs_seconds = 864000
                                    max_binlog_size = 100M
                                    

                                    You can also run the following command to get the live configuration:

                                    mysql -uroot -ppassword -e "SELECT VERSION(), @@log_bin, @@binlog_expire_logs_seconds, @@expire_logs_days, @@max_binlog_size, @@binlog_expire_logs_auto_purge;"
                                    
                                    J 1 Reply Last reply
                                    0
                                    • jamesJ james moved this topic from FreeScout
                                    • jamesJ james

                                      Hello @jayonrails

                                      Since 26.02.2021 all Cloudron installation ship the MySQL system server with:

                                      skip-log-bin
                                      

                                      in the /etc/mysql/my.cnf file.

                                      Is your Cloudron installation older than 26.02.2021?

                                      Please check your /etc/mysql/my.cnf file if skip-log-bin is set or if for example:

                                      expire_logs_days = 10
                                      max_binlog_size = 100M
                                      

                                      or

                                      binlog_expire_logs_seconds = 864000
                                      max_binlog_size = 100M
                                      

                                      You can also run the following command to get the live configuration:

                                      mysql -uroot -ppassword -e "SELECT VERSION(), @@log_bin, @@binlog_expire_logs_seconds, @@expire_logs_days, @@max_binlog_size, @@binlog_expire_logs_auto_purge;"
                                      
                                      J
                                      J
                                      jayonrails
                                      translator
                                      wrote last edited by
                                      #17

                                      hey @james,

                                      thanks for your investigation. Let met get straight to your questions:

                                      @james sagte:

                                      Is your Cloudron installation older than 26.02.2021?

                                      No, never has been.

                                      @james sagte:

                                      /etc/mysql/my.cnf

                                      #
                                      # The MySQL database server configuration file.
                                      #
                                      # You can copy this to one of:
                                      # - "/etc/mysql/my.cnf" to set global options,
                                      # - "~/.my.cnf" to set user-specific options.
                                      # 
                                      # One can use all long options that the program supports.
                                      # Run program with --help to get a list of available options and with
                                      # --print-defaults to see which it would actually understand and use.
                                      #
                                      # For explanations see
                                      # http://dev.mysql.com/doc/mysql/en/server-system-variables.html
                                      
                                      #
                                      # * IMPORTANT: Additional settings that can override those from this file!
                                      #   The files must end with '.cnf', otherwise they'll be ignored.
                                      #
                                      
                                      !includedir /etc/mysql/conf.d/
                                      !includedir /etc/mysql/mysql.conf.d/
                                      

                                      @james sagte:

                                      You can also run the following command to get the live configuration:

                                      root@my:~# mysql -uroot -ppassword -e "SELECT VERSION(), @@log_bin, @@binlog_expire_logs_seconds, @@expire_logs_days, @@max_binlog_size, @@binlog_expire_logs_auto_purge;"
                                      mysql: [Warning] Using a password on the command line interface can be insecure.
                                      ERROR 1193 (HY000) at line 1: Unknown system variable 'expire_logs_days'
                                      
                                      1 Reply Last reply
                                      1
                                      • jamesJ
                                        jamesJ
                                        james
                                        Staff
                                        wrote last edited by
                                        #18

                                        Hello @jayonrails
                                        Your MySQL config looks empty and that is wrong.
                                        Is also assume there is nothing in the folders listed in the config.

                                        Since that command also failed for you please try again without the expire_logs_days:

                                        mysql -uroot -ppassword -e "SELECT VERSION(), @@log_bin, @@binlog_expire_logs_seconds, @@max_binlog_size, @@binlog_expire_logs_auto_purge;"
                                        
                                        1 Reply Last reply
                                        0
                                        • C
                                          C
                                          CRBear
                                          wrote last edited by
                                          #19

                                          Hello,

                                          I seem to have the same or a very similar issue as reported here:

                                          https://forum.cloudron.io/topic/15988/mysql-binlogs-growing-unbounded-145-gb-no-expiry-set/18

                                          My system:

                                          Cloudron: 10.0.5
                                          Ubuntu:   Ubuntu 26.04.1 LTS
                                          Kernel:   Linux 7.0.0-31-generic
                                          

                                          The root filesystem is currently at 93%:

                                          /dev/sda1   150G   134G   11G   93% /
                                          

                                          The unexpected usage is mainly:

                                          /var/lib/mysql   76G
                                          

                                          There are hundreds of binlog.xxxxxx files, mostly about 101 MB each.

                                          MySQL version:

                                          8.4.11-0ubuntu0.26.04.1
                                          

                                          Current MySQL settings:

                                          log_bin                       = 1
                                          binlog_expire_logs_seconds    = 2592000
                                          max_binlog_size               = 104857600
                                          binlog_expire_logs_auto_purge = 1
                                          

                                          SHOW REPLICA STATUS\G returns no replica.

                                          /etc/mysql/my.cnf only contains:

                                          !includedir /etc/mysql/conf.d/
                                          !includedir /etc/mysql/mysql.conf.d/
                                          

                                          There is no skip-log-bin entry.

                                          I have not deleted any binlogs or changed the MySQL configuration yet.

                                          Could you please advise what I should do next?

                                          In particular, is it safe/recommended in this situation to use:

                                          RESET BINARY LOGS AND GTIDS;
                                          

                                          and should skip-log-bin then be restored manually, or is there another Cloudron-supported fix?

                                          The server currently has only about 11 GB free, so I would prefer to resolve this before the root filesystem fills up.

                                          Thanks in advance for your help!

                                          1 Reply Last reply
                                          1
                                          • C CRBear referenced this topic
                                          • nebulonN
                                            nebulonN
                                            nebulon
                                            Staff
                                            wrote last edited by
                                            #20

                                            So just to confirm here, this is about the Ubuntu host system mysql, not the mysql addon service running in docker?

                                            If so how old is that Cloudron? The skip-log-bin was added in Cloudron around 2021 to /etc/mysql/mysql.cnf and a Cloudron update should theoretically ensure that this file gets updated accordingly.

                                            1 Reply Last reply
                                            1

                                            Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                            Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                            With your input, this post could be even better 💗

                                            Register Login
                                            Reply
                                            • Reply as topic
                                            Log in to reply
                                            • Oldest to Newest
                                            • Newest to Oldest
                                            • Most Votes


                                            • Login

                                            • Don't have an account? Register

                                            • Login or register to search.
                                            • First post
                                              Last post
                                            0
                                            • Categories
                                            • Recent
                                            • Tags
                                            • Popular
                                            • Bookmarks
                                            • Search