Cloudron makes it easy to run web apps like WordPress, Nextcloud, GitLab on your server. Find out more or install now.


Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo

Cloudron Forum

Offical apps | Community apps | Demo | Docs | Install
imc67I

Marcel C

@imc67
translator
Unfollow Follow
About
Posts
1.0k
Topics
192
Shares
0
Groups
1
Followers
3
Following
0

Posts

Recent Best Controversial

  • Mail app fails the internet.nl mail test on cipher suites
    imc67I imc67

    The opportunistic-TLS point is fair and I'll concede it — on port 25 a weak cipher beats a plaintext fallback, and internet.nl's model implicitly assumes an enforcing world (DANE, MTA-STS) that doesn't match how inbound mail actually works.

    One correction on the Node side though, because it affects the "we track Node's defaults" reasoning. Read off the running mail container:

    # node -p 'require("tls").DEFAULT_CIPHERS'
    ...:ECDHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA256:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!SRP:!CAMELLIA
    

    That list names 12 suites explicitly and then ends with the HIGH: catch-all, which expands it to 58. Every flagged suite arrives through the catch-all — not one of them is named in Node's own list:

                                     with HIGH:   without
    ECDHE-ECDSA-AES256-CCM8             yes         no
    ECDHE-ECDSA-ARIA256-GCM-SHA384      yes         no
    ECDHE-ECDSA-AES256-SHA              yes         no
    

    So upgrading Node doesn't move this. The list has had the same shape for years, and Node 24 in Cloudron 10 will produce the same test result that Node 22 does today.

    And the compatibility argument, while true, covers a narrower set than the whole list. The long tail of old MTAs lives on the CBC-SHA1 suites. ARIA is a Korean national standard and CCM_8 is a short-tag variant from the IoT world — no MTA in the wild speaks only those. Just noting where the observation stands; the trade-off is yours to make.

    Support email

  • nginx accepts SHA-224 signature algorithms in TLS 1.2
    imc67I imc67

    Fair enough — you're right that it isn't exploitable. The client sends signature_algorithms and the server picks from the intersection, so nothing modern ends up on SHA-224 unless it explicitly asks for it, which is exactly what my test did.

    Where it still bites is compliance rather than security: organisations measured against the NCSC guidelines get it reported on every scan, and internet.nl states they expect to reclassify it from phase out to insufficient in a future update — at which point it stops being a recommendation and becomes a failed subtest. That's the only thing that really changes the calculus, and whether it's worth pre-empting is your call.

    Same algorithm turns up on the mail server for the same reason, which is in the other thread.

    Support nginx

  • Fix postgresql shared_buffers size?
    imc67I imc67

    That makes sense, I hadn't looked at it that way. The page cache gives the memory back and shared_buffers doesn't, which matters a lot more on a box with thirty containers than on a dedicated database server — and the 25% guidance assumes the latter.

    What might still be worth it is effective_cache_size, which doesn't allocate anything and only tells the planner how much cache it can expect. On three of my servers with 8, 8 and 10 GB limits it reads 4GB everywhere, straight from boot_val — it doesn't track the limit at all. Deriving that one from the memory limit would improve plan choices without reserving any RAM.

    No rush on this one, it's much smaller than the MySQL case. It just seemed inconsistent that the buffer pool now follows the memory limit while the planner hint doesn't.

    Support postgresql services

  • Bug: WordPress object-cache redis writes a full RDB dump every five minutes, which turned out to be my biggest disk I/O contributor
    imc67I imc67

    Thanks, that was fast.

    One thing I couldn't tell: do the app packages have to opt into this, and if so, do existing installs pick it up on an app update — or does it only apply to new installs?

    WordPress (Developer) redis wordpress

  • Bug: WordPress object-cache redis writes a full RDB dump every five minutes, which turned out to be my biggest disk I/O contributor
    imc67I imc67

    Following up on my old topic about high disk I/O (14639, which @nebulon split into 15206, 15207 and 15211) : there is one more contributor, and on one of my servers it is bigger than everything we found back then.

    Measured over a quiet hour on a server with 14 WordPress apps: 13.8 GB written in 60 minutes, of which mysql accounts for 4.1 GB and the per-app redis containers for 5.1 GB together. The object caches write more than the database does.

    It comes from the default save policy. An app's redis runs with save 900 1 300 10 60 10000, and a WordPress object cache always changes more than 10 keys within 300 seconds, so it dumps its entire dataset every five minutes. One site here has 323k keys and a 63 MB dump.rdb, which is 12 x 63 MB = 756 MB/hour predicted against 743 MB/hour measured.

    I set CONFIG SET save "" on one of them and left a comparable site untouched as a control. Over ten minutes the first wrote 0.0 MB and the control 125.4 MB. Nothing was lost: same key count, same memory use, site fine. The dataset lives in memory either way, the dump is only a copy on disk.

    To be clear, this isn't an argument for turning redis off on WordPress. Those caches are doing real work: across 22 WordPress sites here they serve about 4.7 million object lookups a day, with hit rates between 45% and 99%. It is the persistence that looks pointless for a cache, and a backup doesn't seem to depend on it either, since the box asks the redis service for a fresh dump at backup time rather than picking up the periodically written file.

    Would it make sense to turn off RDB persistence for app redis instances, or to make the save policy configurable per app?

    WordPress (Developer) redis wordpress

  • Services memory percentage counts page cache, so database services always sit at 99%
    imc67I imc67

    The memory percentage on the Services page counts page cache, so a database service settles at 99% whatever limit you give it. Raise the limit and a few days later it reads 99% again, which makes the number hard to act on.

    Here is the postgresql service on a 4 GB limit, straight from its cgroup:

    anon              112 MB
    file             3915 MB   (inactive_file 2974 MB, reclaimable)
    oom_kill            0
    

    docker stats reports 27% for that same container at that same moment, because it subtracts inactive page cache.

    The tricky part is that 99% looks identical whether the cache is comfortably full or the container is thrashing. Here it happened to be the latter: the cgroup hit its ceiling 154 times per minute and the container read 117 MB/min from disk. After raising the limit those dropped to 0 and 44 kB/min. Nothing in the percentage told us which of the two situations we were in.

    Would it make sense to base the figure on memory.current minus inactive_file, or to show the reclaim and oom counters from memory.events next to it?

    Support postgresql memory

  • Fix postgresql shared_buffers size?
    imc67I imc67

    @girish Following up on this one for PostgreSQL, since you're in that code anyway: shared_buffers there is still at the built-in default of 128 MB, on a service that had a 4 GB limit here.

    shared_buffers         16384 (8kB)  = 128 MB
    effective_cache_size  524288 (8kB)  = 4 GB    (also the built-in default)
    work_mem                4 MB
    

    Postgres leans on the OS page cache far more than InnoDB does, so this is less dramatic than the MySQL case. But the usual guidance is somewhere around 25% of available memory, and 3% is a long way below that. effective_cache_size is a planner hint and is also left at its default, so it does not track the limit either.

    Would the same clamp you just added for MySQL make sense here, with a smaller fraction?

    Support postgresql services

  • Bug: changing an app's Redis memory limit does not restart the container, so maxmemory stays at the old value
    imc67I imc67

    On 9.2.x, changing the Redis memory limit for an app under Services updates the container's cgroup limit right away, but does not restart the Redis container. maxmemory is written when the container starts, so redis keeps running with the value from its previous start.

    That is harmless when you raise the limit. It is not when you lower it. Two days after lowering one app's Redis from 512 MB to 256 MB:

    docker inspect -f '{{.HostConfig.Memory}}' redis-<appid>
    268435456
    
    redis-cli config get maxmemory
    526385152
    

    With maxmemory-policy noeviction, redis will grow to 502 MB before it starts refusing writes, while the cgroup kills it at 256 MB. So instead of an "OOM command not allowed" error you get a killed container.

    What made this hard to notice: the very first time you configure an app's Redis, the service does get rebuilt. Only the changes after that don't. A single test therefore looks like it works. Restarting the service by hand afterwards puts maxmemory right again.

    Would it make sense to restart the service container on a memory limit change, the same way the app container is now rebuilt when its memory limit changes?

    Support redis memory

  • Mail app fails the internet.nl mail test on cipher suites
    imc67I imc67

    The https://en.internet.nl mail test fails the "Secure mail server connection (STARTTLS and DANE)" category on all three of my Cloudron servers — 9.2.0, mail app 3.18.2 — with identical findings on each, which suggests a platform default rather than local configuration.

    Cipher suites offered that the NCSC guidelines (2025-05) rate insufficient:

    TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8
    TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8
    TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
    TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
    

    And rated phase out:

    TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384
    TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256
    TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
    TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
    

    Cipher suite order fails as well — the server prefers ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384 over suites rated good. Lower severity in the same test: SHA-224 for the key exchange hash (separate topic, it applies to nginx too) and client-initiated renegotiation reported as unlimited.

    Everything else passes — IPv6, DNSSEC, DKIM/SPF/DMARC, DANE and RPKI — so the failed category comes down entirely to the cipher list. Each server scores 97%.

    With v10 in progress: worth including?

    Support email

  • nginx accepts SHA-224 signature algorithms in TLS 1.2
    imc67I imc67

    On Cloudron 9.2.0 (nginx 1.24.0, OpenSSL 3.0.13) the generated vhosts pin ssl_protocols and ship a curated ssl_ciphers list, but the TLS 1.2 signature algorithms are left at OpenSSL's defaults, which still include the SHA-224 variants. A client offering nothing but ECDSA+SHA224 gets a completed handshake with Peer signature type: ecdsa_sha224.

    https://en.internet.nl flags this under "Hash function for key exchange" as phase out, following the NCSC TLS guidelines 2025-05 §3.3.5, and notes it will likely be rated insufficient in a future update. It's the only remaining recommendation on an otherwise 100% site, and it sits in the platform's nginx configuration rather than anywhere an app or admin can reach.

    Since v10 is in the works — would this be worth picking up along the way?

    Support nginx

  • Mail: spf=temperror on a large share of inbound mail, embedded DNS drops concurrent queries
    imc67I imc67

    yes, direct url is working again.

    That was it! Applied the one line change on all three servers here and the 60 query test goes from 20/60 to 60/60 on each of them.

    For reference, temperror was running at 48%, 34% and 18% of inbound mail over the last 24 hours on those three. I will report back once enough new mail has come in to see where it lands.

    Thanks for digging into this so quickly.

    Support mail mail server docker dns

  • Mail: spf=temperror on a large share of inbound mail, embedded DNS drops concurrent queries
    imc67I imc67

    @girish don't know what is changed but I can't login to git.cloudron.io anymore (previously a separate account, later I remember it's changed to OIDC?) with/after OIDC it keeps asking for "Enter verification code" but the one for the Cloudron.io-account doesn't work (I can login there). Any idea?

    update: strange, when I click your URL I have to login, if I surf to git. and your commit I can read it ....

    Support mail mail server docker dns

  • Mail: spf=temperror on a large share of inbound mail, embedded DNS drops concurrent queries
    imc67I imc67

    @girish Small update with 24 hour numbers from the three servers, plus a couple of things I ruled out on my own side.

    Over the last 24 hours, the share of delivered mail that got an SPF verdict of temperror was 48%, 34% and 18% on the three servers. Their own 30 day averages are 21%, 31% and 14.5%, so all three are sitting above their baseline at the moment.

    Ruled out locally: conntrack is at roughly 0.5% of the table on all three with no table-full events in dmesg, and UDP InErrors, RcvbufErrors and SndbufErrors are all zero, both on the host and inside the mail container. Nothing is being dropped by the kernel, so it really does look like the embedded resolver just does not answer part of the concurrent queries.

    One thing worth adding for balance, because it changes how urgent this looks. It mostly hits large legitimate senders with heavy SPF records. Of the 24 messages delivered with temperror in those 24 hours, 23 were from the likes of Dutch Government, GitHub, WeTransfer, etc. So the practical effect is not that spam suddenly floods in but the risk of doing so is higher. It is that SPF and DMARC quietly stop being enforced for a sizeable share of mail, including for the senders who do everything right.

    Support mail mail server docker dns

  • Mail: spf=temperror on a large share of inbound mail, embedded DNS drops concurrent queries
    imc67I imc67

    I went looking for why a spoofed sender got delivered to an inbox here, and ended up somewhere I did not expect. On three of my Cloudrons, counting the Authentication-Results of everything delivered in the last 30 days, spf=temperror comes back on 14%, 21% and 31% of the messages that got an SPF verdict at all. For comparison, fail and softfail together never reach three messages per server. So temperror is by far the most common non-pass outcome, not a rare hiccup.

    It is not the senders. The temperror messages are spread over 31 to 34 different sending domains per server, including icloud.com, overheid.nl, apple.com, github.com and proton.me. Those are not domains with broken SPF.

    It is not a broken resolver either, at least not when you test it the obvious way. From inside the mail container, thirty lookups in a row all succeed in 23 to 40 ms, and the whole include chain resolves fine. It only falls apart when queries for different names run at the same time, which is exactly what evaluating an SPF record does:

    # serial, and 30 parallel queries for the same name: fine
    docker exec mail bash -c 'for i in $(seq 1 30); do dig +time=2 +tries=1 +short TXT kvk.nl >/dev/null 2>&1 && echo ok || echo fail; done' | sort | uniq -c
    
    # 60 parallel queries across six different names
    docker exec mail bash -c 'for d in kvk.nl icloud.com overheid.nl apple.com github.com amazon.nl; do for i in $(seq 1 10); do (dig +time=2 +tries=1 +short TXT $d >/dev/null 2>&1 && echo ok || echo fail) & done; done; wait' | sort | uniq -c
    

    On both servers the last one returns 20 ok and 40 fail. resolv.conf in the container points at 127.0.0.11, Docker's embedded DNS.

    Fair warning about that test: I used +tries=1 with a 2 second timeout, so no retries. With retries the loss is smaller. The 14 to 31 percent measured on real delivered mail is the part that matters, and that is independent of how I tested.

    What makes this more than a cosmetic issue is that temperror carries no penalty, and rightly so, since it means "could not be checked" rather than "failed". The effect is that SPF and DMARC are quietly not enforced on a large slice of mail, including mail from senders who publish -all and p=quarantine and do everything right. That is how a spoof of such a domain reached an inbox here at score 2.3. The same resolver is also used for the DNSBL, URIBL, DKIM and rDNS lookups, and a dropped query there just means a rule silently does not fire.

    Would it be possible to give the mail container a real resolver instead of the embedded one, or is there a reason it has to be 127.0.0.11?

    Support mail mail server docker dns

  • MySQL service still runs with the default 128M innodb_buffer_pool_size
    imc67I imc67

    Yes, that covers it. Writing it into the cnf when the container starts is exactly the piece that is missing today, because starting up is the moment the value falls back to 128M.

    It also fits how things already work: /etc/mysql/my.cnf is a symlink into /run, which gets rebuilt on every start anyway, so there is no new mechanism needed for it.

    One question while you are in there: does changing the memory limit of a service actually restart its container? In my eventlog a service.configure with a new memoryLimit isn't always followed by a rebuild or a restart. If it isn't, then someone who raises the limit because things feel slow would not see the buffer pool follow until the next restart, which could be months later. Since innodb_buffer_pool_size is a dynamic variable, setting it live at that moment as well would cover that case.

    Support mysql

  • MySQL service still runs with the default 128M innodb_buffer_pool_size
    imc67I imc67

    That shape looks right to me. Two things from having done it by hand here.

    It needs to be applied on every start of the service, not just once at install. That is the actual problem I ran into: the value drops back to 128M whenever the mysql service restarts, so a one-off calculation would put people back at the default after the next update.

    innodb_buffer_pool_size is a dynamic variable, so you can apply it straight after someone changes the memory limit too, without restarting mysql. I resized three servers live yesterday and Innodb_buffer_pool_resize_status tells you when it has finished. One detail worth knowing: MySQL rounds the value up to a multiple of the chunk size, so the effective figure can differ slightly from what the formula produces.

    Support mysql

  • Mail app: expose munge_subject_threshold so users get a warning in the subject
    imc67I imc67

    Most of the people I host mail for are older volunteers. They never open the Spam folder, and mail scoring somewhere between 2 and 5 lands straight in their inbox, where they do click it. A visible warning in the subject line would help them a lot, because that is the one place every mail client shows, on a phone and in a preview pane alike.

    The Haraka spamassassin plugin already supports exactly this: munge_subject_threshold together with subject_prefix. It is commented out in the plugin's own spamassassin.ini, and the mail app's config does not set it.

    I cannot enable it myself in a way that survives. /app/code/haraka-config is part of the image, and the config Haraka actually runs with is rebuilt in /run/haraka on every start. Sieve is not an alternative here either, since editheader is not in this Pigeonhole build.

    @girish Would you consider exposing those two settings, or simply shipping a sensible default? This one isn't about exposing SpamAssassin internals, it is a plain user-facing feature, and I suspect a lot of admins running mail for non-technical users would get something out of it.

    Feature Requests mail server spamassassin spam

  • MySQL service still runs with the default 128M innodb_buffer_pool_size
    imc67I imc67

    The risk of forgetting and setting this by hand after every reboot is real, so I put a small systemd timer on all three servers. Sharing it in case it's useful to someone else, and to show how little is needed if the value were just set at startup.

    The script only issues a runtime SET GLOBAL, the same thing I was typing by hand. It doesn't touch any Cloudron file:

    #!/bin/bash
    set -euo pipefail
    
    WANT_GB=3
    WANT=$(( WANT_GB * 1024 * 1024 * 1024 ))
    
    CT=$(docker ps --format '{{.Names}}' | grep -ix mysql | head -1)
    [ -z "$CT" ] && exit 0
    
    CUR=$(printf '%s\n' 'SELECT @@innodb_buffer_pool_size;' \
      | docker exec -i "$CT" bash -c 'mysql -uroot -p"$CLOUDRON_MYSQL_ROOT_PASSWORD" -N' 2>/dev/null | tail -1) || exit 0
    case "$CUR" in ''|*[!0-9]*) exit 0 ;; esac
    [ "$CUR" -ge "$WANT" ] && exit 0
    
    printf '%s\n' "SET GLOBAL innodb_buffer_pool_size=$WANT;" \
      | docker exec -i "$CT" bash -c 'mysql -uroot -p"$CLOUDRON_MYSQL_ROOT_PASSWORD"'
    logger -t mysql-bufferpool "buffer pool restored: $CUR -> $WANT"
    

    With a timer on OnBootSec=3min and OnUnitActiveSec=15min. The quarter-hourly check is there because the service also restarts on platform updates, not just on reboot. It does nothing when the value is already fine, and the password stays inside the container.

    WANT_GB is a constant I fill in once per server, at install time, from how much data is actually in there:

    DATA_MB=$(printf '%s\n' 'SET SESSION information_schema_stats_expiry=0;
      SELECT ROUND(SUM(data_length+index_length)/1024/1024) FROM information_schema.tables
       WHERE table_schema NOT IN ("mysql","information_schema","performance_schema","sys");' \
      | docker exec -i mysql bash -c 'mysql -uroot -p"$CLOUDRON_MYSQL_ROOT_PASSWORD" -N' | tail -1)
    
    # data x 1.25, rounded up to whole GB, min 1, capped at half the service limit and at 4
    WANT_GB=$(( (DATA_MB * 125 / 100 + 1023) / 1024 ))
    

    That gives 3G, 3G and 2G on my three servers (1855, 2077 and 1215 MB of data). The script itself never runs this, it only reads the constant. A bug in a runtime calculation could set something absurd and OOM the container, and this value changes maybe once a year.

    For Cloudron it could be much simpler than this, because you don't need to know how much data is in there: a fraction of the service memory limit the admin has already configured would be fine. My formula only looks at the data because that's the part I can measure from outside.

    One correction to my first post: the "366 MB" I quoted for that server was wrong. That was the FreeScout database on its own, not the total. All apps together on that server are 2077 MB, so the default 128M was even further off than I said. If you check this yourself, note that information_schema.tables caches its statistics for 24 hours by default (information_schema_stats_expiry), which is easy to trip over when measuring before and after.

    Still hoping the value can be derived from the service memory limit, so this timer can go away.

    Support mysql

  • MySQL service still runs with the default 128M innodb_buffer_pool_size
    imc67I imc67

    To show this isn't just the one server: on another Cloudron of mine the MySQL service holds 2.4 GB of InnoDB data across all apps, also on the default 128M buffer pool. The four biggest are a monitoring app(LAMP) (689 MB), MainWP (569 MB), Matomo (443 MB) and a geocoding cache (LAMP) (386 MB), so this isn't a "FreeScout only" thing. Memory limit there is 8 GB as well.

    I've now set 3G there and 2G on a third server, sized to how much data is actually in them. All of it goes back to 128M as soon as the service restarts, which is the main reason I'd rather see the value derived from the memory limit than set by hand.

    Support mysql

  • MySQL service still runs with the default 128M innodb_buffer_pool_size
    imc67I imc67

    On one of my servers I gave the MySQL service a 10 GB memory limit, but innodb_buffer_pool_size is still MySQL's own default of 128M. All InnoDB data of all apps on that server together is 366 MB, so nothing stays cached and every query reads from disk again.

    What I measured on FreeScout (38k rows in threads, 332 MB of message bodies): a search took 7.8 seconds. I raised the buffer pool to 1G at runtime and the same search took 1.4 seconds. I've left it at 1G for now, but it won't survive a restart of the service.

    Would it be possible to derive innodb_buffer_pool_size from the memory limit that is already configured for the service? That setting already says how much memory I want it to use. I can't make it stick myself, since /etc/mysql/my.cnf points into /run.

    Support mysql
  • Login

  • Don't have an account? Register

  • Login or register to search.
  • First post
    Last post
0
  • Categories
  • Recent
  • Tags
  • Popular
  • Bookmarks
  • Search