Redis ran out of memory

[RazielKanos]

Hey there,

Once a week, if i don't shut the service down and reboot it, i get the problem that redis claims to run out of memory (i have set it to 2 GB)
It's not really an isue since i can silence it for a week or two with restarting the app, just wanted to mention it:

This is how my graphs look:

[murgero]

@RazielKanos If you go into Services, redis, then increase the ram:

[d19dotca]

I'm running into this as well. What's strange is there hasn't been any changes (that I'm aware of at least) to the website. I run ~20 websites but only have one with this issue, and it's an older site which hasn't been updated in quite a while in terms of changes other than plugin updates. It's possibly related but it's intermittent which makes it difficult to narrow down.

I'm not sure what is going on, as I don't have any unusual traffic on the website at all (it only gets maybe 10 hits a day), and I'm not making changes at all to it. I've tried increasing the memory and that tied me over for a bit but that is on repeat again now, and I'm not sure why this one site is using so much redis stuff compared to all the others.

Here's what my redis log looks like on repeat:

2023-06-21T03:57:02.000Z 550:C 21 Jun 2023 03:57:02.190 # Configuration loaded
2023-06-21T03:57:02.000Z 550:C 21 Jun 2023 03:57:02.190 # Redis version=6.0.16, bits=64, commit=00000000, modified=0, pid=550, just started
2023-06-21T03:57:02.000Z 550:C 21 Jun 2023 03:57:02.190 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2023-06-21T03:57:02.000Z 550:M 21 Jun 2023 03:57:02.194 # Server initialized
2023-06-21T03:57:02.000Z 550:M 21 Jun 2023 03:57:02.194 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2023-06-21T03:57:02.000Z 550:M 21 Jun 2023 03:57:02.194 * Running mode=standalone, port=6379.
2023-06-21T03:57:02.000Z 550:M 21 Jun 2023 03:57:02.195 * Loading RDB produced by version 6.0.16
2023-06-21T03:57:02.000Z 550:M 21 Jun 2023 03:57:02.195 * RDB age 2 seconds
2023-06-21T03:57:02.000Z 550:M 21 Jun 2023 03:57:02.195 * RDB memory usage when created 705.47 Mb
2023-06-21T03:57:03.000Z 2023-06-21 03:57:03,198 INFO success: redis entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2023-06-21T03:57:03.000Z 2023-06-21 03:57:03,201 INFO reaped unknown pid 549 (exit status 0)
2023-06-21T03:57:03.000Z 550:M 21 Jun 2023 03:57:03.461 * DB loaded from disk: 1.267 seconds
2023-06-21T03:57:03.000Z 550:M 21 Jun 2023 03:57:03.461 * Ready to accept connections
2023-06-21T04:02:03.000Z 550:M 21 Jun 2023 04:02:03.044 * 10 changes in 300 seconds. Saving...
2023-06-21T04:02:03.000Z 550:M 21 Jun 2023 04:02:03.060 * Background saving started by pid 555
2023-06-21T04:02:07.000Z 550:M 21 Jun 2023 04:02:07.068 * Background saving terminated with success
2023-06-21T04:02:07.000Z 555:C 21 Jun 2023 04:02:07.025 * DB saved on disk
2023-06-21T04:02:07.000Z 555:C 21 Jun 2023 04:02:07.040 * RDB: 0 MB of memory used by copy-on-write
2023-06-21T04:07:08.000Z 550:M 21 Jun 2023 04:07:08.023 * 10 changes in 300 seconds. Saving...
2023-06-21T04:07:08.000Z 550:M 21 Jun 2023 04:07:08.047 * Background saving started by pid 556
2023-06-21T04:07:10.000Z 556:C 21 Jun 2023 04:07:10.932 * DB saved on disk
2023-06-21T04:07:10.000Z 556:C 21 Jun 2023 04:07:10.947 * RDB: 1 MB of memory used by copy-on-write
2023-06-21T04:07:11.000Z 550:M 21 Jun 2023 04:07:11.053 * Background saving terminated with success
2023-06-21T04:12:12.000Z 550:M 21 Jun 2023 04:12:12.043 * 10 changes in 300 seconds. Saving...
2023-06-21T04:12:12.000Z 550:M 21 Jun 2023 04:12:12.059 * Background saving started by pid 557
2023-06-21T04:12:14.000Z 550:M 21 Jun 2023 04:12:14.363 * Background saving terminated with success
2023-06-21T04:12:14.000Z 557:C 21 Jun 2023 04:12:14.277 * DB saved on disk
2023-06-21T04:12:14.000Z 557:C 21 Jun 2023 04:12:14.291 * RDB: 1 MB of memory used by copy-on-write
2023-06-21T04:17:15.000Z 550:M 21 Jun 2023 04:17:15.025 * 10 changes in 300 seconds. Saving...
2023-06-21T04:17:15.000Z 550:M 21 Jun 2023 04:17:15.041 * Background saving started by pid 558
2023-06-21T04:17:17.000Z 550:M 21 Jun 2023 04:17:17.246 * Background saving terminated with success
2023-06-21T04:17:17.000Z 558:C 21 Jun 2023 04:17:17.146 * DB saved on disk
2023-06-21T04:17:17.000Z 558:C 21 Jun 2023 04:17:17.161 * RDB: 1 MB of memory used by copy-on-write
2023-06-21T04:22:18.000Z 550:M 21 Jun 2023 04:22:18.030 * 10 changes in 300 seconds. Saving...
2023-06-21T04:22:18.000Z 550:M 21 Jun 2023 04:22:18.044 * Background saving started by pid 559
2023-06-21T04:22:20.000Z 550:M 21 Jun 2023 04:22:20.348 * Background saving terminated with success
2023-06-21T04:22:20.000Z 559:C 21 Jun 2023 04:22:20.270 * DB saved on disk
2023-06-21T04:22:20.000Z 559:C 21 Jun 2023 04:22:20.284 * RDB: 1 MB of memory used by copy-on-write
2023-06-21T04:27:21.000Z 550:M 21 Jun 2023 04:27:21.031 * 10 changes in 300 seconds. Saving...
2023-06-21T04:27:21.000Z 550:M 21 Jun 2023 04:27:21.046 * Background saving started by pid 560
2023-06-21T04:27:22.000Z 550:M 21 Jun 2023 04:27:22.750 * Background saving terminated with success
2023-06-21T04:27:22.000Z 560:C 21 Jun 2023 04:27:22.682 * DB saved on disk
2023-06-21T04:27:22.000Z 560:C 21 Jun 2023 04:27:22.696 * RDB: 1 MB of memory used by copy-on-write
2023-06-21T04:32:23.000Z 550:M 21 Jun 2023 04:32:23.033 * 10 changes in 300 seconds. Saving...
2023-06-21T04:32:23.000Z 550:M 21 Jun 2023 04:32:23.049 * Background saving started by pid 561
2023-06-21T04:32:24.000Z 2023-06-21 04:32:24,748 INFO exited: redis (terminated by SIGKILL; not expected)
2023-06-21T04:32:24.000Z 2023-06-21 04:32:24,766 INFO spawned: 'redis' with pid 562
2023-06-21T04:32:24.000Z 2023-06-21 04:32:24,804 INFO reaped unknown pid 561 (exit status 0)
2023-06-21T04:32:24.000Z 561:C 21 Jun 2023 04:32:24.741 * DB saved on disk

I'm looking at possible variables such as plugins and such, but nothing has really changed anytime recently other than existing plugin updates, and nearly all the plugins are installed and used on the other client sites too so it's strange to me it's happening on only this one site. Any ideas or suggestions to help me narrow this down would be appreciated. Thank you in advance.

[robi]

@d19dotca how much memory is it using?

drop a SS of the elevated usage from services view.

Could be a mem leak or some bot/script trying exploits with ever larger payloads which fill it up.

Does the site run wordfence/cerber or another security plugin that can prevent such?

[d19dotca]

I think I may have subsided it for not at least... I raised the memory up even further and then things stabilized, I then cleared the redis cache from the site and reduced the memory on the associated redis service, and so far it's good again. Early times though, will continue to watch. Also switched out one of the Rank Math SEO plugins with Slim SEO as it was something I've been int he middle of a few client sites anyways, so figured I'd jump and do it here as Slim SEO is quite a bit more lightweight then Rank Math, hoping that'll help a bit too.

When it was there, it was using about 800 MB at the time. It's down to about 100 MB now, back to where the other sites are.