Are cron jobs executed without kill capabilities?

[fbartels]

Hi,

I have a bit of a weird one here. I have an internal piece of software (a license activator) here that I am running as a cloudron app. for auditing purposes we want to have this software log into its own (rotated) file instead of the standard container output. To rotate the file I am using the savelog command followed by a kill signal to the process that is logging.

This works as long as I am running it from the exec shell of the container, but when I am trying to do this from cron it fails like this:

(for testing I am running the cron every five minutes, so look at 10:20:51, 10:25:51, ..)

Apr 21 10:20:51 + savelog -c 90 -l /app/data/logs/acme-access.log
Apr 21 10:20:51 Rotated `/app/data/logs/acme-access.log' at Thu Apr 21 08:20:51 UTC 2022.
Apr 21 10:20:51 + savelog -c 90 -l /app/data/logs/acme-error.log
Apr 21 10:20:51 Rotated `/app/data/logs/acme-error.log' at Thu Apr 21 08:20:51 UTC 2022.
Apr 21 10:20:51 + cat /run/nginx.pid
Apr 21 10:20:51 + kill -USR1 19
Apr 21 10:20:51 /app/pkg/rotate-nginx-log.sh: 7: kill: No such process
Apr 21 10:20:51 + savelog -c 90 -l /app/data/logs/license-exchanged.log
Apr 21 10:20:51 Rotated `/app/data/logs/license-exchanged.log' at Thu Apr 21 08:20:51 UTC 2022.
Apr 21 10:20:51 + cat /run/supervisord.pid
Apr 21 10:20:51 + kill -USR2 1
# this is when I haved exec'd into the container and ran the rotate script manually:
Apr 21 10:21:34 2022-04-21 08:21:34,829 INFO received SIGUSR2 indicating log reopen request
Apr 21 10:21:34 2022-04-21 08:21:34,829 INFO supervisord logreopen
Apr 21 10:21:51 2022-04-21 08:21:51,849 INFO received SIGUSR2 indicating log reopen request
Apr 21 10:21:51 2022-04-21 08:21:51,849 INFO supervisord logreopen
# this is again when run through cron
Apr 21 10:25:51 + savelog -c 90 -l /app/data/logs/acme-access.log
Apr 21 10:25:51 Rotated `/app/data/logs/acme-access.log' at Thu Apr 21 08:25:51 UTC 2022.
Apr 21 10:25:51 + savelog -c 90 -l /app/data/logs/acme-error.log
Apr 21 10:25:51 Rotated `/app/data/logs/acme-error.log' at Thu Apr 21 08:25:51 UTC 2022.
Apr 21 10:25:51 + cat /run/nginx.pid
Apr 21 10:25:51 + kill -USR1 19
Apr 21 10:25:51 /app/pkg/rotate-nginx-log.sh: 7: kill: No such process

That makes me think that the cron triggered from cloudron is maybe missing the kill capability (or even has capabilities completely deactivated). Is my assumption correct? Can this be changed?

[girish]

@fbartels The cron jobs are run using a separate sidecar container. They are using the same network as the "parent" app container but the file system (/tmp and /run) and also the process namespace are different. This is done because a) in future multi-server setups, this allows cron to be run on any node and b) cron jobs cannot interfere with app container stuff.

A fix/workaround for this is to make a curl call to http://127.0.0.:port/rotate_logs or something like that. Would that work for you?

[nebulon]

This is indeed strange, there should not be any deviation between the container configs for cron/scheduler tasks and the actual main app container.

Could there be any kind of side-effect, that this process by the referenced PID may not exist within the cron container, but in another one?

[fbartels]

@nebulon I have not yet checked if the pid also exists in another container, but I have verified that the pid in general is the correct one for the container itself.

[girish]

@fbartels The cron jobs are run using a separate sidecar container. They are using the same network as the "parent" app container but the file system (/tmp and /run) and also the process namespace are different. This is done because a) in future multi-server setups, this allows cron to be run on any node and b) cron jobs cannot interfere with app container stuff.

A fix/workaround for this is to make a curl call to http://127.0.0.:port/rotate_logs or something like that. Would that work for you?

[fbartels]

Hi @girish, ah that fully explains why I cannot interact with the processes running in the container. While your design idea is probably suitable for most cases it won't really work for this specific one.

But no harm done, then I will just add a cron service to my app container directly.