I remember some cleanup features in the unattended-upgrades package.

Good point, I thought this would be triggered by the security update job from ubuntu already, but apparently not.

Nice, thanks @girish

Thanks all! I just installed it on 1 of my Cloudron servers (at DO) to see and experience if and how it works.