@girish thanks for the explanation! Yeah the change happened ~14 hours ago and the TTL was set to 24 hours I reckon. I'll just wait until tomorrow to continue 😉
My plan would have been to lower the TTL a few days prior to the A record, so the caches would invalidate sooner.