I was digging deeper and found this article in the docs giving closure!
So the correct env vars are:
export CONTENT_SECURITY_POLICY_DIRECTIVES__FRAME_SRC="array:'self',http://localhost:3000"
export CONTENT_SECURITY_POLICY_DIRECTIVES__CONNECT_SRC="array:'self',http://localhost:3000,ws://localhost:3000"
export CONTENT_SECURITY_POLICY_DIRECTIVES__SCRIPT_SRC="array:'self','unsafe-inline'"
export CONTENT_SECURITY_POLICY_DIRECTIVES__STYLE_SRC="array:'self','unsafe-inline'"
Or specifically adjusted to your use case.