Mostly, I think, because doing basically anything with Ghost seems to require messing with code.
This is definitely a reason. It can be difficult to make any changes without going into the code.
I also don't like their 'integrations' which is a sloppy alternative to an app store for add-ons. A prime example, inserting images using unsplash, they have that built-in but it is kind of useless. If you directly add an image using the unsplash integration it can't resize it so your site ends up with large non-optimized images. Solution? Download the image and then upload it. WTAF?
Most of their integrations require adding things to the header, CDNs and so on, it just ends up bloating the site and slowing it down.
Far too opinionated in the way you should, perhaps must use it.
It is a great idea, works well for many who just want to start blogging and have no specific theme/plugin requirements but done in node.js has created so many limitations it's just not very flexible.