Figured it out. There was throttling happening at the subscription level. The error is not very clear about exactly why it was failing.
To find this I started to open a ticket via the support area in the portal. That runs the Subscription throttling diagnostic which found the issue and the calls being made.