Hi @Eda Amos ,
I have reproduced same scenario in my environment. I am successfully deleting the constraint that I had added earlier. Here are the steps below.
1.Log in with the guest account.
2.Select the owner role of the subscription level.
2.Configure constraint role as Key vault administrator.
4.Delete the Key vault administrator role.
Could you confirm whether my previous steps are in line with your intentions? If the alignment is correct, please send us an email on azcommunity [at] microsoft [dot] com referencing this issue with a subject line "ATTN: Navya" along with you Azure subscription id and we will help you with alternative support options on this.
If the above is not your scenario, please correct me and provide us with detailed step.
Thanks,
Navya.