@Finn
Thank you for your post! Have you looked into the Get Key REST API, which gets the public part of a stored key. There's also the Get Keys REST API, which retrieves a list of the keys in the Key Vault as JSON Web Key structures that contain the public part of a stored key.
For a full list of Azure Key Vault REST APIs.
For a full list of Azure Key Vault PS commands.
If you're looking to allow access to a specific key/set of keys out of "x" amount within your vault, this currently isn't supported. However, you can definitely leverage our User Voice forum to create a feature request so our engineering team can be aware of this.
I hope this helps, if you have any other questions, please let me know.
Thank you for your time and patience throughout this issue.