Thanks for reaching out to Microsoft Q&A
If the custom application or API is registered with Entra (App Registrations), you can create a Conditional Access policy scoping that app and configure the policy to require MFA for all or some users. The documentation below ha smore information about creating such policy for all apps and users, but you can filter it by users and scope it per application as well. Let me know if I'm missing anything:
Thanks,
Fabio