Unable to moderate images using custom categories rapid api

Mandru Venkata Durga Prasad 0 Reputation points
2024-09-10T09:25:13.64+00:00

I have created an image-based incident in azure custom categories rapid api and trained it well with lot of image samples and deployed it. After deploying i have tested it with a sample image it is not giving proper moderation result. Despite giving a non-incident image, it is giving it as a matching to the incident. The api response is as follows for a non-matching image

{"incidentMatches":[{"incidentName":"smoking-incident"}]}

Azure AI Content Safety
Azure AI Content Safety
An Azure service that enables users to identify content that is potentially offensive, risky, or otherwise undesirable. Previously known as Azure Content Moderator.
17 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. santoshkc 8,775 Reputation points Microsoft Vendor
    2024-09-10T12:07:05.2266667+00:00

    Hi @Mandru Venkata Durga Prasad,

    Thank you for reaching out to Microsoft Q&A forum!

    It seems that your custom-trained image moderation model is incorrectly flagging non-incident images as incidents. To address this, review the diversity and accuracy of your training data, consider adjusting the model's threshold, and retrain the model with these improvements. Additionally, you might want to test the model with a broader range of images and consider using the Standard API instead of the Rapid API, as it might offer more reliable results.

    I hope this helps! Thank you.

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.