Azure Guidance for Phi-3.5 endpoint not working

Glenn Wright 15

Several weeks ago, I set up an Phi-3.5-mini-instruct endpoint using Azure AI Studio. I then code based on the following tutorial:

https://github.com/microsoft/Phi-3CookBook/blob/main/code/01.Introduce/guidance.ipynb

The code worked as of Wednesday, October 30th. As of Thursday, October 31st, I get 404 not found errors for the /guidance route of the endpoint. The endpoint itself is still available in the playground and works properly for chat completions. I tried creating other endpoints and /guidance does not work on any of them. I'm not sure what changed.

YutongTie-MSFT 53,126 Reputation points

2024-11-04T18:27:31.8266667+00:00

Hello Glenn,

Thanks for reaching out to us, which region you are in? If the model is working in Azure AI Studio but not working with guidance, I will high recommend you post the question to the repo so that the code sample team may share insight as well -

https://github.com/guidance-ai/guidance/issues

https://github.com/microsoft/Phi-3CookBook/issues

Regards,

Yutong
Glenn Wright 15 Reputation points

2024-11-04T20:06:01.1366667+00:00

Looks like East US: https://Phi-3-5-mini-instruct-two.eastus.models.ai.azure.com
Glenn Wright 15 Reputation points

2024-11-04T21:30:45.0833333+00:00

Looking at the swagger.json, it looks like the /guidance route is no longer there. Was it removed intentionally?
Glenn Wright 15 Reputation points

2024-11-04T22:29:07.25+00:00

According to the swagger.json, the /guidance route is no longer there. Was that disabled intentionally on that date?
YutongTie-MSFT 53,126 Reputation points

2024-11-04T22:45:38.1666667+00:00

Hello Glenn, thanks for sharing this point, we will also check with internal resource for this point.
Jaya Krishna Mandivarapu 0 Reputation points Microsoft Employee

2024-11-08T17:41:16.5033333+00:00

Is there any update on this? I am also facing this issue on production suddenly?
YutongTie-MSFT 53,126 Reputation points

2024-11-09T03:22:38.62+00:00

Hello @Glenn Wright and Jaya,

I would high recommend you raise a support ticket for this issue, we don't see any opening ongoing issue related to this, I am happy to enable you a free ticket for this issue, please let us know if you need it.

Regards,

Yutong
Glenn Wright 15 Reputation points

2024-11-11T15:31:01.9733333+00:00

Yes, Yutong, what steps would I need to follow to raise that ticket?
Jaya Krishna Mandivarapu 0 Reputation points Microsoft Employee

2024-11-11T15:31:26.1533333+00:00

Hi @YutongTie-MSFT , thank you for your response. I reached out to guidance team too (https://github.com/guidance-ai/guidance/issues/1074). I am surprised by their response that they took it down. My question is incase if it true 1. Why it's messing the previous versions of our deployments. 2. Why there is no update poste anywhere so we can fix the production codes
Jaya Krishna Mandivarapu 0 Reputation points Microsoft Employee

2024-11-11T15:34:19.5766667+00:00

Hi @YutongTie-MSFT , I reached out to guidance team and suprised by their response (https://github.com/guidance-ai/guidance/issues/1074) that they took it down it seems. My questions would be 1. Why there is no update posted about this so we can fix our production code in hand before
YutongTie-MSFT 53,126 Reputation points

2024-11-11T21:33:38.6733333+00:00

@Jaya Krishna Mandivarapu It's a good question, I will reach out to the Microsoft Guidance Cookbook owner to provide this feedback.

Hello @Glenn Wright, please be aware that this feature is temporary unavailable, if you need further help from support team, please check on your private message.

Regards,

Yutong

Share via

Azure Guidance for Phi-3.5 endpoint not working

Your answer