評估及監視生成式 AI 的計量

Azure AI Studio 可讓您評估單一回合或複雜的多回合交談，其中您可以使用特定資料建立生成式 AI 模型 (也稱為檢索增強生成或 RAG)。 您也可以評估一般單回合查詢及回應情境採用無內容來源的生成式 AI 模型 (非 RAG) 時的表現。 目前，我們支援下列工作類型的內建計量：

查詢與回應 (單一回合)

在此設定中，使用者會提出個別的查詢或提示，並使用生成式 AI 模型來立即產生回應。

測試集格式會遵循下列資料格式：

{"query":"Which tent is the most waterproof?","context":"From our product list, the Alpine Explorer tent is the most waterproof. The Adventure Dining Table has higher weight.","response":"The Alpine Explorer Tent is the most waterproof.","ground_truth":"The Alpine Explorer Tent has the highest rainfly waterproof rating at 3000m"} 

交談 (單一回合和多回合)

在此內容中，使用者會透過一系列回合或單一對話來參與交談式互動。 具備擷取機制的生成式 AI 模型會產生回應，且能夠存取並納入外部來源的資訊 (例如文件)。 檢索增強生成 (RAG) 模型會使用外部文件和知識來增強回應的品質和相關性。

測試集格式會遵循下列資料格式：

{"messages":[{"role":"user","content":"How can I check the status of my online order?"},{"content":"Hi Sarah Lee! To check the status of your online order for previous purchases such as the TrailMaster X4 Tent or the CozyNights Sleeping Bag, please refer to your email for order confirmation and tracking information. If you need further assistance, feel free to contact our customer support at support@contosotrek.com or give us a call at 1-800-555-1234.
","role":"assistant","context":{"citations":[{"id":"cHJvZHVjdF9pbmZvXzYubWQz","title":"Information about product item_number: 6","content":"# Information about product item_number: 6\n\nIt's essential to check local regulations before using the EcoFire Camping Stove, as some areas may have restrictions on open fires or require a specific type of stove.\n\n30) How do I clean and maintain the EcoFire Camping Stove?\n   To clean the EcoFire Camping Stove, allow it to cool completely, then wipe away any ash or debris with a brush or cloth. Store the stove in a dry place when not in use."}]}}]}

支援的計量

如同評估大型語言模型的方法所述，測量方法分為手動和自動化兩種。 自動化測量適用於大規模測量，擁有更大的涵蓋範圍，可提供更全面的結果。 在系統、使用方式和降低風險方式發展時，此方法也很適合持續測量來監視任何迴歸。

針對生成式 AI 應用程式的自動化測量，我們支援兩種主要方法：

• 傳統機器學習計量
• AI 輔助計量

AI 輔助的計量會利用 GPT-4 等語言模型來評估 AI 生成的輸出，特別是在預期答案因缺乏已定義根據事實而無法使用的情況下。 傳統的機器學習計量 (例如 F1 分數) 可測量 AI 產生的回應與預期答案之間的精確度和召回率。

我們的 AI 輔助計量會評估產生 AI 應用程式的安全性和生成品質。 這些計量分為兩個不同的類別：

• 風險和安全性計量：

  這些計量著重於識別潛在的內容與安全性風險，並確保所產生內容的安全性。

  其中包含：

  • 仇恨和不公平的內容
  • 性內容
  • 暴力內容
  • 自我傷害相關內容
  • 直接攻擊越獄 (UPIA，使用者提示插入攻擊)
  • 間接攻擊越獄 (XPIA，跨網域提示插入攻擊)
  • 受保護素材內容

• 生成品質計量：

  這些計量會評估所產生內容的整體品質和連貫性。

  AI 輔助計量包括:

  • 連貫性
  • 流暢度
  • 根據性
  • 相關性
  • 相似度

  傳統的 ML 計量包括:

  • F1 分數
  • ROUGE 分數
  • BLEU 分數
  • GLEU 分數
  • METEOR 分數

我們針對上述工作類型支援下列 AI 輔助計量：

風險和安全性計量

風險和安全性計量會利用我們先前大型語言模型專案 (例如 GitHub Copilot 和 Bing) 中取得的見解。 這可確保評估針對風險和安全性嚴重性分數所產生之回應的完整方法。 這些計量是透過我們的安全評估服務所產生，該服務採用了一組 LLM。 每個模型都負責評估回應中可能出現的特定風險 (例如性內容、暴力內容等)。 這些模型會提供風險定義和嚴重性規模，並據以標註生成的對話。 目前，我們會計算以下風險和安全性計量的「不良率」。 針對每個計量，服務會測量是否偵測到這些類型的內容，以及偵測到的嚴重性層級。 這四種類型各有四個嚴重性層級 (非常低、低、中、高)。 使用者指定容錯閾值，而我們的服務所產生的不良率，會對應至每個閾值層級及以上產生的執行個體數目。

內容類型：

• 仇恨和不公平的內容
• 性內容
• 暴力內容
• 自我傷害相關內容
• 間接攻擊越獄
• 直接攻擊越獄
• 受保護素材內容

您可以透過紅隊演練或我們的對立模擬器所產生的綜合測試資料集，測量您自己的資料或測試資料集上的這些風險和安全性計量。 這會輸出具有內容風險嚴重性層級 (非常低、低、中或高) 的已標註測試資料集，並在 Azure AI 中顯示結果，從而為您提供整個測試資料集的總體不良率，以及每個內容風險標籤和推理的執行個體檢視。

評估越獄弱點

我們支援評估下列類型越獄攻擊的弱點:

• 直接攻擊破解 (也稱為 UPIA 或使用者提示插入攻擊) 會在對生成式 AI 應用程式的使用者角色對話或查詢回合，插入提示。 越獄是指模型回應略過其限制時。 當 LLM 偏離預定的任務或主題時，也會發生越獄。
• 間接攻擊破解 (也稱為 XPIA 或跨網域提示插入攻擊) 會在使用者對生成式 AI 應用程式的查詢所傳回的文件或內容中，插入提示。

評估直接攻擊是使用內容安全評估工具做為控制項的比較測量。 它不是它自己的 AI 輔助計量。 在兩個不同的紅色小組資料集上執行 ContentSafetyEvaluator:

• 基準對立測試資料集。
• 第一回合有直接攻擊越獄插入的對立測試資料集。

您可以使用具有相同隨機植入之直接攻擊模擬器所產生的功能和攻擊資料集來執行此動作。 接著，您可以針對每個安全評估工具比較兩個測試資料集彙總分數之間的內容安全評估工具結果，藉此評估越獄弱點。 當第二個直接攻擊插入資料集中偵測到內容危害回應時，在第一個控制資料集中偵測到沒有或較低的嚴重性時，就會偵測到直接攻擊越獄缺陷。

評估間接攻擊 是 AI 輔助的計量，不需要比較測量，例如評估直接攻擊。 使用間接攻擊模擬器產生間接攻擊破解插入資料集，然後使用 IndirectAttackEvaluator 評估。

仇恨和不公平的內容定義和嚴重性規模

性內容定義和嚴重性規模

暴力內容定義和嚴重性規模

自我傷害相關內容定義和嚴重性規模

受保護的素材定義和標籤

定義：

受保護的素材是著作權下的任何文字，包括歌曲歌詞、食譜和文章。 受保護的素材評估會使用 Azure AI 內容安全適用於文字服務的受保護素材來執行分類。

標籤:

間接攻擊定義和標籤

定義：

間接攻擊，也稱為跨網域提示插入攻擊 (XPIA)，是在將越獄攻擊插入至文件或來源的內容時，可能會導致改變、非預期的行為。

標籤:

生成品質計量

生成品質計量可用來評估生成式 AI 應用程式所產生的內容整體品質。 以下是這些計量含義的詳細說明：

AI 輔助：根據性

針對根據性，我們提供兩個版本：

• 透過整合至 Azure AI Studio 安全評估，利用 Azure AI 內容安全服務 (AACS) 進行根據性偵測。 使用者不需要部署，後端服務將提供模型供您輸出分數和推理。 目前支援下列區域：美國東部 2 和瑞典中部。
• 使用自身模型只輸出分數的提示型根據性。 目前所有區域都支援。

AACS 型根據性

提示型根據性

大型語言模型判斷用來評分此計量的內建提示：

You will be presented with a CONTEXT and an ANSWER about that CONTEXT. You need to decide whether the ANSWER is entailed by the CONTEXT by choosing one of the following rating: 

1. 5: The ANSWER follows logically from the information contained in the CONTEXT. 

2. 1: The ANSWER is logically false from the information contained in the CONTEXT. 

3. an integer score between 1 and 5 and if such integer score does not exist,  

use 1: It is not possible to determine whether the ANSWER is true or false without further information. 

Read the passage of information thoroughly and select the correct answer from the three answer labels. 

Read the CONTEXT thoroughly to ensure you know what the CONTEXT entails.  

Note the ANSWER is generated by a computer system, it can contain certain symbols, which should not be a negative factor in the evaluation. 

AI 輔助：相關性

大型語言模型判斷用來評分此計量的內建提示 (針對查詢及回應資料格式)：

Relevance measures how well the answer addresses the main aspects of the query, based on the context. Consider whether all and only the important aspects are contained in the answer when evaluating relevance. Given the context and query, score the relevance of the answer between one to five stars using the following rating scale: 

One star: the answer completely lacks relevance 

Two stars: the answer mostly lacks relevance 

Three stars: the answer is partially relevant 

Four stars: the answer is mostly relevant 

Five stars: the answer has perfect relevance 

This rating value should always be an integer between 1 and 5. So the rating produced should be 1 or 2 or 3 or 4 or 5. 

大型語言模型判斷用來評分此計量的內建提示 (針對交談資料格式) (無可用的根據性事實)：

You will be provided a query, a conversation history, fetched documents related to the query and a response to the query in the {DOMAIN} domain. Your task is to evaluate the quality of the provided response by following the steps below:  
 
- Understand the context of the query based on the conversation history.  
 
- Generate a reference answer that is only based on the conversation history, query, and fetched documents. Don't generate the reference answer based on your own knowledge.  
 
- You need to rate the provided response according to the reference answer if it's available on a scale of 1 (poor) to 5 (excellent), based on the below criteria:  
 
5 - Ideal: The provided response includes all information necessary to answer the query based on the reference answer and conversation history. Please be strict about giving a 5 score.  
 
4 - Mostly Relevant: The provided response is mostly relevant, although it might be a little too narrow or too broad based on the reference answer and conversation history.  
 
3 - Somewhat Relevant: The provided response might be partly helpful but might be hard to read or contain other irrelevant content based on the reference answer and conversation history.  
 
2 - Barely Relevant: The provided response is barely relevant, perhaps shown as a last resort based on the reference answer and conversation history.  
 
1 - Completely Irrelevant: The provided response should never be used for answering this query based on the reference answer and conversation history.  
 
- You need to rate the provided response to be 5, if the reference answer can not be generated since no relevant documents were retrieved.  
 
- You need to first provide a scoring reason for the evaluation according to the above criteria, and then provide a score for the quality of the provided response.  
 
- You need to translate the provided response into English if it's in another language. 

- Your final response must include both the reference answer and the evaluation result. The evaluation result should be written in English.  

大型語言模型判斷用來評分此計量的內建提示 (針對交談資料格式) (有可用的根據性事實)：

Your task is to score the relevance between a generated answer and the query based on the ground truth answer in the range between 1 and 5, and please also provide the scoring reason.  
 
Your primary focus should be on determining whether the generated answer contains sufficient information to address the given query according to the ground truth answer.   
 
If the generated answer fails to provide enough relevant information or contains excessive extraneous information, then you should reduce the score accordingly.  
 
If the generated answer contradicts the ground truth answer, it will receive a low score of 1-2.   
 
For example, for query "Is the sky blue?", the ground truth answer is "Yes, the sky is blue." and the generated answer is "No, the sky is not blue.".   
 
In this example, the generated answer contradicts the ground truth answer by stating that the sky is not blue, when in fact it is blue.   
 
This inconsistency would result in a low score of 1-2, and the reason for the low score would reflect the contradiction between the generated answer and the ground truth answer.  
 
Please provide a clear reason for the low score, explaining how the generated answer contradicts the ground truth answer.  
 
Labeling standards are as following:  
 
5 - ideal, should include all information to answer the query comparing to the ground truth answer， and the generated answer is consistent with the ground truth answer  
 
4 - mostly relevant, although it might be a little too narrow or too broad comparing to the ground truth answer, and the generated answer is consistent with the ground truth answer  
 
3 - somewhat relevant, might be partly helpful but might be hard to read or contain other irrelevant content comparing to the ground truth answer, and the generated answer is consistent with the ground truth answer  
 
2 - barely relevant, perhaps shown as a last resort comparing to the ground truth answer, and the generated answer contradicts with the ground truth answer  
 
1 - completely irrelevant, should never be used for answering this query comparing to the ground truth answer, and the generated answer contradicts with the ground truth answer  

AI 輔助：連貫性

大型語言模型判斷用來評分此計量的內建提示：

Coherence of an answer is measured by how well all the sentences fit together and sound naturally as a whole. Consider the overall quality of the answer when evaluating coherence. Given the query and answer, score the coherence of answer between one to five stars using the following rating scale: 

One star: the answer completely lacks coherence 

Two stars: the answer mostly lacks coherence 

Three stars: the answer is partially coherent 

Four stars: the answer is mostly coherent 

Five stars: the answer has perfect coherency 

This rating value should always be an integer between 1 and 5. So the rating produced should be 1 or 2 or 3 or 4 or 5. 

AI 輔助：流暢度

大型語言模型判斷用來評分此計量的內建提示：

Fluency measures the quality of individual sentences in the answer, and whether they are well-written and grammatically correct. Consider the quality of individual sentences when evaluating fluency. Given the query and answer, score the fluency of the answer between one to five stars using the following rating scale: 

One star: the answer completely lacks fluency 

Two stars: the answer mostly lacks fluency 

Three stars: the answer is partially fluent 

Four stars: the answer is mostly fluent 

Five stars: the answer has perfect fluency 

This rating value should always be an integer between 1 and 5. So the rating produced should be 1 or 2 or 3 or 4 or 5. 

AI 輔助：擷取分數

大型語言模型判斷用來評分此計量的內建提示：

A chat history between user and bot is shown below 

A list of documents is shown below in json format, and each document has one unique id.  

These listed documents are used as context to answer the given question. 

The task is to score the relevance between the documents and the potential answer to the given question in the range of 1 to 5.  

1 means none of the documents is relevant to the question at all. 5 means either one of the document or combination of a few documents is ideal for answering the given question. 

Think through step by step: 

- Summarize each given document first 

- Determine the underlying intent of the given question, when the question is ambiguous, refer to the given chat history  

- Measure how suitable each document to the given question, list the document id and the corresponding relevance score.  

- Summarize the overall relevance of given list of documents to the given question after # Overall Reason, note that the answer to the question can be solely from single document or a combination of multiple documents.  

- Finally, output "# Result" followed by a score from 1 to 5.  

  

# Question 

{{ query }} 

# Chat History 

{{ history }} 

# Documents 

---BEGIN RETRIEVED DOCUMENTS--- 

{{ FullBody }} 

---END RETRIEVED DOCUMENTS--- 

AI 輔助：GPT 相似度

大型語言模型判斷用來評分此計量的內建提示：

GPT-Similarity, as a metric, measures the similarity between the predicted answer and the correct answer. If the information and content in the predicted answer is similar or equivalent to the correct answer, then the value of the Equivalence metric should be high, else it should be low. Given the question, correct answer, and predicted answer, determine the value of Equivalence metric using the following rating scale: 

One star: the predicted answer is not at all similar to the correct answer 

Two stars: the predicted answer is mostly not similar to the correct answer 

Three stars: the predicted answer is somewhat similar to the correct answer 

Four stars: the predicted answer is mostly similar to the correct answer 

Five stars: the predicted answer is completely similar to the correct answer 

This rating value should always be an integer between 1 and 5. So the rating produced should be 1 or 2 or 3 or 4 or 5. 

傳統機器學習：F1 分數

傳統機器學習：BLEU 分數

傳統機器學習: ROUGE 分數

傳統機器學習：GLEU 分數

傳統機器學習: METEOR 分數

下一步

• 透過遊樂場評估生成式 AI 應用程式
• 使用 Azure AI SDK 進行評估
• 使用 Azure AI Studio 評估您的生成式 AI 應用程式
• 檢視評估結果
• Azure AI Studio 安全評估的透明度資訊