Chunked responses from Azure OpenAI and Azure AI Search in Angular app

Mithun 30 Reputation points
2024-06-15T21:50:54.44+00:00

I'm implementing Retrieval-Augmented Generation (RAG) using Azure AI Search and Azure OpenAI to summarize search results in an Angular application. The search index is not chunked or vectorized, but the OpenAI chat completion is returning responses in a chunked array format with improper references like [doc1], [doc2], etc. This format is making it difficult to process the data correctly. Please help in resolve this issue.

Below is the OpenAI service code used:

import { Injectable } from '@angular/core';
import { AzureKeyCredential, OpenAIClient } from '@azure/openai';
import { environment } from '../../environments/environment';

@Injectable({
  providedIn: 'root',
})
export class OpenAiService {
  private client: OpenAIClient;

  constructor() {
    const endpoint = environment.openAiEndpoint;
    const apiKey = environment.openAiApiKey;
    this.client = new OpenAIClient(endpoint, new AzureKeyCredential(apiKey));
  }

  async search(prompt: string): Promise<any> {
    const deploymentId = environment.openAiDeploymentId;

    let systemMessage = `You are a cheerful news assistant who generates news summaries related to the user query. Please ensure each summary is clear, concise, and includes relevant details from the articles referenced. Avoid unnecessary punctuation such as commas between words.`;

    const messages = [
      { role: 'system', content: systemMessage },
      { role: 'user', content: prompt },
    ];

    const options = {
      maxTokens: 800,
      topP: 0.95, 
      temperature: 0.7, 
      frequencyPenalty: 0,
      presencePenalty: 0,
      azureExtensionOptions: {
        extensions: [
          {
            type: 'azure_search',
            endpoint: environment.searchEndpoint,
            indexName: environment.indexName,
            authentication: {
              type: 'api_key',
              key: environment.searchApiKey,
            },
          },
        ],
      },
    };

    const events = await this.client.streamChatCompletions(deploymentId, messages, options);

    const results = [];
    for await (const event of events) {
      for (const choice of event.choices) {
        results.push(choice.delta?.content);
      }
    }
    return results.join(' ');
  }
}

The sample output is as below:

,Here, are, some, recent, news, articles, related, to, Ukraine,:

,1,., **,Russian, War,ships, in, the, Caribbean,**,:, A, fleet, of, Russian, war,ships, began, entering, the, Caribbean,,, which, is, seen, as, a, show, of, strength, amid, growing, tensions, over, Western, support, for, Ukraine, [doc1].

,2,., **,N,ATO, and, Hungary,'s, Position,**,:, NATO, Secretary, General, Jens, Stol,ten,berg, agreed, to, allow, Hungary, to, opt, out, of, the, alliance,'s, deep,ening, support, for, Ukraine,., In, return,,, Hungary, will, not, block, any, related, NATO, decisions, [doc2].

,3,., **,U,.S,., L,ifts, Ban, on, Az,ov, Brigade,**,:, The, U,.S,., has, lifted, restrictions, on, the, transfer, of, American, weapons, and, training, to, the, Ukrainian, Az,ov, Brigade,,, a, military, unit, with, a, controversial, history, but, recognized, as, one, of, Ukraine,'s, most, effective, fighting, forces, [doc3].

,4,., **,Res,ignation, of, Ukrainian, Official,**,:, Mustafa, N,ayy,em,,, the, top, official, overseeing, Ukraine,'s, defense, fort,ifications, and, reconstruction, efforts,,, resigned,., This, marks, the, latest, high,-profile, departure, in, the, Zel,ensky, administration, [doc4].
Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
831 questions
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,538 questions
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,577 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Amira Bedhiafi 18,501 Reputation points
    2024-06-23T09:40:07.11+00:00

    I am not expert in this matter but after checking the code and some old thread, you may need to update the search method and add a formatResponse method:

    
    async search(prompt: string): Promise<any> {
    
      const deploymentId = environment.openAiDeploymentId;
    
      const systemMessage = `You are a cheerful news assistant who generates news summaries related to the user query. Please ensure each summary is clear, concise, and includes relevant details from the articles referenced. Avoid unnecessary punctuation such as commas between words.`;
    
      const messages = [
    
        { role: 'system', content: systemMessage },
    
        { role: 'user', content: prompt },
    
      ];
    
      const options = {
    
        maxTokens: 800,
    
        topP: 0.95, 
    
        temperature: 0.7, 
    
        frequencyPenalty: 0,
    
        presencePenalty: 0,
    
        azureExtensionOptions: {
    
          extensions: [
    
            {
    
              type: 'azure_search',
    
              endpoint: environment.searchEndpoint,
    
              indexName: environment.indexName,
    
              authentication: {
    
                type: 'api_key',
    
                key: environment.searchApiKey,
    
              },
    
            },
    
          ],
    
        },
    
      };
    
      const events = await this.client.streamChatCompletions(deploymentId, messages, options);
    
      let responseChunks = [];
    
      for await (const event of events) {
    
        for (const choice of event.choices) {
    
          if (choice.delta?.content) {
    
            responseChunks.push(choice.delta.content);
    
          }
    
        }
    
      }
    
      const fullResponse = responseChunks.join('').replace(/, /g, ' ');
    
      return this.formatResponse(fullResponse);
    
    }
    
    private formatResponse(response: string): string {
    
      // Clean up extra commas and spaces
    
      response = response
    
        .replace(/,\s*/g, ' ')
    
        .replace(/\s+/g, ' ')
    
        .trim();
    
      // Properly format references
    
      const formattedResponse = response.replace(/\[doc(\d+)\]/g, (match, docId) => {
    
        return `<sup>[${docId}]</sup>`;
    
      });
    
      return formattedResponse;
    
    }
    

    Add the onSearch method in your Angular component:

    
    import { Component, OnInit } from '@angular/core';
    
    import { OpenAiService } from './services/open-ai.service';
    
    @Component({
    
      selector: 'app-root',
    
      templateUrl: './app.component.html',
    
      styleUrls: ['./app.component.css'],
    
    })
    
    export class AppComponent implements OnInit {
    
      searchResult: string = '';
    
      constructor(private openAiService: OpenAiService) {}
    
      ngOnInit() {}
    
      async onSearch(prompt: string) {
    
        try {
    
          this.searchResult = await this.openAiService.search(prompt);
    
        } catch (error) {
    
          console.error('Error fetching search result', error);
    
        }
    
      }
    
    }
    

    Add the following HTML to your template:

    
    <div>
    
      <input #promptInput type="text" placeholder="Enter your query">
    
      <button (click)="onSearch(promptInput.value)">Search</button>
    
    </div>
    
    <div [innerHTML]="searchResult"></div>
    
    
    0 comments No comments