Build an LLM solution in Azure

Hello Thomas Neubauer

Thanks for reaching out to us, there are three resource I think you may interested in. Basically you may need to use Azure Machine Learning Service(prompt flow or model catalog),

First one is Azure Machine Learning prompt flow, Azure Machine Learning prompt flow is a development tool designed to streamline the entire development cycle of AI applications powered by Large Language Models (LLMs). Prompt flow provides a comprehensive solution that simplifies the process of prototyping, experimenting, iterating, and deploying your AI applications.

With prompt flow's methodical process, you can develop, test, refine, and deploy sophisticated AI applications confidently.

Document -

https://video2.skills-academy.com/en-us/azure/machine-learning/prompt-flow/overview-what-is-prompt-flow?view=azureml-api-2

Second one is Azure Machine Learning catalog, model catalog in Azure Machine Learning studio is the hub to discover and use a wide range of models that enable you to build Generative AI applications. The model catalog features hundreds of models from model providers such as Azure OpenAI service, Mistral, Meta, Cohere, Nvidia, Hugging Face, including models trained by Microsoft. Models from providers other than Microsoft are Non-Microsoft Products, as defined in Microsoft's Product Terms, and subject to the terms provided with the model.

Document - https://video2.skills-academy.com/en-us/azure/machine-learning/concept-model-catalog?view=azureml-api-2

Finally there is a tutorial can help you draw a outline of the LLM deployment in Azure - Integrating a Large Language Model on Azure with Power Platform

https://video2.skills-academy.com/en-us/microsoft-cloud/dev/tutorials/aml-powerapps-powerautomate

This tutorial contains a basic outline of the LLM deployment in Azure - you'll learn how to create an Azure Machine Learning Workspace and deploy a Large Language model (LLM). You'll then integrate the LLM with Power Apps and Power Automate. Enhance your technical skills and explore the power of Azure and Power Automate in text generation and creative writing.

• Azure Machine Learning: Empowers data scientists and developers to build, deploy, and manage high-quality models faster and with confidence. It accelerates time to value with industry-leading machine learning operations (MLOps), open-source interoperability, and integrated tools. This trusted AI learning platform is designed for responsible AI applications in machine learning.
• Power Apps: Empower your team to start building and launching apps right away using an AI copilot, prebuilt templates, drag-and-drop simplicity, and quick deployment then roll out continuous improvements as needed. Give everyone the power to build the apps they need with advanced functionality previously only available to professional developers including pre-built components and AI-assisted natural language development. Provide professional developers the tools to seamlessly extend app capabilities with Azure Functions and custom connectors to proprietary or on-premises systems.
• Power Automate: Empower everyone to build automated processes using low-code, drag-and-drop tools. With hundreds of prebuilt connectors, thousands of templates, and AI assistance, it’s easy to automate repetitive tasks. Record and visualize your end-to-end processes with Power Automate Process Mining. It takes the guesswork out of what to automate by providing guided recommendations for creating flows. Make your automation even smarter using generative AI capabilities in AI Builder. Create user-intuitive flows by embedding powerful language models and build unique scenarios with advanced low-code AI.

In this tutorial, you'll construct a Power App that captures user input, sends it to a Power Automate cloud flow, retrieves a response from your machine learning model, and displays the result on-screen. Here's an overview of the solution:

For other service you may also want to refer to -

Data Preparation

• Azure Data Factory: Use for data integration and preparation. It helps in orchestrating and automating data workflows.
• Azure Blob Storage: Store large volumes of raw and processed data.
• Azure Data Lake Storage: For big data storage and analytics.
• Azure SQL Database / Azure Cosmos DB: For structured data storage and querying.

Model Training

• Azure Machine Learning (Azure ML): This is the primary service for training and managing machine learning models. It provides tools for model training, hyperparameter tuning, experiment tracking, and version control.
  • Azure ML Workspaces: Organize and manage your machine learning projects.
    • Azure ML Compute: Provision compute resources (e.g., GPUs) for model training.
      • Azure ML Pipelines: Create end-to-end workflows for data preparation, training, and deployment.

Model Deployment

• Azure Kubernetes Service (AKS): Use for deploying and scaling containerized applications, including models.
• Azure Container Instances (ACI): For simpler, stateless deployments of containerized models.
• Azure Functions: For serverless execution of lightweight tasks, like triggering model inference.

Model Inference

• Azure Cognitive Services: If you are using pre-built models or APIs for specific tasks (like language understanding or text analytics).
• Azure ML Endpoints: Deploy models as REST endpoints using Azure ML’s real-time or batch inferencing.

Monitoring and Management

• Azure Monitor: Collect and analyze log and performance data from your applications and infrastructure.
• Azure Application Insights: Monitor the health and performance of your deployed models.
• Azure Log Analytics: Query and analyze logs from various sources for troubleshooting and insights.

I hope some of the information helps! Let us know if you have any other questions when you explore those options.

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, thanks a lot.