I just created a new Web App, and selected .NET 8 as the runtime. I then create a new ASP.NET Core and target in .NET 8 too. I add some code to show the runtime on the page. I can run the ASP.NET Core in Azure Web App under .NET 8 without any problem. Could you try to create new Azure Web App, deploy your app and try to have a look if it works?
If it works, then we could check if there is any other settings need to make changes too.