How to stop default .azurewebsites.net domain from being indexed and appearing on Google search(asp.net core mvc 6)

Louis Chan 20 Reputation points
2024-09-04T04:53:40.6966667+00:00

Hello, everyone.

Thanks to read my issue.

I encountered a problem with my website domain. I deployed asp.net core mvc 6 project to azure app service, and worked on free azure default domain "domain.azurewebsite.net".

And then I added a custom domain purchased from other service and configured to use this domain.

It works well without any other problem.

But I found that both the last default domain and custom domain appears in google search.

I tried to add 301 redirection and robots.txt to redirect the request to last domain to custom domain, but the problem continues.

I want to show only the custom domain on search engine.

I checked the following question related to wordpress, but no help.

https://video2.skills-academy.com/en-us/answers/questions/1318669/how-to-stop-default-azurewebsites-net-domain-from

I hope someone help me with great heart. Any advice would be welcome.

Thanks in advance.

ASP.NET Core
ASP.NET Core
A set of technologies in the .NET Framework for building web applications and XML web services.
4,515 questions
Microsoft Deployment Toolkit
Microsoft Deployment Toolkit
A collection of Microsoft tools and documentation for automating desktop and server deployment. Previously known as Microsoft Solution Accelerator for Business Desktop Deployment (BDD).
886 questions
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
7,663 questions
0 comments
Accepted answer
  1. JasonPan - MSFT 5,456 Reputation points Microsoft Vendor
    2024-09-04T08:14:03.2366667+00:00

    Hi @Louis Chan,

    After investigating the issue and I found we can implement this feature by using rewrite rules.

    <?xml version="1.0" encoding="utf-8"?>
<configuration>
  <location path="." inheritInChildApplications="false">
    <system.webServer>
      <rewrite>
        <!--<rules>-->
          <!-- Block access to Azure default domain names -->
          <!--<rule name="Block Azure Default Domain" stopProcessing="true">
            <match url="(.*)" />
            <conditions>
              <add input="{HTTP_HOST}" pattern="^your_app_name\.azurewebsites\.net$" />
            </conditions>
            <action type="CustomResponse" statusCode="404" statusReason="Not Found" statusDescription="This page is not available." />
          </rule>
        </rules>-->
        <outboundRules>
          <!-- Add X-Robots-Tag header -->
          <rule name="Noindex Azure Default Domain" preCondition="ResponseIsHtml">
            <match serverVariable="RESPONSE_X_ROBOTS_TAG" pattern=".*" />
            <conditions>
              <!-- if request from default domain -->
              <add input="{HTTP_HOST}" pattern="^webapplication220240628111356\.azurewebsites\.net$" />
            </conditions>
            <!-- set X-Robots-Tag  -->
            <action type="Rewrite" value="noindex, nofollow" />
          </rule>
          <!-- Precondition to check if response is HTML -->
          <preConditions>
            <preCondition name="ResponseIsHtml">
              <add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/html" />
            </preCondition>
          </preConditions>
        </outboundRules>
      </rewrite>
      <handlers>
        <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
      </handlers>
      <aspNetCore processPath="dotnet" arguments=".\WebApplication1.dll" stdoutLogEnabled="false" stdoutLogFile="\\?\%home%\LogFiles\stdout" hostingModel="inprocess" />
    </system.webServer>
  </location>
</configuration>

    If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

    Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

    Best regards,

    Jason

    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Sreeju Nair 12,346 Reputation points
    2024-09-04T06:21:25.9133333+00:00

    Here are few things you could do.

    1. 301 Redirects: Ensure that all traffic from the Azure default domain (domain.azurewebsites.net) is redirected to your custom domain using 301 redirects.
    2. Canonical Tags: Use canonical tags on your web pages to indicate to search engines that the content on the custom domain is the original version and should be the one to be indexed. e.g.
    3. Use google search console, monitor both domains in google search console. This will allow you to see what is indexed from your sites.
    4. Server-Side Checks: Implement server-side checks to ensure that your content is served only under your custom domain

    Hope this helps.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.