How to download an html page using powershell and keep it in storage account using powershell

Deepaklal-FT 66 Reputation points
2022-05-25T12:01:09.047+00:00

How to download an html page using powershell and keep it in azure storage account using powershell.

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,871 questions
Windows Server PowerShell
Windows Server PowerShell
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.PowerShell: A family of Microsoft task automation and configuration management frameworks consisting of a command-line shell and associated scripting language.
5,446 questions
Azure ISV (Independent Software Vendors) and Startups
Azure ISV (Independent Software Vendors) and Startups
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.ISV (Independent Software Vendors) and Startups: A Microsoft program that helps customers adopt Microsoft Cloud solutions and drive user adoption.
91 questions
0 comments No comments
{count} votes

5 answers

Sort by: Most helpful
  1. shiva patpi 13,161 Reputation points Microsoft Employee
    2022-05-25T22:14:36.95+00:00

    Hello @Deepaklal-FT ,
    Try out the below complete script for your 2 requirements: (First part is using Invoke-webrequest as suggested by Rich and later part is using azure powershell cmdlets)
    Before running the script try to set the current susbcription context using : az account set --subscription <subid>

    $filename="bing.html"
    (Invoke-webrequest -URI "http://www.bing.com").Content > $filename
    $uploadstorage=Get-AzStorageAccount -ResourceGroupName "forsharing" -Name "forsharing"
    $storcontext=$uploadstorage.Context
    Set-AzStorageBlobContent -Container "test2" -File $filename -Context $storcontext

    1 person found this answer helpful.

  2. Rich Matheisen 45,596 Reputation points
    2022-05-25T14:23:53.553+00:00

    The "download an html page" part of your question is answered by using the Invoke-WebRequest cmdlet.

    I don't use Azure so I can't answer the 2nd part. This might help you: upload-file-to-blob-storage-with-azure-functions-in-powershell-using-azure-modul

    0 comments No comments

  3. Deepaklal-FT 66 Reputation points
    2022-05-27T09:37:00.937+00:00

    @shiva patpi see the below script complete.

    $filename="terms.html"
    (Invoke-webrequest -URI "https://www.google.com/index.html").Content > $filename
    $uploadstorage=Get-AzStorageAccount -ResourceGroupName "rgname" -Name "storagename"
    $storcontext=$uploadstorage.Context
    Set-AzStorageBlobContent -Container "`$web" -File $filename -Context $storcontext -Properties @{"ContentType" = "text/html"} -Force

    (Invoke-webrequest -URI "https://www.google.com/index.html").Content > $filename1
    (Invoke-webrequest -URI "https://storageaccounturl/index.html").Content > $filename2

    Compare-Object (Get-Content $filename1) (Get-Content $filename2)

    Coparison result is not "=="

    And after uploading the spaces are coming in between the content it looks.


  4. Deepaklal-FT 66 Reputation points
    2022-06-06T12:55:05.107+00:00

    @shiva patpi @SaiKishor-MSFT @Dharshinika-FT

    Below is the answer for my problem statement. Working fine as expected. you can accept answer if possible.

    Replace in code:
    this---> (Invoke-webrequest -URI $uri).Content > $filename1
    with this ----> Invoke-webrequest -URI $uri -OutFile $filename1


  5. Limitless Technology 44,081 Reputation points
    2022-05-26T15:25:18.383+00:00

    Hi Deepaklal-FT,

    If you would like to download the azure app service contents to on-prem server and restore back to azure app service using powershell script, you can refer following steps.

    First, we can use KUDU Zip REST API download the folders as ZIP files.

    GET /api/zip/{path}/

    Since we need to use automated method way to do this, we can use service principal to connect azure account, then generate the token to call REST API.

    For service principal, you can refer following document to create service principal.

    https://video2.skills-academy.com/en-us/azure/active-directory/develop/howto-create-service-principal-porta...

    In order to call REST API, powershell usually use the Invoke-RestMethod module to call API.

    However, Invoke-RestMethod may have some problem when the site contains a lot of files.

    There is a workaround posted in above link you can refer.

    It looks like it's tied to Invoke-RestMethod and Invoke-WebRequest though, it seems to work fine when using System.Net.WebClient instead, so that can be a workaround for this.

    So the final script would be as following:

    $resourceGroupName = "henryapprg"
    $webAppName = "freewinhe"
    $slotName = ""
    $localPath = "c:\test\henry.zip"
    $ClientSecret = "serviceprincipalclientsecret"
    $ApplicationID = "serviceApplicationID"
    $TenantID = "tenantid"

    use service principal to connect Azure Account

    $passwd = ConvertTo-SecureString $ClientSecret -AsPlainText -Force
    $pscredential = New-Object System.Management.Automation.PSCredential($ApplicationID, $passwd)
    Connect-AzAccount -Credential $pscredential -Tenant $TenantID -ServicePrincipal

    generate token with Azure Account.

    $azProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile
    $context =Get-AzContext
    $profileClient = New-Object Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient($azProfile)
    Write-Debug ("Getting access token for tenant" + $currentAzureContext.Subscription.TenantId)
    $token = $profileClient.AcquireAccessToken($context.Subscription.TenantId)
    $token.AccessToken
    $kuduApiAuthorisationToken = 'Bearer {0}' -f $token.AccessToken

    if ($slotName -eq ""){  
        $kuduApiUrl = "https://$webAppName.scm.azurewebsites.net/api/zip/site/wwwroot/"  
    }  
    else{  
        $kuduApiUrl = "https://$webAppName`-$slotName.scm.azurewebsites.net/api/zip/site/wwwroot/"  
    }  
    $virtualPath = $kuduApiUrl.Replace(".scm.azurewebsites.", ".azurewebsites.").Replace("/api/zip/site/wwwroot", "")  
    Write-Host " Downloading File from WebApp. Source: '$kuduApiUrl'. Target: '$localPath'..." -ForegroundColor DarkGray  
    

    Call zip Rest API to download the Zip file from webapp.

    `
    $WebClient = New-Object System.Net.WebClient
    $WebClient.Headers.Add('Authorization', $kuduApiAuthorisationToken)
    $WebClient.Headers.Add('ContentType', 'multipart/form-data')
    $WebClient.DownloadFile($kuduApiUrl, $localPath)

    Second, there is a document that introduce a powershell command to zip deploy.

    https://video2.skills-academy.com/en-us/azure/app-service/deploy-zip#with-powershell

    Publish-AzWebapp -ResourceGroupName <group-name> -Name <app-name> -ArchivePath <zip-file-path>

    So the restore script would be like this:

    $localPath = "c:\test\henry.zip"
    $ClientSecret = "serviceprincipalclientsecret"
    $ApplicationID = "serviceApplicationID"
    $TenantID = "tenantid"

    use service principal to connect Azure Account

    $passwd = ConvertTo-SecureString $ClientSecret -AsPlainText -Force
    $pscredential = New-Object System.Management.Automation.PSCredential($ApplicationID, $passwd)
    Connect-AzAccount -Credential $pscredential -Tenant $TenantID -ServicePrincipal

    Publish-AzWebapp -ResourceGroupName "henryapprg" -Name "phpwinhenry" -ArchivePath $localPath -Force


    Thanks.

    --If the reply is helpful, please Upvote and Accept as answer--

    0 comments No comments