Invoke-AzureHDInsightHiveJob

Submits Hive queries to an HDInsight cluster, shows progress of the query execution, and gets query results in one operation.

Note

The cmdlets referenced in this documentation are for managing legacy Azure resources that use Azure Service Manager (ASM) APIs. This legacy PowerShell module isn't recommended when creating new resources since ASM is scheduled for retirement. For more information, see Azure Service Manager retirement.

The Az PowerShell module is the recommended PowerShell module for managing Azure Resource Manager (ARM) resources with PowerShell.

Syntax

Invoke-AzureHDInsightHiveJob
      [-Arguments <String[]>]
      [-Defines <Hashtable>]
      [-File <String>]
      [-Files <String[]>]
      [-JobName <String>]
      [-Query <String>]
      [-RunAsFileJob]
      [-StatusFolder <String>]
      [-Profile <AzureSMProfile>]
      [<CommonParameters>]

Description

This version of Azure PowerShell HDInsight is deprecated. These cmdlets will be removed by January 1, 2017. Please use the newer version of Azure PowerShell HDInsight.

For information about how to use the new HDInsight to create a cluster, see Create Linux-based clusters in HDInsight using Azure PowerShell (https://azure.microsoft.com/en-us/documentation/articles/hdinsight-hadoop-create-linux-clusters-azure-powershell/). For information about how to submit jobs by using Azure PowerShell and other approaches, see Submit Hadoop jobs in HDInsight (https://azure.microsoft.com/en-us/documentation/articles/hdinsight-submit-hadoop-jobs-programmatically/). For reference information about Azure PowerShell HDInsight, see Azure HDInsight Cmdlets.

The Invoke-AzureHDInsightHiveJob cmdlet submits Hive queries to an HDInsight cluster, displays the progress of the query execution, and gets the query results in one operation. You must run the Use-AzureHDInsightCluster cmdlet before running Invoke-AzureHDInsightHiveJob to specify the HDInsight cluster to which to submit a query.

Examples

Example 1: Submit a Hive query

PS C:\>Use-AzureHDInsightCluster "Cluster01" -Subscription (Get-AzureSubscription -Current).SubscriptionId
PS C:\> Invoke-AzureHDInsightHiveJob "select * from hivesampletable limit 10"

The first command uses the Use-AzureHDInsightCluster cmdlet to specify a cluster in the current subscription to use for a Hive query.

The second command uses the Invoke-AzureHDInsightHiveJob cmdlet to submit the Hive query.

Parameters

-Arguments

Specifies an array of arguments for a Hadoop job. The arguments are passed as command-line arguments to each task.

Type:String[]
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Defines

Specifies Hadoop configuration values to set when a job runs.

Type:Hashtable
Aliases:Params
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-File

Specifies the Windows Azure Storage Blob (WASB) path to a file in Azure blob storage that contains the query to run. You can use this parameter instead of the Query parameter.

Type:String
Aliases:QueryFile
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Files

Specifies a collection of files that are required for a Hive job.

Type:String[]
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-JobName

Specifies the name of a Hive job. If you do not specify this parameter, this cmdlet uses the default value: "Hive: <first 100 characters of Query>".

Type:String
Aliases:Name
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Profile

Specifies the Azure profile from which this cmdlet reads. If you do not specify a profile, this cmdlet reads from the local default profile.

Type:AzureSMProfile
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-Query

Specifies a Hive query.

Type:String
Aliases:QueryText
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-RunAsFileJob

Indicates that this cmdlet creates a file in the default Azure storage account in which to store a query. This cmdlet submits the job that references this file as a script to run.

You can use this functionality to handle special characters such as percent sign (%) that would fail on a job submission through Templeton, because Templeton interprets a query with a percent sign as a URL parameter.

Type:SwitchParameter
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False

-StatusFolder

Specifies the location of the folder that contains standard outputs and error outputs for a job, including its exit code and task logs.

Type:String
Position:Named
Default value:None
Required:False
Accept pipeline input:False
Accept wildcard characters:False