Measure Azure Cosmos DB for NoSQL performance with a benchmarking framework

Navigate to your existing API for NoSQL account in the Azure portal.

In the resource menu, select Data Explorer.

On the Data Explorer page, select the New Container option in the command bar.

In the New Container dialog, create a new container with the following settings:

If you haven't already, sign in to the Azure CLI using the az login command.

Create shell variables for the following values:

• Name of your existing Azure Cosmos DB for NoSQL account named cosmosAccountName.
• Name of your first resource group with resources named sourceResourceGroupName.
• Name of your second empty resource group named targetResourceGroupName.
• Existing Azure Cosmos DB for NoSQL account endpoint URI named cosmosEndpoint
• Existing Azure Cosmos DB for NoSQL account primary key named cosmosPrimaryKey

# Variable for Azure Cosmos DB for NoSQL account name
cosmosAccountName="<cosmos-db-nosql-account-name>"

# Variable for resource group with Azure Cosmos DB and Azure Storage accounts
sourceResourceGroupName="<first-resource-group-name>"

# Variable for empty resource group
targetResourceGroupName="<second-resource-group-name>"

# Variable for API for NoSQL endpoint URI
cosmosEndpoint="<cosmos-db-nosql-endpoint-uri>"

# Variable for API for NoSQL primary key
cosmosPrimaryKey="<cosmos-db-nosql-primary-key>"

# Variable for Azure Storage account name
storageAccountName="<storage-account-name>"

# Variable for storage account connection string
storageConnectionString="<storage-connection-string>"

Using the az cosmosdb sql database create command, create a new database with the following settings:

az cosmosdb sql database create \
    --resource-group $sourceResourceGroupName \
    --account-name $cosmosAccountName \
    --name "ycsb" \
    --throughput 400

Using the az cosmosdb sql container create command, create a new container with the following settings:

az cosmosdb sql container create \
    --resource-group $sourceResourceGroupName \
    --account-name $cosmosAccountName \
    --database-name "ycsb" \
    --name "usertable" \
    --partition-key-path "/id"

Deploy the benchmarking framework using an Azure Resource Manager template available at this link.

On the Custom Deployment page, the following parameters

Select Review + create and then Create to deploy the template.

Wait for the deployment to complete.

Use az deployment group create to deploy the benchmarking framework using an Azure Resource Manager template.

# Variable for raw template JSON on GitHub
templateUri="https://raw.githubusercontent.com/Azure/azure-db-benchmarking/main/cosmos/sql/tools/java/ycsb/recipes/read/try-it-read/azuredeploy.json"

az deployment group create \
    --resource-group $targetResourceGroupName \
    --name "benchmarking-framework" \
    --template-uri $templateUri \
    --parameters \
        adminPassword='P@ssw.rd' \
        resultsStorageConnectionString=$storageConnectionString \
        cosmosURI=$cosmosEndpoint \
        cosmosKey=$cosmosPrimaryKey

Wait for the deployment to complete.

Navigate to your existing Azure Storage account in the Azure portal.

Navigate to a storage table named ycsbbenchmarkingmetadata and locate the entity with a partition key of ycsb_sql.

Observe the JobStatus field of the table entity. Initially, the status of the job is Started and it includes a timestamp in the JobStartTime property but not the JobFinishTime property.

Wait until the job has a status of Finished and includes a timestamp in the JobFinishTime property.

Navigate to the storage container in the same account with a prefix of ycsbbenchmarking-*. Observe the output and diagnostic blobs for the tool.

Open the aggregation.csv blob and observe the content. You should now have a CSV dataset with aggregated results from all the benchmark clients.

Operation,Count,Throughput,Min(microsecond),Max(microsecond),Avg(microsecond),P9S(microsecond),P99(microsecond)
READ,180000,299,706,448255,1079,1159,2867

Query the job record in a storage table named ycsbbenchmarkingmetadata using az storage entity query.

az storage entity query \
    --account-name $storageAccountName \
    --connection-string $storageConnectionString \
    --table-name ycsbbenchmarkingmetadata

Observe the results of this query. The results should return a single job with JobStartTime, JobStatus, and JobFinishTime properties. Initially, the status of the job is Started and it includes a timestamp in the JobStartTime property but not the JobFinishTime property.

{
  "items": [
    {
      "JobFinishTime": "",
      "JobStartTime": "2023-02-02T13:59:42Z",
      "JobStatus": "Started",
      "NoOfClientsCompleted": "0",
      "NoOfClientsStarted": {
        "edm_type": "Edm.Int64",
        "value": 1
      },
      "PartitionKey": "ycsb_sql",
      ...
    }
  ],
  ...
}

If necessary, run az storage entity query multiple times until the job has a status of Finished and includes a timestamp in the JobFinishTime property.

{
  "items": [
    {
      "JobFinishTime": "2023-02-02T14:21:12Z",
      "JobStartTime": "2023-02-02T13:59:42Z",
      "JobStatus": "Finished",
      ...
    }
  ],
  ...
}

Find the name of the most recently modified storage container with a prefix of ycsbbenchmarking-* using az storage container list and a JMESPath query.

az storage container list \
    --account-name $storageAccountName \
    --connection-string $storageConnectionString \
    --query "sort_by([?starts_with(name, 'ycsbbenchmarking-')], &properties.lastModified)[-1].name" \
    --output tsv

Store the container string in a variable named storageConnectionString.

storageContainerName=$( \
    az storage container list \
        --account-name $storageAccountName \
        --connection-string $storageConnectionString \
        --query "sort_by([?starts_with(name, 'ycsbbenchmarking-')], &properties.lastModified)[-1].name" \
        --output tsv \
)

Use [az storage blob query]/cli/azure/storage/blob#az-storage-blob-query) to query the job results in a storage blob stored in the previously located container.

az storage blob query \
    --account-name $storageAccountName \
    --connection-string $storageConnectionString \
    --container-name $storageContainerName \
    --name aggregation.csv \
    --query-expression "SELECT * FROM BlobStorage"

Observe the results of this query. You should now have a CSV dataset with aggregated results from all the benchmark clients.

Operation,Count,Throughput,Min(microsecond),Max(microsecond),Avg(microsecond),P9S(microsecond),P99(microsecond)
READ,180000,299,706,448255,1079,1159,2867