Evaluations - List

Gets the list of evaluations for the authenticated subscription.

GET {endpoint}/speechtotext/v3.2/evaluations
GET {endpoint}/speechtotext/v3.2/evaluations?skip={skip}&top={top}&filter={filter}

URI Parameters

Name In Required Type Description
endpoint
path True

string

Supported Cognitive Services endpoints (protocol and hostname, for example: https://westus.api.cognitive.microsoft.com).

filter
query

string

A filtering expression for selecting a subset of the available evaluations.

        - Supported properties: displayName, description, createdDateTime, lastActionDateTime, status and locale.

        - Operators:

          - eq, ne are supported for all properties.

          - gt, ge, lt, le are supported for createdDateTime and lastActionDateTime.

          - and, or, not are supported.

        - Example:

          filter=displayName eq 'My evaluation'
skip
query

integer

int32

Number of datasets that will be skipped.

top
query

integer

int32

Number of datasets that will be included after skipping.

Responses

Name Type Description
200 OK

PaginatedEvaluations

OK

Headers

Retry-After: integer

Other Status Codes

Error

An error occurred.

Security

Ocp-Apim-Subscription-Key

Provide your cognitive services account key here.

Type: apiKey
In: header

Authorization

Provide an access token from the JWT returned by the STS of this region. Make sure to add the management scope to the token by adding the following query string to the STS URL: ?scope=speechservicesmanagement

Type: apiKey
In: header

Examples

Get all evaluations
Get all evaluations with where status is equal to 'Running'

Get all evaluations

Sample request

GET {endpoint}/speechtotext/v3.2/evaluations?skip=0&top=2

Sample response

{
  "values": [
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/a55a327b-c866-40b3-a08d-9c0945738633",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate1": 0.065,
        "sentenceErrorRate1": 0.375,
        "tokenErrorRate1": 0.125,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0,
        "tokenErrors1": {
          "punctuation": {
            "numberOfEdits": 2,
            "percentageOfAllEdits": 0.66
          },
          "capitalization": {
            "numberOfEdits": 1,
            "percentageOfAllEdits": 0.33
          }
        },
        "wordErrorRate2": 0.065,
        "sentenceErrorRate2": 0.375,
        "tokenErrorRate2": 0.125,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0,
        "tokenErrors2": {
          "punctuation": {
            "numberOfEdits": 208,
            "percentageOfAllEdits": 1.22
          },
          "capitalization": {
            "numberOfEdits": 126,
            "percentageOfAllEdits": 0.74
          },
          "inverseTextNormalization": {
            "numberOfEdits": 409,
            "percentageOfAllEdits": 2.4
          },
          "lexical": {
            "numberOfEdits": 16284,
            "percentageOfAllEdits": 95.41
          },
          "others": {
            "numberOfEdits": 41,
            "percentageOfAllEdits": 0.24
          }
        }
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evaluation of one model against another model"
    },
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/daaa45a7-38f0-4918-87d2-bf55fec32ac5",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate1": 0.065,
        "sentenceErrorRate1": 0.375,
        "tokenErrorRate1": 0.125,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0,
        "tokenErrors1": {
          "punctuation": {
            "numberOfEdits": 2,
            "percentageOfAllEdits": 0.66
          },
          "capitalization": {
            "numberOfEdits": 1,
            "percentageOfAllEdits": 0.33
          }
        },
        "wordErrorRate2": 0.065,
        "sentenceErrorRate2": 0.375,
        "tokenErrorRate2": 0.125,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0,
        "tokenErrors2": {
          "punctuation": {
            "numberOfEdits": 208,
            "percentageOfAllEdits": 1.22
          },
          "capitalization": {
            "numberOfEdits": 126,
            "percentageOfAllEdits": 0.74
          },
          "inverseTextNormalization": {
            "numberOfEdits": 409,
            "percentageOfAllEdits": 2.4
          },
          "lexical": {
            "numberOfEdits": 16284,
            "percentageOfAllEdits": 95.41
          },
          "others": {
            "numberOfEdits": 41,
            "percentageOfAllEdits": 0.24
          }
        }
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evaluation of one model against another model"
    }
  ],
  "@nextLink": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/1c50ce42-6ab7-454f-8c39-54a752d1a5b6/files?skip=2&top=2"
}

Get all evaluations with where status is equal to 'Running'

Sample request

GET {endpoint}/speechtotext/v3.2/evaluations?skip=0&top=2&filter=status%20eq%20'Running'

Sample response

{
  "values": [
    {
      "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/f85a328c-c866-40b3-a08d-9c0945738633",
      "model1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/base/8a043172-65b7-4cf3-a8b5-4449efa5a0f1"
      },
      "model2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/models/827712a5-f942-4997-91c3-7c6cde35600b"
      },
      "dataset": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/datasets/9d5f4100-5f8e-4dd6-bd83-9bbbf50d57f1"
      },
      "transcription2": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ad86d0c9-af6d-4d14-a621-c60d7d65b74f"
      },
      "transcription1": {
        "self": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/transcriptions/ba7ea6f5-3065-40b7-b49a-a90f48584683"
      },
      "links": {
        "files": "https://westus.api.cognitive.microsoft.com/speechtotext/v3.2/evaluations/00000000-0000-0000-0000-000000000000/files"
      },
      "properties": {
        "wordErrorRate1": 0.89,
        "sentenceErrorRate1": 0.375,
        "tokenErrorRate1": 0.125,
        "sentenceCount1": 8,
        "wordCount1": 46,
        "correctWordCount1": 43,
        "wordSubstitutionCount1": 3,
        "wordDeletionCount1": 0,
        "wordInsertionCount1": 0,
        "tokenCount1": 48,
        "correctTokenCount1": 42,
        "tokenSubstitutionCount1": 6,
        "tokenDeletionCount1": 0,
        "tokenInsertionCount1": 0,
        "wordErrorRate2": 0.98,
        "sentenceErrorRate2": 0.375,
        "tokenErrorRate2": 0.125,
        "sentenceCount2": 8,
        "wordCount2": 46,
        "correctWordCount2": 43,
        "wordSubstitutionCount2": 3,
        "wordDeletionCount2": 0,
        "wordInsertionCount2": 0,
        "tokenCount2": 48,
        "correctTokenCount2": 42,
        "tokenSubstitutionCount2": 6,
        "tokenDeletionCount2": 0,
        "tokenInsertionCount2": 0
      },
      "lastActionDateTime": "2019-01-07T11:36:07Z",
      "status": "Succeeded",
      "createdDateTime": "2019-01-07T11:34:12Z",
      "locale": "en-US",
      "displayName": "Evaluation of one model against another model"
    }
  ]
}

Definitions

Name Description
DetailedErrorCode

DetailedErrorCode

EditsSummary

EditsSummary

EntityError

EntityError

EntityReference

EntityReference

Error

Error

ErrorCode

ErrorCode

Evaluation

Evaluation

EvaluationLinks

EvaluationLinks

EvaluationProperties

EvaluationProperties

InnerError

InnerError

PaginatedEvaluations

PaginatedEvaluations

Status

Status

TokenErrorDetails

TokenErrorDetails

DetailedErrorCode

DetailedErrorCode

Name Type Description
DataImportFailed

string

Data import failed.

DeleteNotAllowed

string

Delete not allowed.

DeployNotAllowed

string

Deploy not allowed.

DeployingFailedModel

string

Deploying failed model.

EmptyRequest

string

Empty Request.

EndpointCannotBeDefault

string

Endpoint cannot be default.

EndpointNotUpdatable

string

Endpoint not updatable.

EndpointWithoutLogging

string

Endpoint without logging.

ExceededNumberOfRecordingsUris

string

Exceeded number of recordings uris.

FailedDataset

string

Failed dataset.

Forbidden

string

Forbidden.

InUseViolation

string

In use violation.

InaccessibleCustomerStorage

string

Inaccessible customer storage.

InvalidAdaptationMapping

string

Invalid adaptation mapping.

InvalidBaseModel

string

Invalid base model.

InvalidCallbackUri

string

Invalid callback uri.

InvalidChannels

string

Invalid channels.

InvalidCollection

string

Invalid collection.

InvalidDataset

string

Invalid dataset.

InvalidDocument

string

Invalid Document.

InvalidDocumentBatch

string

Invalid Document Batch.

InvalidLocale

string

Invalid locale.

InvalidLogDate

string

Invalid log date.

InvalidLogEndTime

string

Invalid log end time.

InvalidLogId

string

Invalid log id.

InvalidLogStartTime

string

Invalid log start time.

InvalidModel

string

Invalid model.

InvalidModelUri

string

Invalid model uri.

InvalidParameter

string

Invalid parameter.

InvalidParameterValue

string

Invalid parameter value.

InvalidPayload

string

Invalid payload.

InvalidPermissions

string

Invalid permissions.

InvalidPrerequisite

string

Invalid prerequisite.

InvalidProductId

string

Invalid product id.

InvalidProject

string

Invalid project.

InvalidProjectKind

string

Invalid project kind.

InvalidRecordingsUri

string

Invalid recordings uri.

InvalidRequestBodyFormat

string

Invalid request body format.

InvalidSasValidityDuration

string

Invalid sas validity duration.

InvalidSkipTokenForLogs

string

Invalid skip token for logs.

InvalidSourceAzureResourceId

string

Invalid source Azure resource ID.

InvalidSubscription

string

Invalid subscription.

InvalidTest

string

Invalid test.

InvalidTimeToLive

string

Invalid time to live.

InvalidTopForLogs

string

Invalid top for logs.

InvalidTranscription

string

Invalid transcription.

InvalidWebHookEventKind

string

Invalid web hook event kind.

MissingInputRecords

string

Missing Input Records.

ModelCopyAuthorizationExpired

string

Expired ModelCopyAuthorization.

ModelDeploymentNotCompleteState

string

Model deployment not complete state.

ModelDeprecated

string

Model deprecated.

ModelExists

string

Model exists.

ModelMismatch

string

Model mismatch.

ModelNotDeployable

string

Model not deployable.

ModelVersionIncorrect

string

Model Version Incorrect.

NoUtf8WithBom

string

No utf8 with bom.

OnlyOneOfUrlsOrContainerOrDataset

string

Only one of urls or container or dataset.

ProjectGenderMismatch

string

Project gender mismatch.

QuotaViolation

string

Quota violation.

SingleDefaultEndpoint

string

Single default endpoint.

SkuLimitsExist

string

Sku limits exist.

SubscriptionNotFound

string

Subscription not found.

UnexpectedError

string

Unexpected error.

UnsupportedClassBasedAdaptation

string

Unsupported class based adaptation.

UnsupportedDelta

string

Unsupported delta.

UnsupportedDynamicConfiguration

string

Unsupported dynamic configuration.

UnsupportedFilter

string

Unsupported filter.

UnsupportedLanguageCode

string

Unsupported language code.

UnsupportedOrderBy

string

Unsupported order by.

UnsupportedPagination

string

Unsupported pagination.

UnsupportedTimeRange

string

Unsupported time range.

EditsSummary

EditsSummary

Name Type Description
numberOfEdits

integer

The optional number of edits for a given type of error of the recognized transcription in comparison with the human transcription.

percentageOfAllEdits

number

The optional percentage of edits for a given type of error of the recognized transcription in comparison with the human transcription.

EntityError

EntityError

Name Type Description
code

string

The code of this error.

message

string

The message for this error.

EntityReference

EntityReference

Name Type Description
self

string

The location of the referenced entity.

Error

Error

Name Type Description
code

ErrorCode

ErrorCode
High level error codes.

details

Error[]

Additional supportive details regarding the error and/or expected policies.

innerError

InnerError

InnerError
New Inner Error format which conforms to Cognitive Services API Guidelines which is available at https://microsoft.sharepoint.com/%3Aw%3A/t/CognitiveServicesPMO/EUoytcrjuJdKpeOKIK_QRC8BPtUYQpKBi8JsWyeDMRsWlQ?e=CPq8ow. This contains required properties ErrorCode, message and optional properties target, details(key value pair), inner error(this can be nested).

message

string

High level error message.

target

string

The source of the error. For example it would be "documents" or "document id" in case of invalid document.

ErrorCode

ErrorCode

Name Type Description
Conflict

string

Representing the conflict error code.

Forbidden

string

Representing the forbidden error code.

InternalCommunicationFailed

string

Representing the internal communication failed error code.

InternalServerError

string

Representing the internal server error error code.

InvalidArgument

string

Representing the invalid argument error code.

InvalidRequest

string

Representing the invalid request error code.

NotAllowed

string

Representing the not allowed error code.

NotFound

string

Representing the not found error code.

PipelineError

string

Representing the pipeline error error code.

ServiceUnavailable

string

Representing the service unavailable error code.

TooManyRequests

string

Representing the too many requests error code.

Unauthorized

string

Representing the unauthorized error code.

UnprocessableEntity

string

Representing the unprocessable entity error code.

UnsupportedMediaType

string

Representing the unsupported media type error code.

Evaluation

Evaluation

Name Type Description
createdDateTime

string

The time-stamp when the object was created. The time stamp is encoded as ISO 8601 date and time format ("YYYY-MM-DDThh:mm:ssZ", see https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations).

customProperties

object

The custom properties of this entity. The maximum allowed key length is 64 characters, the maximum allowed value length is 256 characters and the count of allowed entries is 10.

dataset

EntityReference

EntityReference

description

string

The description of the object.

displayName

string

The display name of the object.

lastActionDateTime

string

The time-stamp when the current status was entered. The time stamp is encoded as ISO 8601 date and time format ("YYYY-MM-DDThh:mm:ssZ", see https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations).

links

EvaluationLinks

EvaluationLinks

locale

string

The locale of the contained data.

model1

EntityReference

EntityReference

model2

EntityReference

EntityReference

project

EntityReference

EntityReference

properties

EvaluationProperties

EvaluationProperties

self

string

The location of this entity.

status

Status

Status
Describe the current state of the API.

transcription1

EntityReference

EntityReference

transcription2

EntityReference

EntityReference

EvaluationLinks

Name Type Description
files

string

The location to get all files of this entity. See operation "Evaluations_ListFiles" for more details.

EvaluationProperties

EvaluationProperties

Name Type Description
correctTokenCount1

integer

The optional number of correctly recognized tokens by model1.

correctTokenCount2

integer

The optional number of correctly recognized tokens by model2.

correctWordCount1

integer

The number of correctly recognized words by model1.

correctWordCount2

integer

The number of correctly recognized words by model2.

email

string

The email address to send email notifications to in case the operation completes. The value will be removed after successfully sending the email.

error

EntityError

EntityError

sentenceCount1

integer

The number of processed sentences by model1.

sentenceCount2

integer

The number of processed sentences by model2.

sentenceErrorRate1

number

The sentence error rate of recognition with model1.

sentenceErrorRate2

number

The sentence error rate of recognition with model2.

tokenCount1

integer

Optional the number of processed tokens by model1.

tokenCount2

integer

The optional number of processed tokens by model2.

tokenDeletionCount1

integer

The optional number of recognized tokens by model1, that are deletions.

tokenDeletionCount2

integer

The optional number of recognized tokens by model2, that are deletions.

tokenErrorRate1

number

The optional token error rate of recognition with model1.

tokenErrorRate2

number

The optional token error rate of recognition with model2.

tokenErrors1

TokenErrorDetails

TokenErrorDetails

tokenErrors2

TokenErrorDetails

TokenErrorDetails

tokenInsertionCount1

integer

The optional number of recognized tokens by model1, that are insertions.

tokenInsertionCount2

integer

The optional number of recognized tokens by model2, that are insertions.

tokenSubstitutionCount1

integer

The optional number of recognized tokens by model1, that are substitutions.

tokenSubstitutionCount2

integer

The optional number of recognized tokens by model2, that are substitutions.

wordCount1

integer

The number of processed words by model1.

wordCount2

integer

The number of processed words by model2.

wordDeletionCount1

integer

The number of recognized words by model1, that are deletions.

wordDeletionCount2

integer

The number of recognized words by model2, that are deletions.

wordErrorRate1

number

The word error rate of recognition with model1.

wordErrorRate2

number

The word error rate of recognition with model2.

wordInsertionCount1

integer

The number of recognized words by model1, that are insertions.

wordInsertionCount2

integer

The number of recognized words by model2, that are insertions.

wordSubstitutionCount1

integer

The number of recognized words by model1, that are substitutions.

wordSubstitutionCount2

integer

The number of recognized words by model2, that are substitutions.

InnerError

InnerError

Name Type Description
code

DetailedErrorCode

DetailedErrorCode
Detailed error code enum.

details

object

Additional supportive details regarding the error and/or expected policies.

innerError

InnerError

InnerError
New Inner Error format which conforms to Cognitive Services API Guidelines which is available at https://microsoft.sharepoint.com/%3Aw%3A/t/CognitiveServicesPMO/EUoytcrjuJdKpeOKIK_QRC8BPtUYQpKBi8JsWyeDMRsWlQ?e=CPq8ow. This contains required properties ErrorCode, message and optional properties target, details(key value pair), inner error(this can be nested).

message

string

High level error message.

target

string

The source of the error. For example it would be "documents" or "document id" in case of invalid document.

PaginatedEvaluations

PaginatedEvaluations

Name Type Description
@nextLink

string

A link to the next set of paginated results if there are more entities available; otherwise null.

values

Evaluation[]

A list of entities limited by either the passed query parameters 'skip' and 'top' or their default values.

When iterating through a list using pagination and deleting entities in parallel, some entities will be skipped in the results. It's recommended to build a list on the client and delete after the fetching of the complete list.

Status

Status

Name Type Description
Failed

string

The long running operation has failed.

NotStarted

string

The long running operation has not yet started.

Running

string

The long running operation is currently processing.

Succeeded

string

The long running operation has successfully completed.

TokenErrorDetails

TokenErrorDetails

Name Type Description
capitalization

EditsSummary

EditsSummary

inverseTextNormalization

EditsSummary

EditsSummary

lexical

EditsSummary

EditsSummary

others

EditsSummary

EditsSummary

punctuation

EditsSummary

EditsSummary