Azure DMS to migrate MongoDB to Cosmos for MongoDB, with bulk-write error

Su, Hong [Admin] (US) (US) 10 Reputation points
2023-08-19T18:42:16.87+00:00

I am using Azure DMS to migrate MongoDB database from Azure VM to CosmosDB, after create project and migration action, I have following 2 kinds of errors :

  1. bulk-write error
    { "resourceId":"/subscriptions/8c71ef53-4473-4862-af36-bae6e40451b2/resourceGroups/azrg-use-aud-dnav-qashqade-app-tmp/providers/Microsoft.DataMigration/services/pdnavamemongotocosmosdms", "errorType":"MongoBulkWriteException1Error", "errorDetail":"MongoBulkWriteException1 - A bulk write operation resulted in one or more errors. Error=16, RetryAfterMs=0, Details='Batch write error. Error=16, RetryAfterMs=0, Details='Batch write error. Error=16, RetryAfterMs=0, Details='Batch write error. Error=16, RetryAfterMs=0, Details='Batch write error." }
  2. unique index

{ "resourceId":"/subscriptions/8c71ef53-4473-4862-af36-bae6e40451b2/resourceGroups/azrg-use-aud-dnav-qashqade-app-tmp/providers/Microsoft.DataMigration/services/pdnavamemongotocosmosdms", "errorType":"UniqueIndexNotValidWithShardKey", "warningDetail":"A unique index is not valid for the target collection because the shard key on the target is not the prefix for the index. This index will not be migrated. For CosmosDB targets, unique indexes cannot be created once the collection holds data so this collection may need to be migrated again with different indexes if this behavior is not desired." }

what I should do with setting on DMS to avoid this problems?

thank,

Hank

Azure Database Migration service

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ShaktiSingh-MSFT 14,281 Reputation points Microsoft Employee
    2023-08-21T00:48:56.63+00:00

    Hi Su, Hong [Admin] (US) (US) •,

    Welcome to Microsoft Q&A forum and thanks for using Azure Services.

    As I understand, you want to migrate MongoDB to Cosmos for MongoDB, and getting with bulk-write error.

    Could you please share configuration details of the Azure Cosmos Mongo DB API, provisioned/ serverless mode?

    Also Azure Cosmos DB provides an automatic partitioning system that only requires a shard (or a partition key). The automatic partitioning mechanism is shared across all the Azure Cosmos DB APIs and it allows for seamless data and throughout scaling through horizontal distribution.

    If using sharded collections, don't assume that your MongoDB collection shard key becomes your Azure Cosmos DB container partition key. Don't assume that your existing MongoDB data model document structure should be the same model you employ on Azure Cosmos DB.

    • Shard key is the single most important setting for optimizing the scalability and performance of Azure Cosmos DB, and data modeling is the second most important. Both of these settings are immutable and can't be changed once they're set; therefore it's highly important to optimize them in the planning phase. Follow the guidance in the Immutable decisions section for more information.

    Refer to https://video2.skills-academy.com/en-us/azure/cosmos-db/concepts-limits#per-account-limits

    and

    https://video2.skills-academy.com/en-us/azure/cosmos-db/mongodb/pre-migration-steps

    for more details.

    Thank you.

    0 comments