This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 66%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Hello guys,
I'm going nuts on this one.
I'm trying to build an Azure Cosmos DB Container with the Vector feature, I have enabled the feature of Vector Search for NoSQL API (preview).
I first tried to create the container in Azure portal and then changing the settings with no luck.
So I went with a Custom Deployment with my own template code..
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers",
"apiVersion": "2021-06-15",
"name": "[concat(parameters('cosmosDbAccountName'), '/databases/', parameters('databaseName'), '/containers/', parameters('containerName'))]",
"properties": {
"resource": {
"id": "[parameters('containerName')]",
"partitionKey": {
"paths": ["/dataSource"],
"kind": "Hash"
},
"indexingPolicy": {
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/*"
},
{
"path": "/dataSource/?"
},
{
"path": "/trademarkName/?"
},
{
"path": "/trademarkStatus/?"
},
{
"path": "/goodsAndServices/*"
},
{
"path": "/textEmbedding/?"
},
{
"path": "/imageEmbedding/?"
},
{
"path": "/id/?"
}
],
"excludedPaths": [
{
"path": "/_etag/?"
}
],
"vectorIndexes": [
{
"path": "/textEmbedding",
"type": "quantizedFlat"
},
{
"path": "/imageEmbedding",
"type": "quantizedFlat"
}
]
},
"vectorPolicy": {
"vectorEmbeddings": [
{
"path": "/imageEmbedding",
"dataType": "float32",
"distanceFunction": "cosine",
"dimensions": 768
},
{
"path": "/textEmbedding",
"dataType": "float32",
"distanceFunction": "cosine",
"dimensions": 768
}
]
}
}
}
}
],
"parameters": {
"cosmosDbAccountName": {
"type": "string"
},
"databaseName": {
"type": "string"
},
"containerName": {
"type": "string"
}
}
}
This gave me the error that the /textEmbedding.
Vector Indexing Policy's path::\\/textEmbedding not matching in Embedding's path.
So I went, okay maybe its bug.
So I went on creating the stuff in C# with this code:
// Define the connection string and create a new CosmosClient instance
var connectionString = "Your Connection String Here";
CosmosClient client = new(connectionString, new CosmosClientOptions
{
SerializerOptions = new CosmosSerializationOptions
{
IgnoreNullValues = false,
PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase,
},
AllowBulkExecution = true,
ApplicationRegion = Regions.NorthEurope
});
// Create the database if it does not exist
Database db = client.GetDatabase("cosmosomatic");
// Define the list of embeddings to be used in the container
List<Embedding> embeddings = new List<Embedding>()
{
new Embedding()
{
Path = "/imageEmbedding",
DataType = VectorDataType.Float32,
DistanceFunction = DistanceFunction.Cosine,
Dimensions = 768,
},
new Embedding()
{
Path = "/textEmbedding",
DataType = VectorDataType.Float32,
DistanceFunction = DistanceFunction.Cosine,
Dimensions = 768,
}
};
// Convert list to a Collection type
Collection<Embedding> embeddingCollection = new(embeddings);
// Define the container properties including the vector embedding policy
ContainerProperties properties = new ContainerProperties(id: "vector-container", partitionKeyPath: "/dataSource")
{
VectorEmbeddingPolicy = new(embeddingCollection),
IndexingPolicy = new IndexingPolicy()
{
VectorIndexes = new()
{
new VectorIndexPath()
{
Path = "/vector",
Type = VectorIndexType.QuantizedFlat,
}
}
},
};
// Set manual throughput options
var throughput = ThroughputProperties.CreateManualThroughput(400);
// Create the container with the specified properties and throughput
Container container = await db.CreateContainerIfNotExistsAsync(properties, throughput);
System.Console.WriteLine("Container with vector indexing created successfully.");
But then the internal Embedding class have this:
/// <summary>
**/// Represents the embedding settings for the vector index.**
**/// </summary>**
#if PREVIEW
**public**
#else
**internal**
#endif
**class Embedding : IEquatable<Embedding>**
And they haven't any preview versions in their nuget packages..
So hopefully there is someone out there who can see what I'm doing so badly wrong here.
Thanks!
Noticed that the layout in the question went bananas. Sorry for that.
@Anders Blomkvist Thank you for reaching out.
Are you creating the vector search from an existing container.?
A vector indexing policy must be on the path defined in the container's vector policy. Learn more about container vector policies. Vector indexes must also be defined at the time of Container creation and cannot be modified once created.
@Oury Ba-MSFT Hello, thanks for your response.
I have an existing database but trying to create the container from scratch.
So I tried using an deploymenttemplate but get error each time saying that: Vector Indexing Policy's path::\/textEmbedding not matching in Embedding's path.
I have enabled the feature Vector Search for NoSQL API (preview).
As you can see i trying to create a vectorPolicy and a vector indexes.
So then i thought maybe i can solve it with the .NET SDK.
But then it cant use the Embedding class because its internal. Maybe I'm missing something here.
Is there a complete example somewhere that works? With both creating the vector policies and vector indexing
Thanks again! :)
Thank you for getting back.
Here "settings" means the vector policy I presume, you first tried creating the container and then adding the policy to the container?
So, we currently don't support replace operation on container with vector policy. Hence creating container first and then adding a vector embedding policy is not supported.
The container has to be created along with the vector policy.
Also, another observation, you have passed "path": "/textEmbedding/?" which is currently not supported. Nested paths or wild cards are not supported.
Regards,
Oury
The vector embedding policy property name is “vectorEmbeddingPolicy”, not “vectorPolicy”.
Also, the vector paths shouldn’t be listed explicitly in the “includedPaths” property, however, this shouldn’t be a blocker.Regards,
Oury
@Anders Blomkvist Please don't gorget to mark as accept answer if the replies were helpful.
Regards,
Oury
Hello, thanks! You pointed me in the right directions.
I got an error that I couldn't get use shared throughput over database after I changed the typo from vectorPolicy to vectorEmbeddingPolicy :)
So now it's solved :)
Anders Blomkvist I am glad to hear that. Thank you for confirming and for accepting the answer.
Regards,
Oury