How to add a new field with a different record type using Avro in the schema registry?

@Mariia Kharkhuta - Thanks for the question and using MS Q&A platform.

Yes, you can use another record as the type field in the Avro schema. However, you need to make sure that the referenced record is already registered in the schema registry before you can use it as a type in another record.

In your example, you are trying to reference the com.example.Foo record as the type of the foo field in the com.example.Bar record. However, it seems that the com.example.Foo record is not yet registered in the schema registry, which is why you are getting the error message.

To fix this issue, you need to register the com.example.Foo record in the schema registry before you can use it as a type in the com.example.Bar record. Here's an example of how you can register the com.example.Foo record using the Azure CLI:

az eventhubs schema registry create --resource-group <resource-group-name> --namespace-name <namespace-name> --schema-group <schema-group-name> --schema-name com.example.Foo --content '{"type":"record","name":"Foo","namespace":"com.example","fields":[{"name":"foo_string","type":"string"}]}'

Once the com.example.Foo record is registered in the schema registry, you can use it as a type in the com.example.Bar record without any issues.

For more details, refer to Quickstart: Create an Azure Event Hubs schema registry using Azure portal and az eventhubs namespace schema-registry create.

Hope this helps. Do let us know if you any further queries.

Thanks for the detailed answer! Unfortunately, no success has been achieved yet. The Foo scheme is registered in the Registry (via the UI), but when trying to register the Bar scheme, the same error occurs.

However, I tried to add the schema in the way you suggested, but I got the next informing message:

mariia ~ $ az eventhubs schema registry create --resource-group my-group --namespace-name my-namespace --schema-group my-schema-group --schema-name com.example.Foo --content '{"type":"record","name":"Foo","namespace":"com.example","fields":[{"name":"foo_string","type":"string"}]}'
'schema' is misspelled or not recognized by the system.
Examples from AI knowledge base:
https://aka.ms/cli_ref
Read more about the command in reference docs

@Mariia Kharkhuta - Here's an example of how you can register the com.example.Foo record using the Azure CLI:

az eventhubs schema-registry schema create --resource-group <resource-group-name> --namespace-name <namespace-name> --schema-group <schema-group-name> --schema-name com.example.Foo --content '{"type":"record","name":"Foo","namespace":"com.example","fields":[{"name":"foo_string","type":"string"}]}'

Once the com.example.Foo record is registered in the schema registry, you can use it as a type in the com.example.Bar record without any issues. If you are still getting the same error message, please make sure that the com.example.Foo record is registered in the correct schema group and namespace.

Should I install some specific extensions for the client?

mariia ~ $ az eventhubs schema-registry schema create --resource-group my-group --namespace-name my-namespace --schema-group my-schema-group --schema-name com.example.Foo --content '{"type":"record","name":"Foo","namespace":"com.example","fields":[{"name":"foo_string","type":"string"}]}'
'schema-registry' is misspelled or not recognized by the system.

Examples from AI knowledge base:
https://aka.ms/cli_ref
Read more about the command in reference docs

Also, regarding schema group and namespace. I tried different namespaces:
e.g.
Schema Group: testSchemaGroup

Namespace: testNamespace
1.1. Add Foo schema (Success)

{
    "type": "record",
    "name": "Foo",
    "namespace": "com.example",
    "fields": [
        {
            "name": "foo_string",
            "type": "string"
        }
    ]
}

1.2. Add Bar schema (Fail)

{
    "type": "record",
    "name": "Bar",
    "namespace": "com.example",
    "fields": [
        {
            "name": "foo",
            "type": "com.example.Foo"
        }
    ]
}

And
2.1. Add another Foo (after removing the previous one) (Success)

{
    "type": "record",
    "name": "Foo",
    "namespace": "testSchemaGroup.testNamespace",     
    "fields": [
        {
            "name": "foo_string",
            "type": "string"
        }
    ]
}

2.2. Add Bar schema (Fail)

{
    "type": "record",
    "name": "Bar",
    "namespace": "testSchemaGroup.testNamespace",     
    "fields": [
        {
            "name": "foo",
            "type": "testSchemaGroup.testNamespace.Foo"         
        }     
    ] 
}

@Mariia Kharkhuta - You don't need to install any specific extensions for the Azure CLI to use the az eventhubs schema registry commands. However, you need to make sure that you have the latest version of the Azure CLI installed on your machine.

Regarding the error you encountered, it seems that you misspelled the command. The correct command to create a new schema in the schema registry using the Azure CLI is:

az eventhubs schema registry create --resource-group <resource-group-name> --namespace-name <namespace-name> --schema-group-name <schema-group-name> --schema-name <schema-name> --content <schema-content>

Make sure to replace the placeholders with the actual values for your Azure Event Hubs instance.

Regarding the schema group and namespace, you can use any valid string as the schema group and namespace names. However, you need to make sure that the schema group and namespace names are unique within your Azure Event Hubs instance.

In your example, it seems that you created a new schema group and namespace for each schema. You can use the same schema group and namespace for all your schemas if they belong to the same logical group. This can make it easier to manage and organize your schemas.

Regarding the error you encountered when adding the "Bar" schema, it seems that you need to use the fully qualified name of the "Foo" schema in the "type" field of the "foo" field in the "Bar" schema. In your example, you used "testSchemaGroup.testNamespace.Foo" as the type, which is not a valid Avro type. Instead, you should use "com.example.Foo" as the type, which is the fully qualified name of the "Foo" schema.

Here's an example of how to create the "Bar" schema with the correct type:

{
    "type": "record",
    "name": "Bar",
    "namespace": "testSchemaGroup.testNamespace",
    "fields": [
        {
            "name": "foo",
            "type": "com.example.Foo"
        }
    ]
}

Make sure to replace the <resource-group-name>, <namespace-name>, <schema-group-name>, and <schema-name> placeholders with the actual values for your Azure Event Hubs instance. If you encounter any issues, please let me know.