Projection - Operation Array Expansion

Overview

ArrayExpansion is a projection operation that makes n copies of every input attribute and marks each temporarily with an ordinal. This operation has two required properties that are StartOrdinal and EndOrdinal. Based on these two values, the operation will make copies of the input attributes starting at index StartOrdinal and going until EndOrdinal.

1. startOrdinal < 0: Round 0 of an array expansion starts at ordinal 0 or startOrdinal, whichever is larger. Ordinals less than 0 will be skipped by the array expansion.
   Ex. startOrdinal = -2, endOrdinal = 2 will result in an expansion of a[0], a[1], a[2]

2. startOrdinal > endOrdinal: No array expansion occurs. A warning is logged and the input resolved attributes just pass through.

3. startOrdinal == endOrdinal: The array expansion goes through a single round.

4. endOrdinal > maxOrdinalForArrayExpansion: maxOrdinalForArrayExpansion is a configurable value in ResolveOptions for setting the maximum ending ordinal value. The default value is 20. If endOrdinal is greater than maxOrdinalForArrayExpansion, then maxOrdinalForArrayExpansion is used instead.

Note: you can access the API reference for this operation on this link.

Note Doing an ArrayExpansion without a RenameAttributes afterwards will result in the expanded attributes being merged in the final resolved entity. This is because ArrayExpansion does not rename the attributes it expands by default. The expanded attributes end up with the same name and gets merged.
Example: We expand A to A[1], A[2], A[3], but A[1], A[2], A[3] are still named "A".

We must chain a RenameAttributes operation after an ArrayExpansion if we wish to see the expanded attributes (with the ordinal in their attribute name) in the final output. We do this by nesting projections, where the inner projection contains the ArrayExpansion and the outer projection contains the RenameAttributes.
Ex. We expand A to A[1], A[2], A[3], and then rename to {m}_{o}, to get A_1, A_2, A_3

Note: The ArrayExpansion operation is commonly used alongside the AddCountAttribute operation. Although not required, the count attribute is useful to hold the number of elements in the array that have data.

Examples

The examples below refer to the Person entity as defined here.

{
    "entityName": "Person",
    "hasAttributes": [
        {
            "name": "name",
            "dataType": "string"
        },
        {
            "name": "age",
            "dataType": "integer"
        },
        {
            "name": "address",
            "dataType": "string"
        },
        {
            "name": "phoneNumber",
            "dataType": "string"
        },
        {
            "name": "email",
            "dataType": "string"
        }
    ]
}

Using the ArrayExpansion operation on an entity attribute

If we have an entity attribute, we can use ArrayExpansion to expand each attribute.

{
    "name": "ThreeMusketeers",
    "entity": {
        "source": "Person",
        "operations": [
            {
                "$type": "arrayExpansion",
                "startOrdinal": 1,
                "endOrdinal": 3,
            }
        ]
    }
}

The resulting resolved ThreeMusketeers entity typed attribute is:

Note: Since we did not use a RenameAttributes operation after the ArrayExpansion, the expanded attributes are merged in the final resolved attributes due to having the same name.

Adding a RenameAttributes operations:

{ 
    "name": "ThreeMusketeers",
    "entity": {
        "operations": [
            {
                "$type": "renameAttributes",
                "renameFormat": "{m}_{o}"
            }
        ],
        "source": { 
            "operations": [ 
                {
                    "$type": "arrayExpansion",
                    "startOrdinal": 1,
                    "endOrdinal": 3,
                }
            ],
            "source": "Person"
        }
    } 
}

The resulting resolved ThreeMusketeers entity typed attribute is:

Using the ArrayExpansion operation when extending an entity

If we have an entity that extends another entity, we can use ArrayExpansion to expand the attributes inherited from the base entity.

Given an entity, ThreeMusketeers, that extends from the Person entity:

{
    "entityName": "ThreeMusketeers",
    "extendsEntity": {
        "operations": [
            {
                "$type": "renameAttributes",
                "renameFormat": "{m}_{o}"
            }
        ],
        "source": { 
            "operations": [ 
                {
                    "$type": "arrayExpansion",
                    "startOrdinal": 1,
                    "endOrdinal": 3,
                }
            ],
            "source": "Person"
        }
    },
    "hasAttributes": []
}

The resulting resolved ThreeMusketeers entity is:

Using multiple ArrayExpansion operations

We can nest projections if we want an ArrayExpansion operation to use another ArrayExpansion operation's output as its source.

Given the following nested projection:

{ 
    "name": "ThreePeople",
    "entity": {
        "operations": [
            {
                "$type": "renameAttributes",
                "renameFormat": "{m}_{o}"
            }
        ],
        "source": { 
            "operations": [ 
                {
                    "$type": "arrayExpansion",
                    "startOrdinal": 1,
                    "endOrdinal": 3,
                }
            ],
            "source": {
                "operations": [
                    {
                        "$type": "renameAttributes",
                         "renameFormat": "{m}_{o}"
                    }
                ],
                "source": { 
                    "operations": [ 
                        {
                            "$type": "arrayExpansion",
                            "startOrdinal": 1,
                            "endOrdinal": 3,
                        }
                    ],
                    "source": "Person"
                }
            }
        }
    } 
}

The resulting resolved ThreeMusketeers entity typed attribute is: