Unable to add new VM to LoadBalancer

Hiya,

We have a Basic SKU LB with a single front end IP and single backend-pool that has a single Windows server VM in a Vnet + NSG.

The Windows VM has no availability zone <- I think this is what's is killing me

All been running fine for half a year or so.

I want to create a new Linux VM in the same Vnet and add it to the loadbalancer.

So I created a new backend-pool.

Create the new VM with no availability and select no public IP and to add it the Vnet and new backend-pool.

I have tried many different combinations, like creating VM then trying to add to new pool, or trying to add to old pool.

Always same error:

some-vm-1 is using different Availability Set than other Virtual Machines connected to the Load Balancer(s) LoadBalancerA.

One option seems to be to delete the existing VM and recreate with an availability set, but this will be a big disruption and will it even work?

We use the loadblancer as part of firewall solution (with NSG) and to be able to access resources within the Vnet from public internet. Nothing fancy.

What am I doing wrong?