Cluster shutdown after shutting down servers on different networks

Salves 501 Reputation points
2020-08-30T04:05:36.77+00:00

Hi,

we have a cluster with three nodes, two in one subnet and another in another subnet.

when we shut down the two servers that are on the same subnet together the cluster shuts down and should remain online, as we have a witness files share.

when we turn off one of the servers that are on the same subnet, we wait and turn off the other and the third node that is on another subnet is able to keep the cluster online.

The cluster appears offline because it shows in error that the witness is unavailable, but it is available.

What could be wrong?

Thank you.

Windows Server Clustering
Windows Server Clustering
Windows Server: A family of Microsoft server operating systems that support enterprise-level management, data storage, applications, and communications.Clustering: The grouping of multiple servers in a way that allows them to appear to be a single unit to client computers on a network. Clustering is a means of increasing network capacity, providing live backup in case one of the servers fails, and improving data security.
991 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Alex Bykovskyi 2,006 Reputation points
    2020-08-30T06:00:32.783+00:00

    Hey,

    As an option, you can try running validation test to verify File Share witness connectivity. Might be an issue related to connectivity. Check the following article:
    https://video2.skills-academy.com/en-us/windows-server/failover-clustering/manage-cluster-quorum

    Alternatively, you can setup Disk Witness and test how your setup will handle it. As an example:
    https://www.starwindsoftware.com/resource-library/starwind-virtual-san-for-hyper-v-2-node-hyperconverged-scenario-with-windows-server-2016/#5f78c6edd4fbab428146c7af2d5994b6b78f7032

    Cheers,

    Alex Bykovskyi

    StarWind Software

    Note: Posts are provided “AS IS” without warranty of any kind, either expressed or implied, including but not limited to the implied warranties of merchantability and/or fitness for a particular purpose.


  2. TimCerling(ret) 1,156 Reputation points
    2020-08-30T13:41:22.18+00:00

    "when we shut down the two servers that are on the same subnet together the cluster shuts down and should remain online, as we have a witness files share.
    when we turn off one of the servers that are on the same subnet, we wait and turn off the other and the third node that is on another subnet is able to keep the cluster online.
    "

    This sounds absolutely normal. Since Windows Server 2012, clusters have the ability to have a dynamic quorum. In other words, the number of nodes required for quorum changes as the number of nodes in the cluster changes. In your first situation you have a cluster with an odd number of nodes, so the witness does not contribute a vote to the quorum because you have a sufficient of nodes to maintain quorum (two) should a single node fail. But shutting down two nodes, you are leaving a cluster with a single node and a quorum count of one, so the cluster stops.

    In your second situation, you shut down a single node and wait before shutting down the second node. What happens there is that when you shut down the single node, the witness now becomes a quorum member. Your quorum remains at two, but now the witness can provide a vote. When the second node is shut down, the witness keeps the quorum at two and the cluster continues.

    The issue is shutting down the two nodes at the same time. That does not allow the dynamic quorum to adjust for the loss of a single node. In the second case where you wait between shutting down the nodes, the witness jumps in to provide the needed quorum vote. In your first situation you created a double failure by shutting down both nodes. In your second situation, by waiting between shutting down the two nodes, you allowed the dynamic quorum capability to keep the cluster running.

    see https://video2.skills-academy.com/en-us/windows-server/failover-clustering/manage-cluster-quorum?source=docs for information on managing quorum within a cluster.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.